Index on hjalves.dev

Building an admin interface for shell2http

Tue, 07 Feb 2023 13:00:00 +0000

Introduction

Inspired by Simon Willison’s advice, I’ve decided to write about this weird dumb project that I’ve been doing since a couple of weeks ago.

In my home network, I have many remote control functions that need to be called externally. These functions are generally written and deployed as shell scripts, stuff like “powering on a computer”, “turning off a device”, “beep with a specific sound”, and so on.

Now, if I wanted to have perfect security, I should run these scripts via SSH, ideally with a user created for this purpose. But because I’m a lazy person, I want to be able to run this with minimal effort and simple authentication. shell2http is the perfect solution to do just that – it allows the execution of shell scripts via HTTP, in a way that’s very similar to the old CGI style of building web applications.

Using shell2http and a systemd service have been perfect for my own needs. But it gets annoying having to connect to the machine via SSH, changing the systemd unit file, running daemon-reload and restart everytime I want to add or remove scripts.

So I had this great (or terrible) idea to make a web administration interface where I could change the available scripts, username and password used for authentication, and restart the service with the newly configured parameters.

However, I wanted to make the easiest possible deployment, using shell2http as the “application server” and Python as the language for building the app, due to the fact that I’m very familiar with it, and it’s present by default in all major Linux distributions. To achieve these goals my constraints were the following:

It should be written in a single Python file
No Python/CSS/Javascript dependencies whatsoever

Starting from the systemd unit file

When I started making this project, my unit file was very simple, something like this:

[Unit]
Description=shell2http

[Service]
Type=simple
ExecStart=/usr/bin/shell2http -show-errors -include-stderr -form -port 9999 \
    /poweron 'scripts/poweron.sh "$v_host"' \
    /poweroff 'scripts/poweroff.sh "$v_host"' \
    /beep 'scripts/beep.sh "$v_host"'
WorkingDirectory=/home/user/
Restart=always

[Install]
WantedBy=default.target

-form is particularly useful for my needs, because it can convert GET parameters and POST form data into environment variables (ie: GET parameter is passed into the executable as an environment variable v_parameter). File uploads are also supported, see the documentation for more details.

My first change was to remove the path-script declarations from this file and put in a separated .env file. The way to do this is to add a EnvironmentFile declaration:

1
2
3

EnvironmentFile=-/home/user/s2h/s2h.env
ExecStart=/home/dk/s2h/shell2http -show-errors -include-stderr -form -port 9999 \
    /admin "s2h/s2h_admin.py" $SH_ROUTES

I’ve already put there a /admin route to serve this app. Meanwhile, the environment file will have this format:

1
2

SH_BASIC_AUTH=user:pass
SH_ROUTES=/poweron 'poweron.sh "$v_host"' /poweroff 'poweroff.sh "$v_host"'

SH_BASIC_AUTH is used natively by shell2http to add optional basic authorization. SH_ROUTES is where we’ll end up putting the path-script declarations. This way, it gets very easy to programmatically edit the configurable parameters, and it’s not necessary to call daemon-reload anymore for every little change (just restart will do it).

Building the web interface

To concentrate on functionality while minimizing design-related stress, I stole a pre-existing classless CSS library and saved it into a Python string. I chose: sakura.css.

For the base HTML structure, I’m using the relatively obscure Template() from the standard library:

from string import Template

HTML_TEMPLATE = Template(
    f"""<!doctype html>
<html lang="en">
<head>
  <meta charset="utf-8">
  <title>Admin > $title</title>
  <style>{CSS}</style>
</head>
<body>
  <nav>$navigation<hr></nav>
  <main>
    <h1>$title</h1>
    $content
  </main>
  <footer>$footer</footer>
</body>
</html>"""
)

The problem with this method, is that it gets very ugly when there’s a need to generate more dynamic HTML – template languages were invented for a reason!

For these pages, I’ve decided to use a nicer (but very slow) way of building HTML pages, by using the standard library ElementTree, which can be used to build HTML/XHTML documents, and more generally, XML documents.

To simplify and make the page-building process more intuitive and HTML-like, I created wrapper classes called Html and HtmlElement that enables writing a document as follows:

h.section(
    h.h1("Title"),
    h.p("Content"),
    h.form(method="post")(
        h.input(type="submit", value="OK")
    )
)

It looks nice and beautiful, but I don’t recommend doing this for serious projects. If you really want to explore this approach, take a look at the projects pyxl and mixt. They’re modifying the Python language (with monkey patching and other cool ways) by allowing the use of HTML-like syntax natively, akin to JSX.

I think this model of building web applications has so much unexplored potential, combined with libraries like Tailwind CSS and htmx, now you just need a single language to write pages and components.

Making a development server

People generally know that Python has a built-in simple static/CGI web server, python -m http.server, which is a very handy tool. What many don’t know is that it’s really easy to build your own custom development web server on top of the standard library wsgiref module, by implementing the WSGI protocol in your app.

I say development server, because when writing for production, there’s many edge cases that are quite hard to get right, that’s why micro-frameworks like Flask and Bottle exist in the first place.

I think this feature is quite nice, because this way you don’t need shell2http just to see if the pages are being rendered correctly or if the forms are behaving as expected.

This is the full web-server implementation:

def wsgi(environ, start_response):
    try:
        if not environ.get("PATH_INFO") == ADMIN_ROUTE:
            raise LookupError(f"Not found. Use {ADMIN_ROUTE} path only.")

        variables = dict(parse_qsl(environ.get("QUERY_STRING", "")))
        method = environ["REQUEST_METHOD"]
        content_type = environ.get("CONTENT_TYPE")

        # Handle forms (not handling multipart/form-data as we don't have uploads)
        if method == "POST" and content_type == "application/x-www-form-urlencoded":
            payload = environ['wsgi.input'].read(int(environ["CONTENT_LENGTH"]))
            variables.update(parse_qsl(payload.decode()))

        status = "200 OK"
        headers = [("Content-Type", "text/html")]
        body = render(variables).encode()

    except Exception as ex:
        status = "500 Error"
        headers = [("Content-Type", "text/plain")]
        body = f"{ex.__class__.__name__}: {str(ex)}".encode()

    start_response(status, headers)
    return [body]

render(variables) is where all the logic/routing/rendering is done. To run this wsgi() function in the Python’s builtin WSGI server, forever, just call:

1
2

from wsgiref.simple_server import make_server, WSGIServer
make_server("localhost", 9998, wsgi, WSGIServer).serve_forever()

This is a single-threaded webserver, but you can customize it to be threaded or to use a thread-pool, like this:

class ThreadedWSGIServer(ThreadingMixIn, WSGIServer):
    pass

class ThreadPoolWSGIServer(ThreadingMixIn, WSGIServer):
    pool = ThreadPool(4)

    def process_request(self, request, client_address):
        self.pool.apply_async(
            self.process_request_thread, args=(request, client_address)
        )

How neat! I could use this solution to fully replace shell2http, but I’m not inclined to, because I trust way more that project than my own code :)

Some screenshots

Here is some screenshots of the app. It is kinda ugly at the moment, but it is what it is…

Next steps

To make this project perfect for me, it should automatically download and install shell2http binary when missing, create a systemd unit file, and update itself from the repository. The look and feel could also be improved, I guess.

Conclusion

I hope you found anything useful or at least interesting here. I’m not recommending this project or shell2http to anyone, this all feels like a big security nightmare, but if you’re really inclined to use this, at least run it behind a reverse proxy with a better authentication mechanism :)

If you want to take a look at the full code, this is the repository: https://github.com/hjalves/s2h-admin.

The code is not that interesting apart from the stuff I mentioned here, and it’s very messy, but I appreciate if you have any suggestion on how to improve it.

Thanks for reading 😊

Interfacing with SerenityOS from Python

Sat, 07 Jan 2023 17:58:50 +0000

As my first post related to SerenityOS development, I’m going to talk about a topic which I’m really passionate about – Python programming!

SerenityOS has a pretty decent port of the Python language, mainly due to the work done by Linus (and other contributors!). Almost every standard library module works as intended, with many of its optional dependencies also ported, like sqlite, zlib, ncurses. Fun fact: webbrowser module has official support for launching the Serenity’s Ladybird Browser.

One of such optional dependencies is libffi, which is a foreign-function interface library. Having this dependency makes the ctypes module available.

With ctypes, we can easily call C compatible functions exported by a shared library, which means it should be possible to interface with the Serenity C++ classes, providing that we write wrapper functions in the style of C, and use the extern "C" declaration.

Let’s build a small “hello world” C-reni-Py (😅) program!

Creating a shared library

To create a shared library the easiest way possible, I’m going to use the same structure as the other libraries, by creating a new directory in the Userland/Libraries. Let’s call it LibExample:

1
2
3

cd serenity
mkdir -p Userland/Libraries/LibExample && cd $_
touch CMakeLists.txt Example.cpp Example.h

We’re going to build a really simple one-function library that sends (push-like) desktop notifications!

/* Example.h */
#pragma once

extern "C" {
    int notify(char const* title, char const* text);
}

/* Example.cpp */
#include "Example.h"
#include <LibGUI/Notification.h>

int notify(char const* title, char const* text)
{
    auto notification = GUI::Notification::construct();
    notification->set_title(title);
    notification->set_text(text);
    notification->show();
    return 0;
}

Nice! Now, to make the build system compile this code, it’s necessary to edit CMakeLists.txt:

serenity_component(LibExample TARGETS LibExample)
set(SOURCES Example.cpp)
serenity_lib(LibExample example)
target_link_libraries(LibExample PRIVATE LibCore LibGUI LibGfx)

I’m not sure if this declaration is entirely correct (as I don’t fully understand the build system), but I found that serenity_component() must be present. Otherwise, the library won’t be placed in the OS filesystem (I assume it’s because it’s not a dependency of any application).

Now, edit Userland/Libraries/CMakeLists.txt and add an add_subdirectory(LibExample) declaration. Build the system and run it. Check that there is a /usr/lib/libexample.so file!

Using the library from Python

Assuming that you already have the Python port installed, let’s use the library. Open a Python interpreter and type:

1
2
3

from ctypes import CDLL
lib = CDLL("libexample.so")
lib.notify(b"Well", b"Hello friends")

And…

It crashed! (At least the error is pretty cool)

Apparently, before we can create notifications, we need to have an event loop instantiated, which in Serenity it’s done by initializing a GUI::Application object. I’m not entirely sure why, I think it’s related to the way GUI applications communicate with WindowServer.

After declaring a global variable static RefPtr<GUI::Application> app, I’m going to initialize it in the first invocation of notify(). Something like this:

1
2
3

if (!app) {
    app = GUI::Application::construct(Main::Arguments {});
}

And there we go! Displaying a notification on the SerenityOS desktop, via Python:

(I also introduced an icon file as a third parameter, so I could find a reason to show the wholesome catdog mascot in this post.)

With this approach, it’s theoretically possible to build Python bindings for LibGUI (and other components), but it’s really hard work to do that, I think. Another way to build this kind of integration is to use the CPython C API.

That’s all for today. I hope you find this post interesting in any way. x)

SerenityOS

Mon, 06 Jun 2022 17:00:00 +0100

SerenityOS is a POSIX compliant system with a retro-style UI (heavily inspired in Windows 2000), written in modern C++ and a custom, friendlier alternative to the STL (the C++ stdlib equivalent). It was started in 2018 by Andreas Kling, and since then, it just kept growing and amassing a vibrant community of developers and enthusiasts.

There is an ongoing effort to make a custom programming language to (hopefully) replace C++ for developing userland applications (and maybe also the kernel?), but it’s still in early development.

Every piece of software from the kernel to the userland applications live in the same repository (monorepo approach). And one of the core philosophies is to implement every component of the system ‘in home’ - drivers, protocols and tools. It even has a browser built completely from scratch (which is quite insane!).

Although it tries to not depend on third-party software and libraries, it still gives you the choice to install many ports of popular software, like Python 3 or Quake.

Since discovering the project last year, I started keeping track of its progress by watching many of the videos posted by the creator of the system in his YouTube channel (especially the monthly updates and the weekly Q&A on Fridays afternoons).

But after lurking around for many months, I finally decided to contribute, even thought I don’t really know how to code in C++. I have a small list of things I would like to fix and improve.

I will try to post regular updates describing my journey and experiences into the system and documenting the progress of the projects I have in mind.

New Blog

Sun, 05 Jun 2022 16:41:03 +0100

Hello! This is my new blog. It uses the Hugo static site generator, and a theme inspired in an older version of Tania Rascia’s website available here: https://github.com/WingLim/hugo-tania.

My previous blog was based on Pelican, and although I really loved the fact that it was written in Python, I needed to try something new to keep me interested. I only had a couple of really old blog posts, written in Portuguese. I feel that they’re not that relevant and useful anymore, so I decided to simply delete them.

I’m going to try to write exclusively in English this time, because even though I’m terrible at expressing myself in English (and in general), I hope I can improve my writing skills while doing so.

Hope there’s any relevant or interesting content to you, my dear reader! 😃