Planet GNOME

I made another weird side project while unemployed. In fact I’ve wanted it for a while but once I learned that “Rebonjour” is the word for “hello again” I just had to finish the library.

librebonjour is an asynchronous DNS-SD and mDNS client library for GLib applications. Or, more practically, it is a small GObject API over the two local service-discovery providers you are likely to find on a Linux system: Avahi and systemd-resolved.

It does not link against either of them. It only talks to them over D-Bus.

The reason for that is mostly boring, which is usually where the useful things are. Applications should not need to care if a machine has Avahi running, or if it is using systemd-resolved for mDNS. They should be able to discover a service, resolve it, maybe advertise something, and get on with whatever they were actually trying to do.

So RebonjourClient selects a backend internally. If org.freedesktop.Avahi is available on the system bus, it uses Avahi. If not, it falls back to systemd-resolved’s org.freedesktop.resolve1 API. If neither is around, availability checks fail like you would expect.

The public API stays the same either way.

What It Does

There are three common things I wanted to make pleasant.

First, one-shot discovery. Ask for the service types in local, ask for instances of something like _ipp._tcp, then resolve one of those instances into addresses and TXT metadata.

Second, browser-style discovery. A RebonjourBrowser owns a stable GListModel of RebonjourService objects. That fits nicely into GTK code because the model object can stay the same while the contents change underneath it.

Third, registration. You can describe a local service with RebonjourServiceDescription, register it, and keep the returned RebonjourRegistration alive for as long as the service should be advertised.

Resolving a service gives you a RebonjourResolvedService. That contains the SRV result, TXT data, priority, weight, and a model of RebonjourEndpoint objects. The endpoints hold the GSocketAddress you would actually use to connect.

Why Two Backends

Avahi is the nicer backend for browsing. Its D-Bus API gives you long-lived browser objects and emits signals when services appear and disappear. That maps very naturally to GListModel changes.

systemd-resolved is different. It has useful DNS-SD and mDNS operations over D-Bus, but the browsing side is lookup-based. That means you can ask what is there, but you do not get the same live add/remove signal stream that Avahi provides.

I did not want applications to have to care about that distinction unless they really want to. So the browser has auto-refresh and refresh-interval properties. With Avahi, auto-refresh is effectively harmless because the model is already live. With systemd-resolved, it starts an internal refresh loop and updates the model for you.

It is not magic. It is just putting the backend-shaped unpleasantness in one place so application code can stay boring.

Asynchronous with libdex

The whole thing is built on libdex. Anything that might touch D-Bus or the network returns a DexFuture.

That means construction, availability checks, service-type lookup, instance lookup, resolving, registration, browser refresh, and unregistering are all future-based. If you are already writing fiber-style code with libdex, the API fits into that directly:

g_autoptr(RebonjourClient) client = NULL; g_autoptr(GListModel) services = NULL; g_autoptr(GError) error = NULL; if (!(client = dex_await_object (rebonjour_client_new (), &error))) g_error ("%s", error->message); services = dex_await_object (rebonjour_client_lookup_instances (client, 0, "_ipp._tcp", NULL, REBONJOUR_LOOKUP_FLAGS_NONE), &error);

The 0 there means any interface. Passing NULL for the domain uses local. The common case should not require looking up interface indexes which I’m pretty sure most people reading this have never even done before.

Advertising

Advertising is where things get more system-policy-oriented.

With Avahi, registration goes through Avahi’s D-Bus API. With systemd-resolved, registration uses RegisterService and UnregisterService, which are polkit-protected. Also, resolved needs full mDNS enabled with MulticastDNS=yes; MulticastDNS=resolve is enough to browse and resolve, but not enough to respond as a service.

So librebonjour can expose one API for registration, but it cannot make host policy disappear. Applications still need to handle authorization failure, missing mDNS responder support, sandbox boundaries, or whatever policy the system administrator has decided is appropriate.

That seems like the right way to demarcate things. The library should hide the provider mechanics, not the permissions of the platform.

Why

Mostly because I wanted this to exist.

DNS-SD is handy. Local-network service discovery is still useful. But using it from a GLib application means either caring too much about the provider or writing just enough glue that every application gets to have its own slightly different version of the same code.

And even worse is having to bundle things to build projects like Avahi for Flatpak when you only use the library which calls into D-Bus anyway.

This is not a grand platform initiative. It is not something I am employed to maintain. So you know, use wisely.

• Early interview & job offer for a support role, catch up with Anna, sync with Dave, catch up with a partner, 1:1 with Naomi.
• Sync with Thorsten, Skyler, Pedro, poked at slides.
• Dave, Emily, Jake, Pete & Eli arrived - played in the garden somewhat, enjoyed company, tried to cook a meal; caught up happily after a decade+ gap. Bed early.

I have a server at home. It runs a Kubernetes cluster and a few services. I want to expose them to the Internet, so I can e.g. share public links from my Nextcloud, or synchronize my Kobo reader with Grimmory. But I don't want to expose my home IP to the world, and I want to have some reasonable protection against unsophisticated DoS attacks.

I realized that I can achieve that with a cheap VPS that acts as a front, HAProxy, and Wireguard.

I rented a tiny VPS for €4/month at Infrawire (1 vCPU, 2 GB RAM, 25 GB NVMe). I installed a Debian 13 on it, because I want that front server to be as stable and low maintenance as possible, and installed the Debian-packaged HAProxy onto it. I also installed Wireguard. The VPS has a publicly accessible IP, so it will be my Wireguard server: my server at home can reach the VPS to establish a tunnel, the opposite is not true.

On my k3s node, I've installed Wireguard as well. I configured Wireguard on the VPS and my k3s node to establish a tunnel between the two. I've also bound the sshd on my VPS to the wireguard address. Infrawire offers a console so I can unstick myself if I locked me out of my own server (e.g. by misconfiguring Wireguard on any side, or if my server at home had any failure).

I pointed all my DNS records to the VPS. The HAProxy is a "dumb" tcp forwarder, so I can keep operating like before on my cluster. In particular, HAProxy doesn't do TLS termination. My certificates are fetched on my cluster by cert-manager like before, using the http-01 challenge and Let's Encrypt. I could also move to dns-01 challenges, but http-01 just works and lets me switch to a registrar without an API if need be.

That way, I don't need a fixed IP at home, and I don't have to do any port-forwarding from my home router to my k3s cluster. Even better: the VPS has an anti-DDoS protection included, and I can also configure HAProxy to refuse too many connections from a same IP, I can make it close TCP connections that take too long to establish, and more. If my VPS gets hammered, I can still access my services from within my home network.

It's probably not your fault.

On a cache miss, there are two things a reverse proxy (which Fastly is to us) can do. It can make the client wait until the proxy itself fetches the requested content and then serve it, with subsequent requests being served from the cache. From a user's perspective, it means staring at "hung" process, and people tend not to be understanding when a program is stuck seemingly doing nothing.

Instead, the proxy can stream the response from the origin, caching it at the end. This makes the client receive the data right away, although it's not without drawbacks.

In a streaming setup like Flathub's, an all-MISS path adds some upstream latency before the first byte, but also limits the download speed to what the slowest link can deliver. As we don't run servers in the same datacenter or on a single backbone network, the hop from Fastly through the caching proxy to the master server incurs a penalty that may affect how quickly the data gets back.

In order to cache files larger than 20MB, Fastly expects customers who use streaming misses to use segmented caching. Anything larger than that gets broken down into smaller chunks. When Fastly wants the data from us, it will add a Range header specifying which bytes we should respond with. Fastly will then serve the request after reconstructing the file from various chunks. Our caching proxies also use the value of the Range header in the caching key to avoid requesting the full file over and over again from the master server as well.

While great for caching, many concurrent range MISSes can turn what would be a sequential file read into scattered, random reads. It wouldn't matter with SSD or NVMe, but as the repository is stored on HDDs, when combined with streaming misses, it can turn cold transfer speed into min(network bottleneck, ZFS random-read bottleneck).

Counterintuitively, you may improve your download speeds by aborting the ongoing Flatpak operation and starting it again. While the initial request was slow, there's a non-zero chance it went through all the caching layers and it will become a cache hit in the meantime.

Flatpak

Let's talk Flatpak. When installing or upgrading applications, Flatpak will try to use delta files. A typical delta is an update file that contains only the difference between versions. There are also from-scratch deltas, which effectively are an archive with all files required to install an app from scratch, thus the name.

Flathub generates a single upgrade delta and a from-scratch delta for the latest version. Delta generation is an expensive process in terms of disk reads and writes, but also disk space. Because our ZFS setup isn't exactly the fastest, generating more delta files also affects how quickly we can publish an update. Yes, in theory we could be doing this out of band but we don't. In hindsight, Titanic wasn't unsinkable after all.

What happens if you are not updating often enough? A lot of suffering. Flatpak will download each missing file between the version you are on and the one you want to upgrade to, separately. This is an almost certain cache miss causing even more random seeks on the master server. At this point Flatpak would be better off downloading the from-scratch delta but it can't. The behaviour is controlled by OSTree, which doesn't offer any knobs to affect it. It is the right choice if the goal is to limit the bandwidth used by the client to fetch updates, but an incredibly bad one for anyone on a reliable connection; downloading a single large file is almost always faster than fetching multiple smaller ones.

What do? Some brave soul could fix OSTree to apply a better heuristic on when to use from-scratch deltas for upgrades, or at least make it expose an API that lets Flatpak choose. For the rest of us mere mortals, we can only update regularly or wait patiently for the update to finish.

I often write about how when stuff works well, you take it for granted.

It’s true for technology: when’s the last time you hit a compiler bug in GCC? Once upon a time these were a common thing and you had to choose your C compiler wisely. Yet I haven’t recently seen an article that says “GCC is going great” .

It’s true for people too. When someone does an excellent job maintaining an open source project then, they do occasionally get some gratitude, but — if you do a bad job, it’s amazing how quickly the negative comments pile up in the issue tracker, many of which taking subtle or not-so-subtle digs at the project owners. Maybe we created this situation for ourselves by having a prominent “report issue” button but no corresponding “send flowers to the maintainer” button.

On that note, a hat tip to Carlos Garnacho for all his work on the Localsearch extractor sandbox which recently got a shout out its “extremely strong” design.

(It’s worth noting that Localsearch also stopped using GStreamer to parse media files altogether, which the discussion in that thread missed. We love GStreamer but it isn’t the right tool for metadata scanning. The 3.9 and 3.10 series use libav/ffmpeg instead, but given that US software patent laws make it tricky for USA folk to distribute that, the plan is to move to using MediaInfoLib)

Fairphone 5

It’s coming up to two years since I switched to a Fairphone 5. The real proof of this device will be in 2033 when I manage ten years of using the same phone.

Meanwhile, I recently had some issues with it not charging via the USB-C port. I thought it might be a bit tricky to fix, but it really is easy: buy the replacement part (about 20€), take off the back cover, remove a few small screws and switch over the whole USB port + speaker unit.

I hear some fellow Android users complaining about Alphabet/Google’s intrusive AI integration. Apparently the power button is now the AI button? I use the stock Android, and I know vendors have their hands tied somewhat by Alphabet/Google, so its worth noting that disabling the AI integration on the Fairphone 5 is a single config setting.

I’d be interested to know more about the kernel version as it is old as hell. I guess this is a vendor/Android thing, and hopefully most of the many known vulnerabilities in this old version of Linux are mitigated by sandboxing higher up in Android. If you’re a high risk cybercrime target then I would definitely not recommend using the vendor Android OS on this device. (Probably best to avoid Android altogether if this is your situation!)

So its not perfect, but I just wanted to shout out again that there are some good people doing good work here. If only all smartphones were built like this one.

Korg Minilogue XD

One reason I’m not writing much about open source software is that I’m spending a lot of my time outside work making music in various guises, these days mainly as part of soon to be huge Galician disco revival group Muaré. This band needs a website, so in future I don’t have to link you to Instagram, but you know how the world is at the moment. We do at least have a Bandcamp page.

When it comes to music gear, I seem to be a Yamaha guy. It’s amazing actually that the same company that made my trombone also makes excellent digital pianos, and if and when I need a motorbike, Yamaha also sells those.

When it comes to synths though I’ve been really enjoying the Korg Minilogue XD. It’s cheap, built like a tank and its ten years old so there are plenty of second hand models around. It’s not fucking Behringer (please don’t give money to Behringer). It’s simple and sounds great.

But most impressively, it support plugins via a freely available SDK. You can develop your own custom digital oscillators and effects for this thing and deploy them over USB. Of all major pro audio manufacturers, Korg are the only company I know to support this. So even though the hardware is now 10 years old, it can still learn new tricks, and there is an active scene of both free and commercial plugins for the platform. Perhaps the most active commercial outfit is Sinevibes. There is, of course, reddit. The SDK is not truly open source (and few things in pro audio ever are) but it’s free from any licensing fees, and the whole thing is sat here in a Git repo. Pretty good.

If I’d had more time to prepare I might have a video here of some cool Minilogue XD tunes I made. But I guess you’ll have to wait til next month for that. Until then!

I’ve been putting together varlink-glib, which is a library for writing Varlink clients and services in C. The basic idea is to keep the transport policy out of the library. You get a connected GIOStream however you want, whether that is GLib networking, socket activation, or something more specialized, and then wrap it in a VarlinkClientProtocol or VarlinkServerProtocol.

The API is built around DexFuture, which makes the async parts feel a lot nicer in C than the usual callback layering. Client calls return a future, server replies return a future, and internally the protocol can use fibers for the work that wants to look sequential while still integrating with the GLib main context. This is very much the style of API I want for system services: explicit enough that you can debug it, but not so painfully manual that every call site becomes a state machine.

future = example_calc_add_call_invoke (proxy, call, VARLINK_METHOD_CALL_DEFAULT, add_reply_cb, NULL, NULL);

There is also a varlink-codegen tool which takes a .varlink interface and generates typed C wrappers for it. That gives you proxy objects, server skeletons, call objects, reply objects, and error constants instead of making every application hand-roll JSON. You can still drop down to VarlinkMessage, VarlinkMessageBuilder, and VarlinkMessageReader for forwarding or weird infrastructure cases, but most code should get to stay typed.

File descriptor passing works when the transport is a Unix socket connection. This follows the same general model as systemd’s Varlink support: the JSON payload contains an integer index, while the actual descriptors are sent out-of-band with SCM_RIGHTS and attached to the containing message as a GUnixFDList. Generated code can continue to treat the field as an integer, while the actual descriptor list stays attached to the underlying VarlinkMessage.

fd_list = g_unix_fd_list_new (); fd_index = g_unix_fd_list_append (fd_list, fd, &error); varlink_message_set_unix_fd_list (parameters, fd_list);

Protocol upgrades are supported too. A method call can ask to upgrade the connection, and once the final successful reply is sent, Varlink stops being valid on that connection. The VarlinkProtocol is still a GIOStream, so the next protocol can continue reading and writing through the same object. That keeps the handoff explicit without requiring a separate transport abstraction.

I also wired in optional Sysprof capture support. When enabled, client and server RPCs can show up as Sysprof marks with useful bits like method name, result, reply count, one-way, multi-reply, upgrade, Varlink error, and GError details. That matters because once you have concurrent calls, generated dispatch through VarlinkServerRegistry, and services doing real work, “it got slow somewhere” is not enough information.

There is still more polish to do, but the shape is there: typed generated APIs for normal users, low-level message APIs for infrastructure, DexFuture for async flow, Unix FD passing for the system service cases, protocol upgrades for handoff cases, and profiler hooks so it can be debugged when the happy path stops being happy.

Numbers

So what does that look like performance wise you might ask? For a very simple Echo interface in both D-Bus and Varlink you can get a rough estimate. No daemons, just serialization on a socketpair(). I haven’t started performance tuning yet so there may be ground to make up on both sides. But the answer is that the testcase for varlink-glib is about 5x faster than the testcase for GDBusConnection in either synchronous or asynchronous modes.

This doesn’t apply to all use-cases of D-Bus of course. But for a specific case I use it for (P2P IPC between peer processes), it is pretty big difference.

A small personal note: as I wrote in my recent update from France, I am no longer employed by Red Hat. Work like this is currently self-funded, out of pocket, while my family and I settle into a new chapter. If you find it useful, a note of encouragement or a contribution means a lot right now. It helps make it possible to keep improving the free software infrastructure many of us rely on.

Some more great news: I’m pleased to announce that HP has also agreed to be premier sponsor for the Linux Vendor Firmware Service (LVFS) as part of our sustainability effort.

With the industry support from HP (and our existing sponsors of Lenovo, Dell, Framework, OSFF and of course Linux Foundation and Red Hat) we can turbo-charge the growth of the LVFS even more. Thanks!