Planet GNOME

The GNOME Foundation invites bids for hosting GUADEC 2025. For next year’s annual gathering of GNOME users and developers, we’re looking for location proposals from anywhere in the world. This is your chance to bring GUADEC to your city!

If you are interested in submitting a proposal to host GUADEC 2025 in your city, please submit an intention to bid by filling out this short form. Intent to Bid submissions are due by end-of-day May 3, 2024.

Final proposals should be submitted using our online form and are due by May 31, 2024.

Be prepared to include the following details in your proposal:

• Location information
• A list of local team members
• Information about local support (GNOME or open source communities, universities, government or businesses affiliated with GNOME or open source)
• Ideas about local sponsors
• Venue information
• Proposed dates
• Travel information (local airports, trains, etc.)
• Local transportation (how will attendees get around the city)
• Accommodation and dining options
• Social event options
• Group activity options
• Estimated costs and budget for hosting in this location

Deadlines
Intention to bid: May 3, 2024
Final proposal submission: May 31, 2024

Proposals from previous years are available for reference and you can talk to the Foundation Board and previous GUADEC organizers to find out more about what is involved. If you need more time to finalize your bid please contact kprogri@gnome.org.

Organizing GUADEC is a large undertaking but local teams will have help from Foundation Staff members and Engagement Team contributors!

If you have any questions, don’t hesitate to contact Kristi at kprogri@gnome.org or the Foundation board at board@gnome.org.

Submit an Intent to Bid
Submit a Proposal

For the last few months, Benjamin Tissoires and I have been working on and polishing a little tool called udev-hid-bpf [1]. This is the scaffolding required quickly and easily write, test and eventually fix your HID input devices (mouse, keyboard, etc.) via a BPF program instead of a full-blown custom kernel driver or a semi-full-blown kernel patch. To understand how it works, you need to know two things: HID and BPF [2].

Why BPF for HID?

HID is the Human Interface Device standard and the most common way input devices communicate with the host (HID over USB, HID over Bluetooth, etc.). It has two core components: the "report descriptor" and "reports", both of which are byte arrays. The report descriptor is a fixed burnt-in-ROM byte array that (in rather convoluted terms) tells us what we'll find in the reports. Things like "bits 16 through to 24 is the delta x coordinate" or "bit 5 is the binary button state for button 3 in degrees celcius". The reports themselves are sent at (usually) regular intervals and contain the data in the described format, as the devices perceives reality. If you're interested in more details, see Understanding HID report descriptors.

BPF or more correctly eBPF is a Linux kernel technology to write programs in a subset of C, compile it and load it into the kernel. The magic thing here is that the kernel will verify it, so once loaded, the program is "safe". And because it's safe it can be run in kernel space which means it's fast. eBPF was originally written for network packet filters but as of kernel v6.3 and thanks to Benjamin, we have BPF in the HID subsystem. HID actually lends itself really well to BPF because, well, we have a byte array and to fix our devices we need to do complicated things like "toggle that bit to zero" or "swap those two values".

If we want to fix our devices we usually need to do one of two things: fix the report descriptor to enable/disable/change some of the values the device pretends to support. For example, we can say we support 5 buttons instead of the supposed 8. Or we need to fix the report by e.g. inverting the y value for the device. This can be done in a custom kernel driver but a HID BPF program is quite a lot more convenient.

HID-BPF programs

For illustration purposes, here's the example program to flip the y coordinate. HID BPF programs are usually device specific, we need to know that the e.g. the y coordinate is 16 bits and sits in bytes 3 and 4 (little endian):

SEC("fmod_ret/hid_bpf_device_event") int BPF_PROG(hid_y_event, struct hid_bpf_ctx *hctx) { s16 y; __u8 *data = hid_bpf_get_data(hctx, 0 /* offset */, 9 /* size */); if (!data) return 0; /* EPERM check */ y = data[3] | (data[4] << 8); y = -y; data[3] = y & 0xFF; data[4] = (y >> 8) & 0xFF; return 0; } That's it. HID-BPF is invoked before the kernel handles the HID report/report descriptor so to the kernel the modified report looks as if it came from the device.

As said above, this is device specific because where the coordinates is in the report depends on the device (the report descriptor will tell us). In this example we want to ensure the BPF program is only loaded for our device (vid/pid of 04d9/a09f), and for extra safety we also double-check that the report descriptor matches.

// The bpf.o will only be loaded for devices in this list HID_BPF_CONFIG( HID_DEVICE(BUS_USB, HID_GROUP_GENERIC, 0x04D9, 0xA09F) ); SEC("syscall") int probe(struct hid_bpf_probe_args *ctx) { /* * The device exports 3 interfaces. * The mouse interface has a report descriptor of length 71. * So if report descriptor size is not 71, mark as -EINVAL */ ctx->retval = ctx->rdesc_size != 71; if (ctx->retval) ctx->retval = -EINVAL; return 0; } Obviously the check in probe() can be as complicated as you want.

This is pretty much it, the full working program only has a few extra includes and boilerplate. So it mostly comes down to compiling and running it, and this is where udev-hid-bpf comes in.

udev-hid-bpf as loader

udev-hid-bpf is a tool to make the development and testing of HID BPF programs simple, and collect HID BPF programs. You basically run meson compile and meson install and voila, whatever BPF program applies to your devices will be auto-loaded next time you plug those in. If you just want to test a single bpf.o file you can udev-hid-bpf install /path/to/foo.bpf.o and it will install the required udev rule for it to get loaded whenever the device is plugged in. If you don't know how to compile, you can grab a tarball from our CI and test the pre-compiled bpf.o. Hooray, even simpler.

udev-hid-bpf is written in Rust but you don't need to know Rust, it's just the scaffolding. The BPF programs are all in C. Rust just gives us a relatively easy way to provide a static binary that will work on most tester's machines.

The documentation for udev-hid-bpf is here. So if you have a device that needs a hardware quirk or just has an annoying behaviour that you always wanted to fix, well, now's the time. Fixing your device has never been easier! [3].

[1] Yes, the name is meh but you're welcome to come up with a better one and go back in time to suggest it a few months ago.
[2] Because I'm lazy the terms eBPF and BPF will be used interchangeably in this article. Because the difference doesn't really matter in this context, it's all eBPF anyway but nobody has the time to type that extra "e".
[3] Citation needed

We first introduced support for dmabufs and graphics offload last fall, and it is included in GTK 4.14. Since then, some improvements have happened, so it is time for an update.

Improvements down the stack

The GStreamer 1.24 release has improved support for explicit modifiers, and the GStreamer media backend in GTK has been updated to request dmabufs from GStreamer.

Another thing that happens on the GStreamer side is that dmabufs sometimes come with padding: in that case GStreamer will give us a buffer with a viewport and expect us to only show that part of the buffer. This is sometimes necessary to accommodate stride and size requirements of hardware decoders.

GTK 4.14 supports this when offloading, and only shows the part of the dmabuf indicated by the viewport.

Improvements inside GTK

We’ve merged new GSK renderers for GTK 4.14. The new renderers support dmabufs in the same way as the old gl renderer. In addition, the new Vulkan renderer produces dmabufs when rendering to a texture.

In GTK 4.16, the GtkGLArea widget will also provide dmabuf textures if it can, so you can put it in a GtkGraphicsOffload widget to send its output directly to the compositor.

You can see this in action in the shadertoy demo in gtk4-demo in git main.

Shadertoy demo with golden outline around offloaded graphics Improved compositor interaction

One nice thing about graphics offload is that the compositor may be able to pass the dmabuf to the KMS apis of the kernel without any extra copies or compositing. This is known as direct scanout and it helps reduce power consumption since large parts of the GPU aren’t used.

The compositor can only do this if the dmabuf is attached to a fullscreen surface and has the right dimensions to cover it fully. If it does not cover it fully, the compositor needs some assurance that it is ok to leave the outside parts black.

One way for clients to provide that assurance is to attach a specially constructed black buffer to a surface below the one that has the dmabuf attached. GSK will do this now if it finds black color node in the rendernode tree, and the GtkGraphicsOffload widget will put that color there if you set the “black-background” property. This should greatly increase the chances that you can enjoy the benefits of direct scanout when playing fullscreen video.

Offloaded content with fullscreen black background

In implementing this for GTK 4.16, we found some issues with mutter’s support for single-pixel buffers, but these have been fixed quickly.

To see graphics offload and direct scanout in action in a GTK4 video player, you can try the Light Video Player.

If you want to find out if graphics offload works on your system or debug why it doesn’t, this recent post by Benjamin is very helpful.

Summary

GTK 4 continues to improve for efficient video playback and drives improvements in this area up and down the stack.

A big thank you for pushing all of this forward goes to Robert Mader.

Perhaps the most interesting feature is that this new version reduces the number of external dependencies by almost 15%. In other words the number of deps went from 7 to 6. This is due to Apple Clang finally shipping with std::format support so fmt::format could be removed. The actual change was pretty much a search & replace from fmt::format to std::format. Nice.

Other features include:

• L*a*b* color support in paint operations
• Reworked raster image APIs
• Kerned text support
• Support for all PDF/X versions, not just 3
• Better outline support

But most importantly, there are now stickers:

Sadly you can't actually buy them anywhere, they can only be obtained by meeting me in person and asking for one.

In which I meet QA testers, bang my head against the GNOME OS initial setup process, and travel overland from Scotland to Spain in 48 hours.

Linux QA meetup

Several companies and communities work on QA testing for Linux distros, and we mostly don’t talk to each other. GUADEC 2023 was a rare occasion where several of us were physically collocated for a short time, and a few folk proposed a “GNOME + openQA hackfest” to try and consolidate what we’re all working on.

Over time, we realized ongoing lines of communication are more useful than an expensive one-off meetup, and the idea turned into a recurring monthly call. This month we finally held the first call. In terms of connecting different teams it was a success – we had folk from Canonical/Ubuntu, Codethink, Debian, GNOME, Red Hat/Fedora and SUSE, and there are some additional people already interested in the next one. Everyone who attended this round is using openQA and we will to use the openqa:opensuse.org chat to organise future events – but the call is not specific to openQA, nor to GNOME: anything Linux-related and QA-related is in scope.

If you want to be involved in the next one, make sure you’re in the openQA chat room, or follow this thread on GNOME Discourse. The schedule is documented here and the next call should be 08:00UTC on Thursday 2nd May.

GNOME OS tests

On the topic of QA, the testsuite for GNOME OS is feeling pretty unloved at the moment. Tests still don’t pass reliably and haven’t done for months. Besides the existing issue with initial setup where GNOME Shell doesn’t start, there is a new regression that breaks the systemd user session and causes missing sound devices. Investigating these issues is a slow and boring process which you can read about in great detail on the linked issues.

Fun fact: most of GNOME OS works fine without a systemd user session – there is still a D-Bus session bus after all; systemd user sessions are quite new and we still (mostly) support non-systemd setups.

One thing is clear, we still need a lot of work on tooling and docs around GNOME OS and the tests, if we hope to get more people involved. I’m trying my best in the odd hours I have available, greatly helped by Valentin David and other folk in the #GNOME OS channel, but it still feels like wading through treacle.

We particularly could do with documentation on how the early boot and initial setup process is intended to work – its very hard to visualize just from looking at systemd unit files. Or maybe systemd itself can generate a graph of what should be happening.

Magic in the ssam_openqa tool

Debugging OS boot failures isn’t my favourite thing. I just want reliable tests. Writing support tooling in Rust is fun though, and it feels like magic to be able to control and debug VMs from a simple CLI tool, and play with them over VNC while the test suite runs.

Using a VNC connection to run shell commands is annoying at times: it’s a terminal in a desktop in a VNC viewer, with plenty of rendering glitches, and no copy/paste integration with the host. I recently noticed that while openQA tests are running, a virtio terminal is exposed on the host as a pair of in/out FIFOs, and you can control this terminal using cat and echo. This feels like actual magic.

I added a new option to ssam_openqa, available whenever the test runner is paused, to open a terminal connection to this virtio console, and now I can debug directly from the terminal on my host. I learned a few things about line buffering and terminal control codes along the way. (I didn’t get ANSI control codes like cursor movement to work, yet – not sure if my code is sending them wrong, or some TERM config is needed on the VM side. – but with backspace, tab and enter working it’s already fairly usable).

Here’s a quick demo of the feature:

Available in the 1.2.0-rc1 release. Happy debugging!

Cross country travel

Most of this month I was on holiday. If you’re a fan of overland travel, how about this: Aberdeen to Santiago in 48 hours; via night train to Crewe, camper van to Plymouth, ferry to Santander and then more driving across to Galicia. A fun trip, although I got pretty seasick on the boat until I’d had a good nights sleep. (This wasn’t a particularly low-carbon trip though despite going overland, as the train, ferry and van are all powered by big diesel engines.)

And now back to regular work – “Moving files from one machine to another using Python and YAML”, etc.

Just a little update about my activity around the Wikimedia Movement.

As said in this blog, I have been invited to guide a workshop on Wikibase at the Murcia University.

Also, I’ve sent two poster proposals for Wikimania 2024:

Towards a Very Small GLAM entities solution: This proposal proposes an activity line for empowering very small GLAM entities with limited resources to preserve and document cultural heritage effectively. It comprises:

• the development an open-source GLAM suite and
• recommendations on affordable, reliable hardware.

The suite includes:

• software such as unRAID OS with ZFS for data preservation and
• Wikibase and packaged software services.
• Also a preload of metadata for museology (ontologies and vocabularies) and a technical information collection in the form of linked open data and documents.

The proposal outlines the project’s timeline, funding sources, and physical and online community involvement.

and

Wikimedia LEADS a Learning Ecosystem and Ameliorating Data Space: To create a free ecosystem and data-space for learning in the Wikimedia Movement. Ecosystem will extends the Movement with new classes of knowledge and addressing sustainability needs. With:

• libraries of:
  • practices, modeled in Wikibase as Linked Open Data (LOD);
  • credentials, also modeled as LOD, based in ELM;
• software extensions and services required for a working implementation in the Movement.

First will address the GLAM Wiki domain, producing incremental results ready to be adopted. This domain strongly intersects with the Wikimedia Movement. Furthermore, the methodologies, tools and many of the specific contents will be applicable to any other knowledge areas.

We have coined the term Very Small GLAM to refer the community of very small GLAM institutions: private, public, formal or informal, etc. It has not a rigorous definition, but you can think in teams of less than 10 members and reduced budget. So we’ll drop the bombastic term of «Small GLAM SLAM».

About Wikimedia LEADS, this is a new line of work also based in Wikibase to develop a Wikimedia ecosystem of models for Essence practices and microcredentials. It has been proposed for an European Union grant so it has no funding yet. This initiative interesects with Very Small GLAM as both focuses first in contents for the GLAM Wiki domain, as project driver.

In this post I’ll try to document the journey starting from a WebKit issue and ending up improving third-party projects that WebKitGTK and WPEWebKit depend on.

I’ve been working on WebKit’s GStreamer backends for a while. Usually some new feature needed on WebKit side would trigger work on GStreamer. That’s quite common and healthy actually, by improving GStreamer (bug fixes or implementing new features) we make the whole stack stronger (hopefully). It’s not hard to imagine other web-engines, such as Servo for instance, leveraging fixes made in GStreamer in the context of WebKit use-cases.

Sometimes though we have to go deeper and this is what this post is about!

Since version 2.44, WebKitGTK and WPEWebKit ship with a WebCodecs backend. That backend leverages the wide range of GStreamer audio and video decoders/encoders to give low-level access to encoded (or decoded) audio/video frames to Web developers. I delivered a lightning talk at gst-conf 2023 about this topic.

There are still some issues to fix regarding performance and some W3C web platform tests are still failing. The AV1 decoding tests were flagged early on while I was working on WebCodecs, I didn’t have time back then to investigate the failures further, but a couple weeks ago I went back to those specific issues.

The WebKit layout tests harness is executed by various post-commit bots, on various platforms. The WebKitGTK and WPEWebKit bots run on Linux. The WebCodec tests for AV1 currently make use of the GStreamer av1enc and dav1ddec elements. We currently don’t run the tests using the modern and hardware-accelerated vaav1enc and vaav1dec elements because the bots don’t have compatible GPUs.

The decoding tests were failing, this one for instance (the ?av1 variant). In that test both encoding and decoding are tested, but decoding was failing, for a couple reasons. Rabbit hole starts here. After debugging this for a while, it was clear that the colorspace information was lost between the encoded chunks and the decoded frames. The decoded video frames didn’t have the expected colorimetry values.

The VideoDecoderGStreamer class basically takes encoded chunks and notifies decoded VideoFrameGStreamer objects to the upper layers (JS) in WebCore. A video frame is basically a GstSample (Buffer and Caps) and we have code in place to interpret the colorimetry parameters exposed in the sample caps and translate those to the various WebCore equivalents. So far so good, but the caps set on the dav1ddec elements didn’t have those informations! I thought the dav1ddec element could be fixed, “shouldn’t be that hard” and I knew that code because I wrote it in 2018 :)

So let’s fix the GStreamer dav1ddec element. It’s a video decoder written in Rust, relying on the dav1d-rs bindings of the popular C libdav1d library. The dav1ddec element basically feeds encoded chunks of data to dav1d using the dav1d-rs bindings. In return, the bindings provide the decoded frames using a Dav1dPicture Rust structure and the dav1ddec GStreamer element basically makes buffers and caps out of this decoded picture. The dav1d-rs bindings are quite minimal, we implemented API on a per-need basis so far, so it wasn’t very surprising that… colorimetry information for decoded pictures was not exposed! Rabbit hole goes one level deeper.

So let’s add colorimetry API in dav1d-rs. When working on (Rust) bindings of a C library, if you need to expose additional API the answer is quite often in the C headers of the library. Every Dav1dPicture has a Dav1dSequenceHeader, in which we can see a few interesting fields:

typedef struct Dav1dSequenceHeader { ... enum Dav1dColorPrimaries pri; ///< color primaries (av1) enum Dav1dTransferCharacteristics trc; ///< transfer characteristics (av1) enum Dav1dMatrixCoefficients mtrx; ///< matrix coefficients (av1) enum Dav1dChromaSamplePosition chr; ///< chroma sample position (av1) ... uint8_t color_range; ... ... } Dav1dSequenceHeader;

After sharing a naive branch with rust-av co-maintainers Luca Barbato and Sebastian Dröge, I came up with a couple pull-requests that eventually were shipped in version 0.10.3 of dav1d-rs. I won’t deny matching primaries, transfer, matrix and chroma-site enum values to rust-av‘s Pixel enum was a bit challenging :P Anyway, with dav1d-rs fixed up, rabbit hole level goes up one level :)

Now with the needed dav1d-rs API, the GStreamer dav1ddec element could be fixed. Again, matching the various enum values to their GStreamer equivalent was an interesting exercise. The merge request was merged, but to this date it’s not shipped in a stable gst-plugins-rs release yet. There’s one more complication here, ABI broke between dav1d 1.2 and 1.4 versions. The dav1d-rs 0.10.3 release expects the latter. I’m not sure how we will cope with that in terms of gst-plugins-rs release versioning…

Anyway, WebKit’s runtime environment can be adapted to ship dav1d 1.4 and development version of the dav1ddec element, which is what was done in this pull request. The rabbit is getting out of his hole.

The WebCodec AV1 tests were finally fixed in WebKit, by this pull request. Beyond colorimetry handling a few more fixes were needed, but luckily those didn’t require any fixes outside of WebKit.

Wrapping up, if you’re still reading this post, I thank you for your patience. Working on inter-connected projects can look a bit daunting at times, but eventually the whole ecosystem benefits from cross-project collaborations like this one. Thanks to Luca and Sebastian for the help and reviews in dav1d-rs and the dav1ddec element. Thanks to my fellow Igalia colleagues for the WebKit reviews.

Retro; the customizable clock widget is now available on Flathub in v2

This new release comes with

Support both 12h and 24h clock format. It follows GNOME Date & Time preference while being sandboxed thanks to libportal new API for the settings portal.

Energy usage has been improved by using a more efficient method to get the time and by making use of the magic GtkWindow.suspended property to stop updating the clock when the window is not visible.

Better support for round clocks. The new GTK renderer fixed the visual glitch on transparent corners caused by large border radius. Retro now restores window dimensions and disables the border radius on maximize to make it look good, no matter the shape.

Controls have been moved to a floating header bar to stay out of the way and prevent interference with customizations.

There are further improvements to do, but I decided to publish early because Retro was using GNOME 43 runtime which is end-of-life and I have limited time to spend on it.

Help welcome https://github.com/sonnyp/Retro/issues

(I need to put that somewhere because people ask about it and having a little post to explain it is nice.)

What’s it about?
GTK recently introduced the ability to offload graphics rendering, but it needs rather recent everything to work well for offloading video decoding.

So, what do you need to make sure this works?

First, you of course need a video to test. On a modern desktop computer, you want a 4k 60fps video or better to have something that pushes your CPU to the limits so you know when it doesn’t work. Of course, the recommendation has to be Big Buck Bunny at the highest of qualities – be aware that the most excellent 4000×2250 @ 60fps encoding is 850MB. On my Intel TigerLake, that occasionally drops frames when I play that with software decoding, and I can definitely hear the fan turn on.
When selecting a video file, keep in mind that the format matters.

Second, you need hardware decoding. That is provided by libva and can be queried using the vainfo tool (which comes in the `libva-utils` package in Fedora). If that prints a long list of formats (it’s about 40 for me), you’re good. If it doesn’t, you’ll need to go hunt for the drivers – due to the patent madness surrounding video formats that may be more complicated than you wish. For example, on my Intel laptop on Fedora, I need the intel-media-driver package which is hidden in the nonfree RPMFusion repository.
If you look at the list from vainfo, the format names give some hints – usually VP9 and MPEG2 exist. H264 and HEVC aka H265 are the patent madness, and recent GPUs can sometimes do AV1. The Big Buck Bunny video from above is H264, so if you’re following along, make sure that works.

Now you need a working video player. I’ll be using gtk4-demo (which is in the gtk4-devel-tools package, but you already have that installed of course) and its video player example because I know it works there. A shoutout goes out to livi which was the first non-demo video player to have a release that supports graphics offloading. You need GTK 4.14 and GStreamer 1.24 for this to work. At the time of writing, this is only available in Fedora rawhide, but hopefully Fedora 40 will gain the packages soon.

If you installed new packages above, now is a good time to check if GStreamer picked up all the hardware decoders. gst-inspect-1.0 va will list all the elements with libva support. If it didn’t pick up decoders for all the formats it should have (there should be a vah264dec listed for H264 if you want to decode the video above), then the easiest way to get them is to delete GStreamer’s registry cache in ~/.cache/gstreamer-1.0.

If you want to make sure GStreamer does the right thing, you can run the video player with GST_DEBUG=GST_ELEMENT_FACTORY:4. It will print out debug messages about all the elements it is creating for playback. If that includes a line for an element from the previous list (like `vah264dec` in our example) things are working. If it picks something else (like `avdec_h264` or `openh264dec`) then they are not.

Finally you need a compositor that supports YUV formats. Most compositors do – gnome-shell does since version 45 for example – but checking can’t hurt: If wayland-info (in the wayland-utils package in Fedora) lists the NV12 format, you’re good.

And now everything works.
If you have a 2nd monitor you can marvel at what goes on behind the scenes by running the video player with GDK_DEBUG=dmabuf,offload and GTK will tell you what it does for every frame, and you can see it dynamically switching between offloading or not as you fullscreen (or not), click on the controls (or not) and so on. Or you could have used it previously to see why things didn’t work.
You can also look at the top and gputop variant of your choice and you will see that the video player takes a bit of CPU to drive the video decoding engine and inform the compositor about new frames and the compositor takes a bit of CPU telling the 3D engine to composite things and send them to the monitor. With the video above it’s around 10% on my laptop for the CPU usage each and about 20% GPU usage.

And before anyone starts complaining that this is way too complicated: If you read carefully, all of this should work out of the box in the near future. This post just lists the tools to troubleshoot what went wrong while developing a fast video player.

Update on what happened across the GNOME project in the week from April 05 to April 12.

GNOME Core Apps and Libraries GLib

The low-level core library that forms the basis for projects such as GTK and GNOME.

Philip Withnall announces

After a lot of preparation, GLib has finally achieved an OpenSSF Best Practices ‘passing’ badge, which certifies that it follows a number of development and security best practices — see https://www.bestpractices.dev/en/projects/6011

GNOME Circle Apps and Libraries Fragments

Easy to use BitTorrent client.

Felix reports

It has finally happened! The long awaited major update of Fragments is now available, which includes many exciting new features.

The most important addition is support for torrent files. It is now possible to select the files you want to download from a torrent. The files can be searched and sorted, individual files can be opened directly from Fragments.

Further new features:

• Added torrents can now be searched
• In addition to magnet links, *.torrent links in the clipboard are now also recognized
• Prevent system from going to sleep when torrents are active
• New torrents can be added via drag and drop
• Automatic trashing of *.torrent files after adding them
• Stop downloads when a metered network gets detected

Improvements:

• When controlling remote sessions, the local Transmission daemon no longer gets started
• Torrents are automatically restarted if an incorrect location has been fixed
• Torrents can now also be added via CLI
• Clipboard toast notification is no longer displayed multiple times
• Reduced CPU/resource consumption through adaptive polling interval
• Improved accessibility of the user interface
• Modernized user interface through the use of new Adwaita widgets
• Update from Transmission 3.0.5 to 4.0.5

More information can be found in the announcement blog post. Your browser doesn't support embedded videos, but don't worry, you can download it and watch it with your favorite video player!

Pika Backup

Keep your data safe.

Fina reports

Pika Backup 0.7.2 was released. It fixes a crash on the schedule page. It also resolves an issue with pre- and post-backup scripts being unable to send desktop notifications.

We have already made lots of progress towards 0.8 which will focus on code maintainability and UI refinements. You can support Pika’s development on Open Collective.

Blanket

Improve focus and increase your productivity by listening to different sounds.

Rafael Mardojai CM announces

Blanket 0.7.0 has been released and is available on Flathub.

This new release features a redesigned user interface and uses the latest GNOME design patterns.

Other changes include:

• Inhibit suspension when playing
• Pause playback if the system enters power saver mode
• MPRIS: Implement Play, Pause and Stop methods in addition to PlayPause
• MPRIS: Implement Next and Prev methods for navigating presets
• Updated Pink Noise sample
• Changed train sound
• Added preference to always start on pause

Third Party Projects

Krafting announces

Hello! This week I released SemantiK. It’s a word-game where you need to find a secret word, similar to Cémantix or Semantle. The default language model is in French, but you can use and import your own !

It is available on Flathub

xjuan says

New Cambalache stable version released! The app ui is now made with Cambablache and Gtk 4! Read more about v 0.90.0 at https://blogs.gnome.org/xjuan/2024/04/06/cambalache-0-90-0-released/

alextee announces

The GTK4/libadwaita-based digital audio workstation Zrythm has released its last beta version v1.0.0-beta.6.7.32 in preparation for a release candidate!

More info on our website. Zrythm is also available on Flathub

Phosh

A pure wayland shell for mobile devices.

Guido reports

Phosh 0.38.0 is out:

Phosh now handles devices with rounded display corners better and supports count and progress indicators in lock screen launcher entries. On the compositor side we now handle always-on-top and move-to-corner keybindings and the on screen keyboard squeekboard got a whole bunch of layout improvements.

There’s more. Check the full details here

Events

mwu reports

TWIG-Bot We are ramping up for GUADEC 2024 and sponsorships are still available. We made it easier this year and put the different opportunities online! Click here for more info: https://events.gnome.org/event/209/registrations/212/

That’s all for this week!

See you next week, and be sure to stop by #thisweek:gnome.org with updates on your own projects!

It has finally happened! The long awaited major update of Fragments is now available, which includes many exciting new features.

The most important addition is support for torrent files. It is now possible to select the files you want to download from a torrent. The files can be searched and sorted, individual files can be opened directly from Fragments.

https://blogs.gnome.org/haeckerfelix/files/2024/04/Screencast-from-2024-04-07-19-53-28.webm

Further new features

• • Added torrents can now be searched
  • In addition to magnet links, *.torrent links in the clipboard are now also recognized
  • Prevent system from going to sleep when torrents are active
  • New torrents can be added via drag and drop
  • Automatic trashing of *.torrent files after adding them
  • Stop downloads when a metered network gets detected

Improvements

• • When controlling remote sessions, the local Transmission daemon no longer gets started
  • Torrents are automatically restarted if an incorrect location has been fixed
  • Torrents can now also be added via CLI
  • Clipboard toast notification is no longer displayed multiple times
  • Reduced CPU/resource consumption through adaptive polling interval
  • Improved accessibility of the user interface
  • Modernized user interface through the use of new Adwaita widgets
  • Update from Transmission 3.0.5 to 4.0.5

Thanks to Maximiliano and Tobias for once again helping with this release. As usual this release contains many other improvements, fixes and new translations thanks to all the contributors and upstream projects.

Also a big shoutout to the Transmission project, without which Fragments would not be possible, for their fantastic 4.0 release!

The new Fragments release can be downloaded and installed from Flathub:

Hi, I am happy to announce a new Cambalache stable release.

With the UI ported to Gtk 4 I bumped the version to 0.90 to reflect the fact we are really close to 1.0

Release Notes:

• • Migrate main application to Gtk 4
  • Update widget catalogs to SDK 46
  • Add support for child custom fragments
  • Add add parent context menu action
  • Mark AdwSplitButton.dropdown-tooltip translatable. (Danial Behzadi)

Where to get it?

You can get it from Flathub

flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo flatpak install flathub ar.xjuan.Cambalache

or checkout cmb-90 branch at gitlab

git clone https://gitlab.gnome.org/jpu/cambalache.git Matrix channel

Have any question? come chat with us at #cambalache:gnome.org

Mastodon

Follow me in Mastodon @xjuan to get news related to Cambalache development.

Happy coding!

As a follow-up in spirit of reversing the USB a blood pressure monitor See Github project I was part of a project to revive a GPS receiver from the late 2000s. This is still work in process since it is still not possible to decode the actual GPS data, but we are getting there.

The code and documentation is also available at github and we’ve done a talk about the project at Easterhegg 21, which is on media.ccc.de (german only, sorry).

VTE (Virtual TErminal library) is the library underpinning various GNOME terminal emulators. It provides a GTK widget that shows a terminal view, which is used in apps like GNOME Terminal, Console, Black Box, Tilix, Terminator, Ptyxis, and others. It also powers embedded terminals in Builder and Workbench.

Over the GNOME 46 cycle, VTE has seen a lot of performance improvements. Christian Hergert mentioned some of them in his blog posts about VTE and about his work in GNOME 46. But how much did the performance actually improve? What should you, the user, expect to feel after installing a fresh Fedora 40 update and launching your favorite terminal?

Let’s measure and find out! If you don’t have time for measuring, you can skip straight to the finding out.

What Are We Measuring? #

There is no shortage of ways to define “performance”, especially when it comes to terminal emulators. One of the more tangible metrics is input latency. Roughly, it describes how quickly the program reacts to your actions: how much time passes from the moment you press a key on your keyboard to the change in color of the pixels on your monitor. Apps with low input latency feel snappy, whereas apps with high input latency can feel sluggish.

When the input latency is small-ish, you can get used to it and think it feels fine. However, comparing lower and higher input latency together (for example, by switching between two apps and typing in both) can make it quite noticeable. If you’ve ever heard people say they can’t go back to a 60 Hz monitor after trying out 144 Hz, that’s a similar effect (and input latency is partially responsible).

So, how do you measure it?

Measuring Input Latency #

There are tools like Typometer that measure the input latency in software by detecting key press events and recording the screen to detect a change in pixel color. This can work reasonably well but requires fiddling with your setup to make sure you’re not accidentally introducing any biases. For example, a screen capture API may return the new pixel colors a few milliseconds before or after they are shown on the monitor, depending on the system setup, and you need to be aware of this when trying to measure something to a millisecond precision.

I’ve got something more interesting, a hardware input latency tester! It consists of a light sensor attached to a Teensy board, which in turn is plugged into the computer via USB.

I should really get around to writing a full blog post about this latency tester, but for now, you should read this post by Tristan Hume about building a similar device.1 I used that post as a reference for building mine, but I wrote my own firmware and analysis scripts (these I am not sharing until they are less of an utter mess).

The main benefit of such a device is that it allows you to measure a full end-to-end input latency, including processing time in the kernel, the compositor, the application, and then the response time of the monitor itself. You are measuring what you really see and feel, excluding only the keyboard firmware (since the latency tester sends key press events directly over USB). There’s also very little extra load on the system, especially compared to using something like a screen capture API.

Here’s a gist of how it works. The light sensor is aimed at a specific, small area on the monitor, which will be affected by the key press (in our case, a specific character cell in the terminal). The board sends a key press over USB (for example, Space) and starts monitoring the light sensor readings. As soon as it detects a jump in the light amount, it releases the key. Then, it presses a second key (for example, Backspace) and waits for the light to change back. Now we’re back to square one; the firmware waits a randomized amount (to prevent “snapping” to the monitor refresh rate) and repeats the experiment.

During all of this process, the board dumps light sensor readings over a serial port as fast as it can manage (I’m getting about 35,500 readings per second with my current board and firmware). On the computer, I save all of this data into a file for offline analysis with Python code. This analysis code finds the timestamp where the light starts to change, and subtracts it from the timestamp of the key press, to get one input latency measurement.

I then aggregate the measurements and plot them with seaborn. Here’s an example of what the result looks like:

Input Latency Plots #

Let’s explore what you can find on this latency plot.

The small black dots represent the individual measurements. As in, every dot shows a real amount of time that had passed between one key press and the corresponding change in light on the sensor. There are 120 of these dots since I repeat each test 120 times.

Looking at the dots can confirm that the data is sensible. We expect the bulk of the measurements to be spread uniformly across an interval roughly the size of one monitor repaint cycle. This is because monitors generally repaint at a constant rate, and pressing a key at a random point in time should land us in a random point of the repaint cycle. We get the lowest latency if the application renders a new frame in response right in time for the monitor to show it. And we get the highest latency when the application finishes rendering a new frame just missing the monitor deadline, having to wait one extra repaint cycle for the pixel colors to change.

In the example above, the dots are spread over 7–8 ms, which is about equal to the ~6.94 ms refresh cycle of my 144 Hz monitor.

High outliers in the dots, or a larger spread, indicate lag or slowness of the application under test: some key presses are taking longer than others to process.

We do not expect to see any gaps between dot clusters. They would usually indicate aliasing with the monitor repaint cycle, or some frame scheduling bug in the compositor.2

The box shows statistics over the individual measurements:

• median (a measurement perfectly “in the middle” with half of the measurements lower and half of the measurements higher),
• lowest and highest measurement,
• 25th and 75th percentiles (with 25% and 75% of the measurements lower than the line, respectively).

All in all, you can compare applications by their spread, then by the median latency, and also look if there are any outliers.

With all that said, we’re almost ready to look at some results. I just need to tell you what exactly I was measuring the latency of.

Test Setup #

I did all tests on this system:

• Lenovo Legion 7 Gen 7 AMD with Ryzen 7 6800H CPU and Radeon RX 6700M dGPU (using the dGPU exclusively via the MUX switch).
• Monitor: Acer Nitro XV320QU, 2560×1440, 144 Hz, using 100% scale.
• Host: Fedora 40 Silverblue Beta, Mesa 24.0.4.
• Compositor: raw Mutter 46.0.

What is raw Mutter, you may ask? Well, Mutter is the compositor that GNOME Shell builds on top of. Turns out, you can start Mutter on its own, without GNOME Shell, by switching to a different VT and running a command like mutter --display-server -- alacritty. This gives you a very bare-bones environment that is only really meant for testing. It is, however, quite useful for benchmarking, as it represents something close to a zero-overhead GNOME Shell ideal case.

I’m testing several terminal applications. In the order of appearance on the plots, they are:

• Alacritty: not VTE-based; serves as a baseline of sorts, because it is consistently one of the fastest terminals according to all of my prior tests.
• Console: GTK 4, the default terminal in GNOME.3
• VTE Test App: GTK 4, a test terminal that lives in the VTE repository.
• GNOME Terminal: GTK 3,4 used to be the default in GNOME, and is still shipped out of the box in several distributions.

Since the intention is to compare GNOME 45 to GNOME 46, I used toolb\0x containers with Fedora 39 and Fedora 40 to install and run all terminals above, as packaged by Fedora with no extra tweaks.

I ran the terminals one by one and put their windows in the top left corner of the monitor. The mouse cursor was outside the window for all tests.5

Input Latency Tests #

The first test is simple: I run cat > /dev/null to get an input field with no readline or similar processing, and then I measure how long it takes for the terminal to move its block cursor one cell to the right after pressing Space.

This is meant to test the best possible scenario for the terminal, with the least overhead.

This is what the test process looks like:

And here are the results:

Alacritty, which is our baseline, did not change from F39 to F40, as expected.

But look at the massive improvement on all of the VTE terminals! They went from quite bad to pretty much on par with Alacritty, even the GTK 3 GNOME Terminal is very close.

The main change that caused this much improvement is likely this one by Christian that moves away from a 40 Hz VTE repaint timer to drawing every frame, synchronized with the monitor, as any self-respecting GTK widget should do.

Console has a few outliers which are maybe caused by its process tracking, but those are nothing new (they may be looked into for GNOME 47).

For the next test, I constructed a more realistic case. I took a snapshot of my neovim setup and opened the README from Ptyxis. I then strategically replaced a square of text with Unicode full-block characters to provide a bright “landing pad” for the light sensor.

The test consists of repeatedly pressing Ctrl+D and Ctrl+U to scroll the text buffer down and up in neovim. The light sensor alternates between an empty line (dark) and the full-block landing pad (bright). The neovim setup has a bunch of bells and whistles, so the terminal gets to have fun drawing the various underlines, undercurls, gutter icons, and the statusline.

This is what the test process looks like:

Here are the results:

The massive improvement is clear on this test too, and our GNOME 46 terminals are still pretty much on par with Alacritty!

Finally, let’s take a closer look at all Fedora 40 results on one plot:

This plot shows how much of a latency toll the neovim test takes compared to a simple cat, but the latency increase is similar across all terminals.

vtebench #

I also ran Alacritty’s vtebench suite across the same set of applications and configurations. This is a fully automated benchmark that measures something completely different from input latency: PTY read and parsing performance. It has also proven quite capable at finding crashes in VTE.

Here’s what vtebench’s README has to say:

This benchmark is not sufficient to get a general understanding of the performance of a terminal emulator. It lacks support for critical factors like frame rate or latency. The only factor this benchmark stresses is the speed at which a terminal reads from the PTY. If you do not understand what this means, please do not jump to any conclusions from the results of this benchmark.

The repaint duration can and does affect the results of this test, especially for terminals that read and parse PTY on the same thread as they run their repaint logic, like VTE.

This is what one of the vtebench benchmarks looks like:

And here are the results:

To avoid making this plot even busier, I drew the green arrows on only one of the benchmarks. As you can see, other benchmarks show a similar trend.

VTE from GNOME 46 shows some welcome improvements here too, although a lot more varied, and not quite on par with Alacritty (which renders in a separate thread from reading and parsing). These improvements likely come from the many other optimizations that happened in VTE during the GNOME 46 cycle.

Note that I omitted two benchmarks from these results: dense_cells and unicode. They are the main stress tests of vtebench that hit the terminal really hard. Unfortunately, VTE still struggles with them and shows a huge spread, which pushes the rest of the results down and makes the plot less readable.

Open this to see the full results if you’re curious.

Conclusion #

VTE had a round of massive performance improvements in GNOME 46 which manifest as something you can really feel during normal terminal use. The input latency is down to almost matching the fastest terminals, even in a non-trivial neovim setup with lots of complexity on screen.

The remaining difference, at least on these test cases, is close to negligible. Some of it can be explained by VTE doing a bit more extra work for accessibility (enabled in GNOME Terminal and currently disabled in the GTK 4 terminals), scrollbar calculations, and other features.

If you’ve been avoiding VTE-based terminals due to sluggishness and input lag, now is the time to give them another chance. Just make sure you’re running VTE 0.76, which includes all of this goodness.

Huge thanks to the VTE maintainers and contributors for making this a reality, and congratulations on an awesome release!

P.S. If you’re curious about Ptyxis or the behavior of GTK’s NGL vs. NVK vs. GL renderers, they all perform similarly to the F40 VTE Test App results shown above. I did more extensive benchmarks of these a month ago, you can find them here.

1. As you can tell from the photo, I did not follow Tristan’s advice to make something fancier than just dangling wires. ↩︎

2. Just a few weeks ago some measurements I took showed a suspicious one-frame-long gap in the dots. And guess what, it was a frame scheduling bug in my compositor, with none other than myself to blame for it. Thankfully, it wasn’t hard to fix, and easy to verify afterward by redoing the same test. ↩︎

3. Your distribution may have a different idea of which terminal should be the default in its GNOME spin. For example, Fedora still ships GNOME Terminal by default. ↩︎

4. GNOME Terminal is being ported to GTK 4 for GNOME 47, but in GNOME 46 it is still a GTK 3 application. ↩︎

5. To avoid the link-under-cursor detection logic skewing the results. ↩︎

A few people (and multi-billion dollar companies!) have asked for my response to the xz backdoor. The fwupd metadata that millions of people download every day is a 9.5MB XML file — which thankfully is very compressible. This used to be compressed as gzip by the LVFS, making it a 1.6MB download for end-users, but in 2021 we switched to xz compression instead.

What actually happens behind the scenes is that the libxmlb library loads the optionally compressed metadata into a mmap-able binary blob, and then it gets used by fwupd to look for new updates for specific hardware. In libxmlb 0.3.3 we added support for xz as a compression format. Then fwupd 1.8.7 was released with xz support, preferring the xz format to the “legacy” gz format — as the metadata became a 1.1MB download, saving significant amounts of data from the CDN.

Then this week we learned that xz wasn’t the kind of thing we want to depend on. Out of an abundance of caution (and to be clear — my understanding is there is no fwupd or LVFS security problem of any kind) I’ve switched the LVFS to also generate zstd metadata, make libxmlb no longer hard depend on lzma and switched fwupd to prefer the zstd metadata over the xz metadata if the installed version of libjcat supports it. The zstd metadata is also ~3% smaller than xz (and faster to decompress), but the real benefit is that I now trust it a lot more than xz.

I’ll be doing new libxmlb and fwupd releases with the needed changes next week.

So Fedora Workstation 40 Beta has just come out so I thought I share a bit about some of the things we are working on for Fedora Workstation currently and also major changes coming in from the community.

Flatpak

Flatpaks has been a key part of our strategy for desktop applications for a while now and we are working on a multitude of things to make Flatpaks an even stronger technology going forward. Christian Hergert is working on figuring out how applications that require system daemons will work with Flatpaks, using his own Sysprof project as the proof of concept application. The general idea here is to rely on the work that has happened in SystemD around sysext/confext/portablectl trying to figure out who we can get a system service installed from a Flatpak and the necessary bits wired up properly. The other part of this work, figuring out how to give applications permissions that today is handled with udev rules, that is being worked on by Hubert Figuière based on earlier work by Georges Stavracas on behalf of the GNOME Foundation thanks to the sponsorship from the Sovereign Tech Fund. So hopefully we will get both of these two important issues resolved soon. Kalev Lember is working on polishing up the Flatpak support in Foreman (and Satellite) to ensure there are good tools for managing Flatpaks when you have a fleet of systems you manage, building on the work of Stephan Bergman. Finally Jan Horak and Jan Grulich is working hard on polishing up the experience of using Firefox from a fully sandboxed Flatpak. This work is mainly about working with the upstream community to get some needed portals over the finish line and polish up some UI issues in Firefox, like this one.

Toolbx

Toolbx, our project for handling developer containers, is picking up pace with Debarshi Ray currently working on getting full NVIDIA binary driver support for the containers. One of our main goals for Toolbx atm is making it a great tool for AI development and thus getting the NVIDIA & CUDA support squared of is critical. Debarshi has also spent quite a lot of time cleaning up the Toolbx website, providing easier access to and updating the documentation there. We are also moving to use the new Ptyxis (formerly Prompt) terminal application created by Christian Hergert, in Fedora Workstation 40. This both gives us a great GTK4 terminal, but we also believe we will be able to further integrate Toolbx and Ptyxis going forward, creating an even better user experience.

Nova

So as you probably know, we have been the core maintainers of the Nouveau project for years, keeping this open source upstream NVIDIA GPU driver alive. We plan on keep doing that, but the opportunities offered by the availability of the new GSP firmware for NVIDIA hardware means we should now be able to offer a full featured and performant driver. But co-hosting both the old and the new way of doing things in the same upstream kernel driver has turned out to be counter productive, so we are now looking to split the driver in two. For older pre-GSP NVIDIA hardware we will keep the old Nouveau driver around as is. For GSP based hardware we are launching a new driver called Nova. It is important to note here that Nova is thus not a competitor to Nouveau, but a continuation of it. The idea is that the new driver will be primarily written in Rust, based on work already done in the community, we are also evaluating if some of the existing Nouveau code should be copied into the new driver since we already spent quite a bit of time trying to integrate GSP there. Worst case scenario, if we can’t reuse code, we use the lessons learned from Nouveau with GSP to implement the support in Nova more quickly. Contributing to this effort from our team at Red Hat is Danilo Krummrich, Dave Airlie, Lyude Paul, Abdiel Janulgue and Phillip Stanner.

Explicit Sync and VRR

Another exciting development that has been a priority for us is explicit sync, which is critical for especially the NVidia driver, but which might also provide performance improvements for other GPU architectures going forward. So a big thank you to Michel Dänzer , Olivier Fourdan, Carlos Garnacho; and Nvidia folks, Simon Ser and the rest of community for working on this. This work has just finshed upstream so we will look at backporting it into Fedora Workstaton 40. Another major Fedora Workstation 40 feature is experimental support for Variable Refresh Rate or VRR in GNOME Shell. The feature was mostly developed by community member Dor Askayo, but Jonas Ådahl, Michel Dänzer, Carlos Garnacho and Sebastian Wick have all contributed with code reviews and fixes. In Fedora Workstation 40 you need to enable it using the command

gsettings set org.gnome.mutter experimental-features "['variable-refresh-rate']"

PipeWire

Already covered PipeWire in my post a week ago, but to quickly summarize here too. Using PipeWire for video handling is now finally getting to the stage where it is actually happening, both Firefox and OBS Studio now comes with PipeWire support and hopefully we can also get Chromium and Chrome to start taking a serious look at merging the patches for this soon. Whats more Wim spent time fixing Firewire FFADO bugs, so hopefully for our pro-audio community users this makes their Firewire equipment fully usable and performant with PipeWire. Wim did point out when I spoke to him though that the FFADO drivers had obviously never had any other consumer than JACK, so when he tried to allow for more functionality the drivers quickly broke down, so Wim has limited the featureset of the PipeWire FFADO module to be an exact match of how these drivers where being used by JACK. If the upstream kernel maintainer is able to fix the issues found by Wim then we could look at providing a more full feature set. In Fedora Workstation 40 the de-duplication support for v4l vs libcamera devices should work as soon as we update Wireplumber to the new 0.5 release.

To hear more about PipeWire and the latest developments be sure to check out this interview with Wim Taymans by the good folks over at Destination Linux.

Remote Desktop

Another major feature landing in Fedora Workstation 40 that Jonas Ådahl and Ray Strode has spent a lot of effort on is finalizing the remote desktop support for GNOME on Wayland. So there has been support for remote connections for already logged in sessions already, but with these updates you can do the login remotely too and thus the session do not need to be started already on the remote machine. This work will also enable 3rd party solutions to do remote logins on Wayland systems, so while I am not at liberty to mention names, be on the lookout for more 3rd party Wayland remoting software becoming available this year.

This work is also important to help Anaconda with its Wayland transition as remote graphical install is an important feature there. So what you should see there is Anaconda using GNOME Kiosk mode and the GNOME remote support to handle this going forward and thus enabling Wayland native Anaconda.

HDR

Another feature we been working on for a long time is HDR, or High Dynamic Range. We wanted to do it properly and also needed to work with a wide range of partners in the industry to make this happen. So over the last year we been contributing to improve various standards around color handling and acceleration to prepare the ground, work on and contribute to key libraries needed to for instance gather the needed information from GPUs and screens. Things are coming together now and Jonas Ådahl and Sebastian Wick are now going to focus on getting Mutter HDR capable, once that work is done we are by no means finished, but it should put us close to at least be able to start running some simple usecases (like some fullscreen applications) while we work out the finer points to get great support for running SDR and HDR applications side by side for instance.

PyTorch

We want to make Fedora Workstation a great place to do AI development and testing. First step in that effort is packaging up PyTorch and making sure it can have working hardware acceleration out of the box. Tom Rix has been leading that effort on our end and you will see the first fruits of that labor in Fedora Workstation 40 where PyTorch should work with GPU acceleration on AMD hardware (ROCm) out of the box. We hope and expect to be able to provide the same for NVIDIA and Intel graphics eventually too, but this is definitely a step by step effort.

The post Win-win for all: How to run a non-engineering Outreachy internship appeared first on /home/jwf/.

/home/jwf/ - Free & Open Source, technology, travel, and life reflections

This year, I am mentoring again with the Outreachy internship program. It is my third time mentoring for Outreachy and my second time with the Fedora Project. However, it is my first time mentoring as a Red Hat associate. What also makes this time different from before is that I am mentoring a non-engineering project with Outreachy. Or in other words, my project does not require an applicant to write any code. Evidently, the internship description was a hook. We received an extremely large wave of applicants literally overnight. Between 40-50 new contributors arrived to the Fedora Marketing Team in the first week. Planning tasks and contributions for beginners already took effort. Scaling that planning work overnight for up to 50 people simultaneously is extraordinarily difficult.

During this round, my co-mentor Joseph Gayoso and I experimented with new approaches at handling the tsunami wave. There are two competing forces at play. One, you need to provide engagement to top performers so they remain motivated to continue. Two, you need to provide new opportunities for emerging contributors to distinguish themselves. It is easier to do one of these but hard to do both simultaneously. However, Joseph and I agreed on something important. We agreed that all applicants should end the contribution phase with something practically useful. As mentors, we asked ourselves how to prepare applicants to be successful open source contributors beyond this one month.

In this article, you will get some practical takeaways for mentoring with Outreachy. First, I will share our practical approach for structuring and planning an open source project during the Outreachy contribution phase. Second, I will detail the guiding philosophy Joseph and I follow for how we planned the contribution phase.

About Outreachy

This article assumes you already know a thing or two about the Outreachy internship program. If not, Outreachy provides internships in open source and open science. Outreachy provides internships to people subject to systemic bias and impacted by underrepresentation in the technical industry where they live. You can read more on the Outreachy website.

What makes Outreachy unique is that the internships are remote and often open without geographic or nationality constraints. Applicants from nearly every continent of the world have participated in Outreachy. Also, Outreachy is distinguished by the contribution phase. For a one-month period, approved Outreachy applicants are encouraged to participate in the project community as a contributor. Applicants spend the month learning about the project, the community, the mentors, and the work involved for the internship. This provides applicants an opportunity to grow their open source identity. It also gives mentors an opportunity to assess applicants on their skills and communication abilities.

However, this contribution phase can be intimidating as a mentor, especially if you are new to mentoring with Outreachy. A wave of people eager to contribute could suddenly appear overnight at your project’s door steps. If you are not prepared, you will have to adapt quickly!

Pre-Requisite Tasks: Raising the Outreachy bar

My co-mentor and I knew that a wave of applicants was coming. However, we didn’t expect the wave to be as big as it was. After the first week of the contribution phase, we knew we needed a better way to scale ourselves. We were limited in our person-power. The approach we took to addressing the mental overload was defining pre-requisite tasks.

We defined pre-requisite tasks as tasks that any applicant MUST complete in order to be considered eligible for our internship. Without completing these tasks, we explained that final applications would not be accepted by mentors. The defining characteristics of these pre-requisite tasks were that they were personalized, repeatable, and measurable. We came up with five pre-requisite tasks that all applicants were required to complete beyond the initial qualification for Outreachy:

1. Set up your Fedora Account System (FAS) account
2. Set up a personal blog
3. Write a blog post that introduces the Fedora community to your audience
4. Promote your intro blog post on social media
5. Write an onboarding guide for Outreachy 2025 applicants

How were initial contributions personalized?

Each of these tasks were personalized to each applicant. They each have a unique account profile, with their pictures, time zones, and chat system usernames. The personal blog is a personal space on the Internet for each applicant to start writing new posts. The blog post prompts encouraged applicants to start filling up their blogs with Fedora content. The social media post helped applicants promote themselves as budding open source enthusiasts in their existing web spaces.

This approach had two benefits. First, it provided clear guidance to all newcomers and early-stage applicants on how to get started with contributing to Fedora for the Outreachy internship. This took a burden off of mentors answering the same questions about getting started. It also gave new applicants something to start on right away. Joseph and I were able to put more time into reviewing incoming contributions and brainstorming new tasks.

Portfolio-driven submissions for Outreachy

Toward the third week, many applicants had completed the pre-requisite tasks and were ready for more advanced tasks. Many had already taken on advanced projects already, beyond the pre-requisite tasks. Although the pre-requisite tasks did reduce the applicant pool, there were still between 20-30 people who completed them all. Again, the approach had to adapt as our ability to keep up with new contributions slowed down.

From here, we encouraged applicants to build personal portfolio pages that described their contributions with Fedora. This encouraged applicants to use the blog they built in the previous tasks, although they are not required to use their blog to host their portfolio. The only requirement we added was that it should be publicly visible on the Internet without a paywall. So, no Google Docs. Most applicants have ended up using their blog for this purpose though.

How did a portfolio help?

Building a portfolio solved multiple challenges for our Outreachy project at once. First, the portfolios will simplify how the project mentors review final applications after the deadline on April 2nd, 2024. It will be streamlined because we will have a single place we can refer to that describes the applicant’s achievements. It gives us a quick, easily shareable place to review and share with other stakeholders.

Second, it ends up being something useful to the applicant as well. The portfolio page captures a month’s worth of contributions to open source. For many applicants, this is their first time ever interacting with an open source community online. So, it is a big deal to block out a month of time to volunteer on a project in a competitive environment for a paid, remote internship opportunity. Writing a portfolio page gives applicants the confidence to represent their contributions to Fedora, regardless of whether they are selected for the Fedora internship. It becomes a milestone marker for themselves and for their professional careers.

Our philosophy: You win, we win.

This idea of applicants building something that is useful for themselves underpins the approach that Joseph and I took on structuring our non-engineering Outreachy internship. If I had to summarize the philosophy in one sentence, it might be like this:

Everyone who participants as an Outreachy applicant to Fedora should finish the contribution phase with more than they had at the start of the contribution phase.

myself

Our philosophy can be applied to engineering and non-engineering internships. However, applying the philosophy to our non-engineering project required improvisation as we went. There are examples of design-centered Outreachy internships, but I have not seen a marketing or community manager internship before. This was a challenge because there were not great models to follow. But it also left us room to innovate and try ideas that we have never tried before.

Adopting this philosophy served as helpful guidance on planning what we directed applicants to do during the contribution phase. It allowed us to think through ways that applicants could make real, recognizable contributions to Fedora. It also enables applicants to achieve a few important outcomes:

1. Get real experience in a real project.
2. Build their own brand as open source contributors.
3. Gain confidence at collaborating in a community.

The contribution phase is not yet over. So, we will continue to follow this philosophy and see where it guides us into the end of this phase!

Share your Outreachy mentoring experience!

Have you experienced or seen a marketing or community manager internship in Outreachy before? Know a project or a person who has done this? Or is this totally new to you? Drop a comment below with your thoughts. Don’t forget to share with someone else if you found this advice useful.

Christian was looking at PGO and BOLT recently I figured I’d write down my notes from the discussions we had on how we’d go about making things faster on our stack, since I don’t have time or the resource to pursue those plans myself atm.

First off let’s start with the basics, PGO (profile guided optimizations) and BOLT (Binary Optimization and Layout Tool) work in similar ways. You capture one or more “profiles” of a workload that’s representative of a usecase of your code and then the tools do their magic to make the common hot paths more efficient/cache-friendly/etc. Afterwards they produce a new binary that is hopefully faster than the old one and functionally identical so you can just replace it.

Now already we have two issues here that arise here:

First of all we don’t really have any benchmarks in our stack, let alone, ones that are rounded enough to account for the majority of usecases. Additionally we need better instrumentation to capture stats like frames, frame-times, and export them both for sysprof and so we can make the benchmark runners more useful.

Once we have the benchmarks we can use them to create the profiles for optimizations and to verify that any changes have the desired effect. We will need multiple profiles of all the different hardware/software configurations.

For example for GTK ideally we’d want to have a matrix of profiles for the different render backends (NGL/Vulkan) along with the mesa drivers they’d use depending on different hardware AMD/Intel and then also different architectures, so additional profile for Raspberrypi5 and Asahi stacks. We might also want to add a profile captured under qemu+virtio while we are it too.

Maintaining the benchmarks and profiles would be a lot of work and very tailored to each project so they would all have to live in their upstream repositories.

On the other hand, the optimization itself has to be done during the Tree/userland/OS composition and we’d have to aggregate all the profiles from all the projects to apply them. This is easily done when you are in control of the whole deployment as we can do for the GNOME Flatpak Runtime. It’s also easy to do if you are targeting an embedded deployment where most of the time you have custom images you are in full control off and know exactly the workload you will be running.

If we want distros to also apply these optimizations and for this to be done at scale, we’d have to make the whole process automatic and part of the usual compilation process so there would be no room for error during integration. The downside of this would be that we’d have a lot less opportunities for aggregating different usecases/profiles as projects would either have to own optimizations of the stack beneath them (ex: GTK being the one relinking pango) or only relink their own libraries.

To conclude, Post-linktime optimization would be a great avenue to explore as it seems to be one of the lower-hanging fruits when it comes to optimizing the whole stack. But it also would be quite the effort and require a decent amount of work to be committed to it. It would be worth it in the long run.

Good day, hackers. Today, a pragmatic note, on hacking on V8 from a Guix system.

I’m going to skip a lot of the background because, as it turns out, I wrote about this already almost a decade ago. But following that piece, I mostly gave up on doing V8 hacking from a Guix machine—it was more important to just go with the flow of the ever-evolving upstream toolchain. In fact, I ended up installing Ubuntu LTS on my main workstations for precisely this reason, which has worked fine; I still get Guix in user-space, which is better than nothing.

Since then, though, Guix has grown to the point that it’s easier to create an environment that can run a complicated upstream source management project like V8’s. This is mainly guix shell in the --container --emulate-fhs mode. This article is a step-by-step for how to get started with V8 hacking using Guix.

get the code

You would think this would be the easy part: just git clone the V8 source. But no, the build wants a number of other Google-hosted dependencies to be vendored into the source tree. To perform the initial fetch for those dependencies and to keep them up to date, you use helpers from the depot_tools project. You also use depot_tools to submit patches to code review.

When you live in the Guix world, you might be tempted to look into what depot_tools actually does, and to replicate its functionality in a more minimal, Guix-like way. Which, sure, perhaps this is a good approach for packaging V8 or Chromium or something, but when you want to work on V8, you need to learn some humility and just go with the flow. (It’s hard for the kind of person that uses Guix. But it’s what you do.)

You can make some small adaptations, though. depot_tools is mostly written in Python, and it actually bundles its own virtualenv support for using a specific python version. This isn’t strictly needed, so we can set the funny environment variable VPYTHON_BYPASS="manually managed python not supported by chrome operations" to just use python from the environment.

Sometimes depot_tools will want to run some prebuilt binaries. Usually on Guix this is anathema—we always build from source—but there’s only so much time in the day and the build system is not our circus, not our monkeys. So we get Guix to set up the environment using a container in --emulate-fhs mode; this lets us run third-party pre-build binaries. Note, these binaries are indeed free software! We can run them just fine if we trust Google, which you have to when working on V8.

no, really, get the code

Enough with the introduction. The first thing to do is to check out depot_tools.

mkdir src cd src git clone https://chromium.googlesource.com/chromium/tools/depot_tools.git

I’m assuming you have git in your Guix environment already.

Then you need to initialize depot_tools. For that you run a python script, which needs to run other binaries – so we need to make a specific environment in which it can run. This starts with a manifest of packages, is conventionally placed in a file named manifest.scm in the project’s working directory, though you don’t have one yet, so you can just write it into v8.scm or something anywhere:

(use-modules (guix packages) (gnu packages gcc)) (concatenate-manifests (list (specifications->manifest '( "bash" "binutils" "clang-toolchain" "coreutils" "diffutils" "findutils" "git" "glib" "glibc" "glibc-locales" "grep" "less" "ld-gold-wrapper" "make" "nss-certs" "nss-mdns" "openssh" "patch" "pkg-config" "procps" "python" "python-google-api-client" "python-httplib2" "python-pyparsing" "python-requests" "python-tzdata" "sed" "tar" "wget" "which" "xz" )) (packages->manifest `((,gcc "lib")))))

Then, you guix shell -m v8.scm. But you actually do more than that, because we need to set up a container so that we can expose a standard /lib, /bin, and so on:

guix shell --container --network \ --share=$XDG_RUNTIME_DIR --share=$HOME \ --preserve=TERM --preserve=SSH_AUTH_SOCK \ --emulate-fhs \ --manifest=v8.scm

Let’s go through these options one by one.

• --container: This is what lets us run pre-built binaries, because it uses Linux namespaces to remap the composed packages to /bin, /lib, and so on.

• --network: Depot tools are going to want to download things, so we give them net access.

• --share: By default, the container shares the current working directory with the “host”. But we need not only the checkout for V8 but also the sibling checkout for depot tools (more on this in a minute); let’s just share the whole home directory. Also, we share the /run/user/1000 directory, which is $XDG_RUNTIME_DIR, which lets us access the SSH agent, so we can check out over SSH.

• --preserve: By default, the container gets a pruned environment. This lets us pass some environment variables through.

• --emulate-fhs: The crucial piece that lets us bridge the gap between Guix and the world.

• --manifest: Here we specify the list of packages to use when composing the environment.

We can use short arguments to make this a bit less verbose:

guix shell -CNF --share=$XDG_RUNTIME_DIR --share=$HOME \ -ETERM -ESSH_AUTH_SOCK -m manifest.scm

I would like it if all of these arguments could somehow be optional, that I could get a bare guix shell invocation to just apply them, when run in this directory. Perhaps some day.

Running guix shell like this drops you into a terminal. So let’s initialize depot tools:

cd $HOME/src export VPYTHON_BYPASS="manually managed python not supported by chrome operations" export PATH=$HOME/src/depot_tools:$PATH export SSL_CERT_DIR=/etc/ssl/certs/ export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt gclient

This should download a bunch of things, I don’t know what. But at this point we’re ready to go:

fetch v8

This checks out V8, which is about 1.3 GB, and then probably about as much again in dependencies.

build v8

You can build V8 directly:

# note caveat below! cd v8 tools/dev/gm.py x64.release

This will build fine... and then fail to link. The precise reason is obscure to me: it would seem that by default, V8 uses a whole Debian sysroot for Some Noble Purpose, and ends up linking against it. But it compiles against system glibc, which seems to have replaced fcntl64 with a versioned symbol, or some such nonsense. It smells like V8 built against a too-new glibc and then failed trying to link to an old glibc.

To fix this, you need to go into the args.gn that was generated in out/x64.release and then add use_sysroot = false, so that it links to system glibc instead of the downloaded one.

echo 'use_sysroot = false' >> out/x64.release/args.gn tools/dev/gm.py x64.release

You probably want to put the commands needed to set up your environment into some shell scripts. For Guix you could make guix-env:

#!/bin/sh guix shell -CNF --share=$XDG_RUNTIME_DIR --share=$HOME \ -ETERM -ESSH_AUTH_SOCK -m manifest.scm -- "$@"

Then inside the container you need to set the PATH and such, so we could put this into the V8 checkout as env:

#!/bin/sh # Look for depot_tools in sibling directory. depot_tools=`cd $(dirname $0)/../depot_tools && pwd` export PATH=$depot_tools:$PATH export VPYTHON_BYPASS="manually managed python not supported by chrome operations" export SSL_CERT_DIR=/etc/ssl/certs/ export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt exec "$@"

This way you can run ./guix-env ./env tools/dev/gm.py x64.release and not have to “enter” the container so much.

notes

This all works fine enough, but I do have some meta-reflections.

I would prefer it if I didn’t have to use containers, for two main reasons. One is that the resulting build artifacts have to be run in the container, because they are dynamically linked to e.g. /lib, at least for the ELF loader. It would be better if I could run them on the host (with the host debugger, for example). Using Guix to make the container is better than e.g. docker, though, because I can ensure that the same tools are available in the guest as I use on the host. But also, I don’t like adding “modes” to my terminals: are you in or out of this or that environment. Being in a container is not like being in a vanilla guix shell, and that’s annoying.

The build process uses many downloaded tools and artifacts, including clang itself. This is a feature, in that I am using the same compiler that colleagues at Google use, which is important. But it’s also annoying and it would be nice if I could choose. (Having the same clang-format though is an absolute requirement.)

There are two tests failing, in this configuration. It is somehow related to time zones. I have no idea why, but I just ignore them.

If the build system were any weirder, I would think harder about maybe using Docker or something like that. Colleagues point to distrobox as being a useful wrapper. It is annoying though, because such a docker image becomes like a little stateful thing to do sysadmin work on, and I would like to avoid that if I can.

Welp, that’s all for today. Hopefully if you are contemplating installing Guix as your operating system (rather than just in user-space), this can give you a bit more information as to what it might mean when working on third-party projects. Happy hacking and until next time!

Another small feature update just in time for gnome 46.

Subscribe via CLI

Lets start with something that already went into version 3.1.4: you can subscribe to feeds via CLI now. The idea is that this is a building block for seamlessly subscribing to websites from within a browser or something similar. Lets see how this develops further.

Scrap all new Articles of a Feed

If Gitlab upvotes is a valid metric, this feature was the most requested one so far. Feed settings gained a new toggle to scrap the content of new articles. The sync will complete normally and in a second operation Newsflash tries to download the full content of all new articles in the background.

This is especially useful when there is no permanent internet connection. Now you can let Newsflash sync & download content while on WiFi and read the complete articles later even without an internet connection.

Update Feed URL

The local RSS backend gained the ability to update the URL where the feed is located (see the screenshot above). Sadly none of the other services support this via their APIs as far as I know.

Clean Database

The preferences dialog gained the ability to drop all old article and “vacuum” the database right away. Depending on the size of the database file this can take a few seconds, that’s why it is not done in the background during normal operations yet.

(btw: I’m not sure if I should keep the button as “destructive-action”) Internal Refactoring

Just a heads up that a lot of code managing the loading of the article list and keeping track of the displayed article and its state was refactored. If there are any regressions, please let me know.

Profiling

Christian Hergerts constant stream of profiling blog posts finally got to me. So I fired up sysprof. Fully expecting to not be knowledgeable enough to draw any meaningful conclusions from the data. After all, the app is pretty snappy on my machine ™, so any improvements must be hard to find and even harder to solve. But much to my surprise about 30 minutes later two absolutely noticeable low hanging fruit performance problems were discovered and fixed.

So I encourage everyone to just try profiling your code. You may be surprised what you find.

Adwaita Dialogs & Removing Configurable Shortcuts

Of course this release makes use of the new Adwaita Dialogs. For all the dialogs but one:

Configuring custom keybindings still spawns a new modal window. Multiple overlapping dialogs isn’t the greatest thing in the world. This and another annoying issue made me think about removing the feature from Newsflash completely.

The problem is that all shortcuts need to be disabled whenever the user is about to enter text. Otherwise the keybindings with a single letter cannot be entered as text.

All major feed readers (feedly, innoreader, etc) have a fixed set of cohesive keyboard shortcuts. I’ve been thinking about either having 2-3 shortcut configurations to choose from or just hard-coding keybindings all together.

I’d like to hear your thoughts. Do you use custom shortcuts? Would you be fine with a well thought out but hard-coded set of shortcuts? Would you prefer to choose from a few pre-defined shorcut configurations? Let me know, and help me find the best keybindings for all the actions that can be triggered via keyboard.