Planet GNOME

Misc news about the gedit text editor, mid-September edition! (Some sections are a bit technical).

Next version will be released when Ready

While the release of GNOME 49.0 was approaching (it's this week!), I came to the conclusion that it's best for gedit to wait more, and to follow the Debian way of releasing software: when it's Ready. "Ready" with an uppercase letter 'R'!

So the question is: what is not ready? Two main things:

• The rework of the file loading and saving: it is something that takes time, and I prefer to be sure that it'll be a solid solution.
• The question about the Python support for implementing plugins. Time will tell what is the answer.

Rework of the file loading and saving (next steps)

Work continues to refactor that part of the code, both in libgedit-gtksourceview and gedit.

I won't go into too much technical details this time. But what the previous developer (Ignacio Casal Quinteiro, aka nacho) wrote (in 2011) in a comment at the top of a class is "welcome to a really big headache."

And naturally, I want to improve the situation. For a long time this class was used as a black box, using only its interface. Time has come to change things! It takes time, but I already see the end of the tunnel and I have good hopes that the code will be better structured. I intend to write about it more once finished.

But I can reveal that there is already a visible improvement: loading a big file (e.g. 200 MB) is now super fast! Previously, it could take one minute to load such file, with a progress bar shown and a Cancel button. Now there is not enough time to even click on (or to see) the Cancel button! (I'm talking about local files, for remote files with a slow network connection, the progress bar is still useful).

To be continued...

If you appreciate the work that I do, you can send a thank-you donation. Your support is much appreciated! For years to come, it will be useful for the project.

Intro:

Hi everyone, it’s the end of GSoc! I had a great experience throughout this whole process. I’ve learned so much. This is essentially the ‘final report’ for GSoC, but not my final report for this project in general by a long shot. I still have so much more I want to do, but here is what I’ve done so far.

Project:

JSON, YAML, and/or XML emitting and parsing integration into Vala’s compiler.

Mentor:

I would like to thank Lorenz Wildberg for being my mentor for this project, as well as the Vala community.

Description:

The main objective of this project is to integrate direct syntax support for parsing and emitting JSON, XML, and/or YAML formats in Vala. This will cut back the boilerplate code, making it more user-friendly and efficient for developers working with these formatting languages.

What I’ve done: Research

• I’ve done significant research in both JSON and YAML parsing and emitting in various languages like C#, Java, Rust and Python.
• Looked into how Vala currently handles JSON using JSON GLib classes, and I then modelled the C code after the examples I collected.
• Modelled the JSON module after other modules in the codegen, specifically, mainly after Dbus, Gvariant, GObject, and GTK.

Custom JSON Overrides and Attribute

• Created Vala syntax sugar specifically making a [JSON] attribute to do serialization.
• Built support for custom overrides as in mapping JSON keys to differently named fields/properties.
• Reduced boilerplate by generating C code behind the scenes.

Structs

• I’ve created both Vala functions to deserialize and serialize structs using JSON boxed functions.
• I created a Vala generate_struct_serialize_func function to create a C code function called _%s_serialize_func to serialize fields.

• I then created a Vala function generate_struct_to_json to create a C code function called _json_%s_serialize_mystruct to fully serialize the struct by using boxed serialize functions.
  

• I created a Vala generate_struct_deserialize_func function to create a C code function called _%s_deserialize_func to deserialize fields.
• I then created a Vala function generate_struct_to_json to create a C code function called _json_%s_deserialize_mystruct to fully deserialize the struct by using boxed deserialized functions.

GObjects

• I’ve created both Vala functions to deserialize and serialize GObjects using json_gobject_serialize and JSON generator.

• I then created a Vala function generate_gclass_to_json to create a C code function called _json_%s_serialize_gobject_myclass to fully serialize GObjects.
  

• I created a Vala generate_gclass_from_json function to create a C code function called _json_%s_deserialize_class to deserialize fields.

Non-GObjects

• I’ve done serializing of non-GObjects using JSON GLib’s builder functions.
• I then created a Vala function generate_class_to_json to create a C code function called _json_%s_serialize_myclass to fully serialize non-objects that aren’t inheriting from Object or Json.Serializable.

Future Work: Research

• Research still needs to be put into integrating XML and determining which library to use.
• The integration of YAML and other formatting languages not only JSON, YAML, or XML.

Custom Overrides and Attributes

• I want to create more specialized attributes for JSON that only do serialization or deserialization. Such as [JsonDeserialize] and [JsonSerialize] or something similar.
• [JSON] attribute needs to do both deserializing and serializing, and at the moment, the deserializing code has problems.
• XML, YAML, and other formating languages will follow very similar attribute patterns: [Yaml], [Xml], [Json].

Bugs

• unref c code functions are calling nulls, which shouldn’t be the cause. They need proper types going through.
• Deserializing prompts a redefinition that needs to be corrected.
• Overridden GObject properties need to have setters made to be able to get the values.

Links

• Here is the JSON module that has majority of the code.
• Here is the merge request.

Another six months have passed, and with that comes another libadwaita release to go with GNOME 49.

This cycle doesn't have a lot of changes due to numerous IRL circumstances I've been dealing with, but let's look at them anyway.

Shortcuts dialog

Last cycle GTK deprecated GtkShortcutsWindow and all of the related classes. Unfortunately, this left it with no replacement, despite being widely used. So, now there is a replacement: AdwShortcutsDialog. Same as shortcuts window, it has very minimal API and is intended to be static and constructed from UI files.

Structure

While the new dialog has a similar feature set to the old one, it has a very different organization, and is not a drop-in replacement.

The old dialog was structured as: GtkShortcutsWindow → GtkShortcutsSection → GtkShortcutsGroup → GtkShortcutsShortcut.

Most apps only have a single shortcuts section, but those that have multiple would have them shown in a dropdown in the dialog's header bar, as seen in Builder:

Each section would have one or more shortcuts groups. When a section has too many groups, it would be paginated. Each group has a title and optionally a view, we'll talk about that a bit later.

Finally each groups contains shortcuts. Or shortcuts shortcuts, I suppose - which describe the actual shortcuts.

When sections and groups specify a view, the dialog can be launched while only showing a subset of shortcuts. This can be seen in Clocks, but was never very widely used. And specifically in Clocks it was also a bit silly, since the dialog actually becomes shorter when the button is clicked.

The new dialog drops the rarely used sections and views, so it has a simpler structure: AdwShortcutsDialog → AdwShortcutsSection → AdwShortcutsItem.

Sections here are closer to the old groups, but are slightly different. Their titles are optional, and sections without titles behave as if they were a part of the previous section with an extra gap. This allows to subdivide the sections further, without adding an extra level of hierarchy when it's not necessary.

Since shortcuts are shown as boxed lists, apps should avoid having too many in a single section. It was already not great with the old dialog, but is much worse in the new one.

Finally, AdwShortcutsItem is functionally identical to GtkShortcutsShortcut, except it doesn't support specifying gestures and icons.

Why not gestures?

This feature was always rather questionable, and sometimes doing more harm than good. For example, take these 2 apps - the old and the current image viewer, also known as Eye of GNOME and Loupe respectively:

Both of them specify a two-finger swipe left/right to go to the next/previous image. Well, does it work? The answer depends on what input device you're using.

In Loupe it will work on a touchpad, but not touchscreen: on a touchscreen you use one finger instead.

Meanwhile, in EoG it only works on touchscreen instead. On touchpad 2-finger swipe scrolls the current image if it's zoomed in.

So - while both of these apps have a swipe gesture, they are completely different - yet the dialog makes no distinction between them.

It's also not discoverable. HIG recommends naming the menu entry Keyboard Shortcuts, and it doesn't make a lot of sense that these gestures would be in there too - they have nothing to do with keyboard or shortcuts.

A much better place to document this would be help pages. And of course, ideally apps should have all of the typical gestures people are used to from other systems (pinch to zoom and rotate, double tap to zoom, swipes to navigate, long press to open context menus when it's not available via other means), and clear feedback while those gestures are performed - so that there's less of a need to remember which app has which gestures in the first place and they can be documented system-wide instead.

Why not icons?

As for icons, the only app I'm aware of that did this was gnome-games - it used them to show gamepad navigation:

This was problematic in a similar way, but also there was no way to open this dialog using a gamepad in the first place. A much better solution (and pretty much the standard for gamepad navigation) would have been always visible hints at the bottom of the window or inline.

Auto-loading

Most apps using GtkShortcutsWindow weren't creating it programmatically - GtkApplication loads it automatically and creates an action for it. So, we do the same thing: if a resource with the name shortcuts-dialog.ui is present in the resource base path, AdwApplication will create the app.shortcuts action which will create and show the dialog in the active window when activated.

Some apps were already using an action with this name, in these cases no action will be created.

One thing that's not possible anymore is overriding the dialog for specific windows (gtk_application_window_set_help_overlay()). This feature was extremely rarely used, and apps that really want different dialogs for different windows can just create the dialogs themselves instead of using auto-loading - this is just convenience API for the most common case.

Shortcut label

One of the widgets that was deprecated is GtkShortcutLabel. However, it had uses outside of the shortcuts dialog as well. So, libadwaita has a replacement as well - AdwShortcutLabel. Unlike the dialog itself, this is a direct fork of the GTK widget, and works the same way - though the separation between individual keycaps looks a bit different now, hopefully to make it clearer:

It also has a slightly different style, but it's been backported for GtkShortcutLabel as well for the most part.

And, unlike the shortcuts dialog, AdwShortcutLabel is a drop-in replacement.

CSS improvements Media queries

This cycle, GTK has added support for CSS media queries, allowing to define styles for light and dark, as well as regular and high contrast styles in the same file.

Media queries is fully supported on libadwaita side, and apps are encouraged to use them instead of style-dark.css, style-hc.css and style-hc-dark.css. Since this happened right at the end of the cycle (after the feature and API freeze, in fact, since GTK doesn't follow it), they are not deprecated just yet, but will be early next cycle.

Since we now have support for both variables and media queries, it's possible to do things like this now:

:root { --card-border: var(--card-shade-color); } @media (prefers-contrast: more) { :root { --card-border: var(--border-color); } } .card-separator { background: var(--card-border); } Typography

Last cycle, I added document and monospace font variables and mentioned that the document font may change in future to be distinct from the UI font.

This has happened now, and it is actually distinct - Adwaita Sans 12pt instead of 11pt.

So - to mirror .monospace, there's now a .document style class as well. It uses the document font, and also increases the line height for better readability.

Additionally, the formerly mostly useless .body style class increases line height as well now, instead of just setting the default font size and weight. Apps should use it when displaying medium-long text, and libadwaita is using it in a bunch of standard widgets, such as in preferences group and status page descriptions, alert dialog body, or various pages in the about dialog.

Fractal and Podcasts are already making use of both, and hopefully soon more apps will follow suit.

Other changes

• AdwPreferencesGroup can now be used with list models via adw_preferences_group_bind_model().

• For convenience, it also allows adding rows that aren't AdwPreferencesRow to the internal list now, rather than treating them as other widgets. These rows will just not be searchable in AdwPreferencesDialog.

• AdwPreferencesPage has a way to insert groups anywhere now, rather than just append them at the end.

• Both AdwPreferencesGroup and AdwPreferencesPage now have a way to inspect their groups/rows without peeking into their implementation: adw_preferences_page_get_group() and adw_preferences_group_get_row().

• AdwWrapBox now has a way to remove all children at once.

• GtkFlowBox children have hover and active styles by default. Apps that use flowboxes in non-standard ways may need to disable this from CSS in some cases.

• AdwHeaderBar now supports native window controls on macOS, same as GtkHeaderBar from the last cycle.

• Window, dialog and sheet shadows have been reduced to increase performance.

Future

While this cycle was pretty short and unexciting, there's a thing in works for the next cycle.

One of the most glaring omissions right now is sidebars. While we have split views, we don't have anything pre-built that could go into the sidebar pane - it's up to the apps to invent something using GtkListBox or GtkListView, combined with the .navigation-sidebar style class.

This is a lot messier than it may seem, and results in every app having sidebars that look and behave slightly different. We have helpers for boxed lists, so why not sidebars too?

There is also GtkStackSidebar, but it's not flexible at all and doesn't play well with mobile phones.

Additionally, on mobile especially sidebars look and behave extremely out of place, and it would be nice to do something about - e.g. use boxed lists instead.

So, next cycle we'll (hopefully) have both a generic sidebar widget, and a stack sidebar replacement. They won't cover all of the use cases (I expect it to be useful for Builder's preferences dialog but not the main window), but a lot of apps don't do anything extraordinary and it should save them a lot of effort.

Thanks to the GNOME STF Team for providing the funding for this work. Also thanks to the GNOME Foundation for their support and thanks to all the contributors who made this release possible.

Introduction

Python-BPF offers a new way to write eBPF programs entirely in Python, compiling them into real object files. This project is open-source and available on GitHub and PyPI. I wrote it alongside R41k0u.

Update: This article has now taken off on Hacker News.

Published Library with Future Plans

Python-BPF is a published Python library with plans for further development towards production-ready use.
You can pip install pythonbpf but it’s certainly not at all production ready and the code is hacky at best with more bugs than I could count. (This was a hackathon project afterall. We plan to fix it after we are done with the hackathon.)

The Old Way: Before Python-BPF

Before Python-BPF, writing eBPF programs in Python typically involved embedding C code within multiline strings, often using libraries like bcc. eBPF allows for small programs to run based on kernel events, similar to kernel modules.

Here’s an example of how it used to be:

from bcc import BPF from bcc.utils import printb # define BPF program prog = """ int hello(void *ctx) { bpf_trace_printk("Hello, World!\\n"); return 0; } """ # load BPF program b = BPF(text=prog) b.attach_kprobe(event=b.get_syscall_fnname("clone"), fn_name="hello") # header print("%-18s %-16s %-6s %s" % ("TIME(s)", "COMM", "PID", "MESSAGE")) # format output while 1: try: (task, pid, cpu, flags, ts, msg) = b.trace_fields() except ValueError: continue except KeyboardInterrupt: exit() printb(b"%-18.9f %-16s %-6d %s" % (ts, task, pid, msg))

This approach, while functional, meant writing C code within Python, lacking support from modern Python development tools like linters.

Features of the Multiline C Program Approach # load BPF program b = BPF(text=""" #include <uapi/linux/ptrace.h> BPF_HASH(last); int do_trace(struct pt_regs *ctx) { u64 ts, *tsp, delta, key = 0; // attempt to read stored timestamp tsp = last.lookup(&key); if (tsp != NULL) { delta = bpf_ktime_get_ns() - *tsp; if (delta < 1000000000) { // output if time is less than 1 second bpf_trace_printk("%d\\n", delta / 1000000); } last.delete(&key); } // update stored timestamp ts = bpf_ktime_get_ns(); last.update(&key, &ts); return 0; } """)

The multiline C program approach allowed for features like BPF MAPS (hashmap type), map lookup, update, and delete, BPF helper functions (e.g., bpf_ktime_get_ns, bpf_printk), control flow, assignment, binary operations, sections, and tracepoints.

Similar Program in Reduced C

For production environments, eBPF programs are typically written in pure C, compiled by clang into a bpf target object file, and loaded into the kernel with tools like libbpf. This approach features map sections, license global variables, and section macros specifying tracepoints.

#include <linux/bpf.h> #include <bpf/bpf_helpers.h> #define u64 unsigned long long #define u32 unsigned int struct { __uint(type, BPF_MAP_TYPE_HASH); __uint(max_entries, 1); __type(key, u32); __type(value, u64); } last SEC(".maps"); SEC("tracepoint/syscalls/sys_enter_execve") int hello(struct pt_regs *ctx) { bpf_printk("Hello, World!\\n"); return 0; } char LICENSE[] SEC("license") = "GPL"; Finally! Python-BPF

Python-BPF brings the true eBPF experience to Python by allowing the exact same functionality to be replaced by valid Python code. This is a significant improvement over multiline C strings, offering support from existing Python tools.

from pythonbpf import bpf, map, section, bpfglobal, compile from ctypes import c_void_p, c_int64, c_int32, c_uint64 from pythonbpf.helpers import ktime from pythonbpf.maps import HashMap @bpf @map def last() -> HashMap: return HashMap(key_type=c_uint64, value_type=c_uint64, max_entries=1) @bpf @section("tracepoint/syscalls/sys_enter_execve") def hello(ctx: c_void_p) -> c_int32: print("entered") return c_int32(0) @bpf @section("tracepoint/syscalls/sys_exit_execve") def hello_again(ctx: c_void_p) -> c_int64: print("exited") key = 0 last().update(key) ts = ktime() return c_int64(0) @bpf @bpfglobal def LICENSE() -> str: return "GPL" compile()

Python-BPF uses ctypes to preserve compatibility, employs decorators to separate the BPF program from other Python code, allows intuitive creation of global variables, and defines sections and tracepoints similar to its C counterpart. It also provides an interface to compile and run in the same file.

How it Works Under the Hood

1. Step 1: Generate AST The Python ast module is used to generate the Abstract Syntax Tree (AST).

2. Step 2: Emit LLVM IR llvmlite from Numba emits LLVM Intermediate Representation (IR) and debug information for specific parts like BPF MAPs. The .py file is converted into LLVM Intermediate Representation.

3. Step 3: Compile LLVM IR The .ll file, containing all code written under the @bpf decorator, is compiled using llc -march=bpf -O2.

Salient Features

Previous Python options for eBPF relied on bcc for compilation, which is not ideal for production use. The only two real options for production-quality eBPF programs were aya in Rust and Clang with kernel headers in C. Python-BPF introduces a third, new option, expanding the horizons for eBPF development.

It currently supports:

• Control flow
• Hash maps (with plans to add support for other map types)
• Binary operations
• Helper functions for map manipulation
• Kernel trace printing functions
• Timestamp helpers
• Global variables (implemented as maps internally with syntactical differences)

TL;DR

• Python-BPF allows writing eBPF programs directly in Python.
• This library compiles Python eBPF code into actual object files.
• Previously, eBPF programs in Python were written as C code strings.
• Python-BPF simplifies eBPF development with Python decorators.
• It offers a new option for production quality BPF programs in Python.
• The tool supports BPF maps, helper functions, and control flow, with plans to extend to completeness later.

Thanks for reading my poorly written blog :)