Planet GNOME

The second month of Google Summer of Code passed quickly. Last weeks I’ve been working on my markers code. My early implementation, while functional, needed a lot of cleaning, refactoring and refining to fit into Pitivi. Mathieu Duponchell and Alexandru Băluț have been guiding me through this process.

In GES I expanded the GESMarkerList with new signals, writed new tests and changed some unusual structures for others more usual in GES.

In Pitivi I added a new module with the markers logic, ‘markers.py’. Roughly speaking, now we have the class MarkersBox, which is a GTK.EventBox containing a GESMarkerList and a GTK.Layout to put on markers. The class Marker is also a GTK.EventBox, so we have a widget for every GESMarker, which allows to move, remove and select markers. The class MarkerPopover brings a popover menu to edit metadata in every marker. I also implemented undo and redo actions.

The process of rewriting a lot of my previous code has been hard and challenging. I knew that my original code wasn’t clear or optimized but I wasn’t sure how to exactly improve it. It implied to learn and apply some concepts which wasn’t clear to me. While hard work it felt as a rewarding and foundamental learning.

For future GSoC students I would like to recap some of the things and concepts I have learned or worked on with Pitivi. Maybe this would be useful for someone:

• MVC:

Pitivi follows the Model-View-Controller pattern. In my early code I was binding together the View and Controller parts. For example, removing a marker: first I called the method to remove a marker and after that I had a line of code to redraw the removed marker. This would make impossible to use other controllers.

To change this I had to implement new signals in GESMarkerList, for removed and moved markers. Then in Pitivi I wrote handlers for these signals which updates the View. So, every time the Model is updated with a ‘remove marker’ it triggers a remote signal and the handler updates the view.

• UNDO/REDO ACTIONS:

Undo/Redo actions are alternative controllers. Alternative controllers need the MVC. Without the signals that I wrote previously my undo/redo actions couldn’t work. The class MarkerObserver monitorize what happen with markers. It have handlers for every GESMarkerList signal. With every signal (add, remove and move) a new action is created and added to an action log. Every action have a method to implement undo and redo.

One complicate thing was that when a marker is removed and the user wants to undo that action, a new marker has to be created. This brings a new problem, if the user wants to undo to previous actions the reference to the original marker get lost, and previous actions in the stack don’t know about the last marker created to substitute the original marker.

Lucky me, Aleb guided me to the UndoableAutomaticObjectAction class who takes care of these situations, but it took me a while to discover how it works.

• GTK:

Pitivi uses GTK, a toolkit for creating graphical user interfaces. And not everything is evident in GTK documentation.

To display markers we chose to use CSS styles. I wanted that the interface of the markers change while they are hovered or selected. But it was hard to make it work in EventBoxes. While in GTK.Button it works just right, it took me time to discover why that wasn’t working in our case. After some talk in the GTK channel we discovered that we needed to manually update the flags state in our widget:

def do_enter_notify_event(self, unused_event): self.set_state_flags(Gtk.StateFlags.PRELIGHT, clear=False)

def do_leave_notify_event(self, unused_event): self.unset_state_flags(Gtk.StateFlags.PRELIGHT)

Slightly out of context but this code made our CSS work

• TIME AND COMMUNICATION:

These things took me more time that I would like. Being a beginner programmer isn’t easy, there are multiple factors which can block you. It’s hard to know if the thing that is blocking you is a trivial one or a hard one. Also, the desire of resolve all the problems by my own often result in a slower and inefficient pace. In my case all my problems got a new light when I talked with my menthors.

• THE DEVIL IS IN THE DETAILS:

When I read quickly through the code I have a loose knowledge of what it does. Stackoverflow gives me the false impression that I understand everything in a hurry. Most of the time these approaches drive me to get stuck.

If I read carefully, line by line, taking my time to understand, to explain things to myself, to make questions, everything starts to make sense and flow. The same happens when I try to go for a big task without subtasking. And I noted that when I have a good understanding of the little parts is easier to refine the code later.

For all you terminal graphics connoisseurs out there (there must be dozens of us!), I released Chafa 1.2.0 this weekend. Thanks to embedded copies of some parallel image scaling code and the quite excellent libnsgif, it’s faster and better in every way. What’s more, there are exciting new dithering knobs to further mangle refine your beautiful pictures. You can see what this stuff looks like in the gallery.

Included is also a Python program by Mo Zhou that uses k-means clustering to produce optimal glyph sets from training data. Neat!

Thanks to all the packagers, unsung heroes of the F/OSS world. Shoutouts go to Michael Vetter (openSUSE) and Guy Fleury Iteriteka (Guix) who got in touch with package info and installation instructions.

The full release notes are on GitHub.

What’s next

I’ve been asked about sixel support and some kind of interactive mode. I think both are in the cards… In the meantime, here’s a butterfly¹.

¹ Original via… Twitter? Tumblr? Imgur? Gfycat? I honestly can’t remember.

Petr and I had an hour to kill on the final evening of the West Coast Hackfest. He’d heard about a local statue that needed to be seen (and photographed to prove we were there). Some research showed that it was a few blocks from the hotel, just over 200 m — the downtown blocks are 61 m on a side. In fact, we must have passed it every morning on the way to the bus stop.

We arrived at the site to find The Portland Building undergoing substantial renovations, the reason we hadn’t paid it much attention. The sidewalk was covered by scaffolding and the view was barred by construction hoarding. Some informative signs attached to the hoarding showed the Portlandia story: photographs of the statue on a barge on the river, its original unveiling, and an architect’s rendering of the finished renovation. However, a peek inside yielded no clue as to where the statue might have stood before the renovations began. The Wikipedia entry later told us that the statue’s location was visible as a protective wrap partway up the front of the building: we’d passed it every morning without noticing.

Aftermath of Scooperfest.

We went looking for another photogenic landmark, and wound up in the Pioneer Courthouse Square as Scooperfest was winding down. Later on our way for supper we passed an interesting column at the Friday evening workspace, Powell’s City of Books.

Column at alternate Powell’s entrance.

I just uploaded the sysprof-3.33.4 tarball as we progress towards 3.34. This alpha release has some interesting new features that some of you may find interesting as you continue your quests to improve the performance of your system by improving the software running upon it.

For a while, I’ve been wondering about various ways to move GtkTextView forward in GTK 4. It’s of particular interest to me because I spent some time in the GTK 3 days making it faster for smooth scrolling. However, the designs that were employed there work better on the traditional Xorg setup than they do on GTK 3’s Wayland backend. Now that GTK 4 can have a proper GL pipeline, there is a lot of room for improvement.

Thanks to the West Coast Hackfest, I had a chance to sit down with Matthias and work through that design. GtkLabel was already using some accelerated text rendering so we started by making that work for GtkTextView. Then we extended the GSK PangoRenderer to handle the rest of the needs of GtkTextView and Matthias re-implemented some features to avoid cairo fallbacks.

After the hackfest I also found time to implement layout caching of PangoLayout. It helps reduce some of the CPU overhead in calculating line layouts.

As we start using the GPU more it becomes increasingly important to keep the CPU usage low. If we don’t it’s very likely to raise overall energy usage. To help keep us honest, I’ve added some RAPL energy statistics to Sysprof.

The Sprint series comes out every 3 weeks or so. Focus will be on the apps I maintain (Calendar, To Do, and Settings), but it may also include other applications that I contribute to.

GNOME Calendar: a new web calendar discoverer & optimizations

After a fairly big push to reimplement the web calendar discoverer code, it landed in Calendar! The new code is a threaded implementation of a web discoverer where we first ping the server to see if the passed URL is an actual file; otherwise, we perform a full CalDAV discovery on the URL.

Credentials are handled automatically — if the server rejects either the file or CalDAV checks due to permission, the user is asked about it.

In addition to that, the Year view is now much optimized and we avoid a big amount of D-Bus traffic by caching the events that appear in the sidebar.

GNOME To Do: minor cleanups

Not a lot to report on To Do. The week was dedicated to fixing a few crashers and warnings. Mostly boring stuff.

GNOME Settings: lots of code reviews, adaptative improvements

The focus of the Settings week was to get the merge request backlog under control. I do not personally enjoy seeing a number bigger than 50 merge requests against it. So I set the entire week to get some reviews.

Most of the merge requests were polishments to Purism’s push towards an adaptative Settings — we should see many improvements in that front.

I think this is new, so I’ll describe a new technique used in GXml to parse a large set of nodes in an XML document.

Parsing Large XML documents

If you have a large XML document, with a root with a number of child nodes, the standard technique is read all of them, including the child’s children ones, to create the XML tree. This process can take a while.

New on-the-fly parser

GXml now has a new custom parser called StreamReader used to read the root element and its children, but without any attribute and without any child’s children; the attributes and the children’s children are stored in a string on-the-fly in order to read the document almost at the same time it is read from the IO stream, for the root and for each children, improving the loading time of large XML documents up to 400% times faster than the previous technique already present in GXml.

On-the-fly-post-parsing

By using this On-the-fly-post-parsing technique, You can’t access the child’s children or the root’s attributes immediately after first read, you have to parse it from a temporally location in the GXml.Element class, using the new GXml.Element.parse_buffer() method, this one use the standard method, already present in GXml, to parse the root’s properties and the children’s children. When GXml.Element.parse_buffer() is called over the root, all children’s children are parsed recursively, but you can choose to parse just one of the root’s child, making a really convenient technique when you need just one root’s child node in a large XML document.

Multi-threading parsing

Currently GXml.Element.parse_buffer_async(), when called on root’s element, uses GLib.ThreadPool to parse each child in a different thread each and uses as many as threads are usable (less one) in your system. The expected behavior is getting a parse boost over the standard technique using in GXml: Xml.TextReader from the veteran libxml2 library running over just one thread. Currently a standard time parsing is provided when GXml.Element.parse_buffer_async() is called on document’s root, this maybe is a limitation on libxml2, because we have lot of Xml.TextReader running at the same time parsing element’s children; or a limitation on GLib.ThreadPool. Maybe the solution is a step away.

Zoom had a vulnerability that allowed users on MacOS to be connected to a video conference with their webcam active simply by visiting an appropriately crafted page. Zoom's response has largely been to argue that:

a) There's a setting you can toggle to disable the webcam being on by default, so this isn't a big deal,
b) When Safari added a security feature requiring that users explicitly agree to launch Zoom, this created a poor user experience and so they were justified in working around this (and so introducing the vulnerability), and,
c) The submitter asked whether Zoom would pay them for disclosing the bug, and when Zoom said they'd only do so if the submitter signed an NDA, they declined.

(a) and (b) are clearly ludicrous arguments, but (c) is the interesting one. Zoom go on to mention that they disagreed with the severity of the issue, and in the end decided not to change how their software worked. If the submitter had agreed to the terms of the NDA, then Zoom's decision that this was a low severity issue would have led to them being given a small amount of money and never being allowed to talk about the vulnerability. Since Zoom apparently have no intention of fixing it, we'd presumably never have heard about it. Users would have been less informed, and the world would have been a less secure place.

The point of bug bounties is to provide people with an additional incentive to disclose security issues to companies. But what incentive are they offering? Well, that depends on who you are. For many people, the amount of money offered by bug bounty programs is meaningful, and agreeing to sign an NDA is worth it. For others, the ability to publicly talk about the issue is worth more than whatever the bounty may award - being able to give a presentation on the vulnerability at a high profile conference may be enough to get you a significantly better paying job. Others may be unwilling to sign an NDA on principle, refusing to trust that the company will ever disclose the issue or fix the vulnerability. And finally there are people who can't sign such an NDA - they may have discovered the issue on work time, and employer policies may prohibit them doing so.

Zoom are correct that it's not unusual for bug bounty programs to require NDAs. But when they talk about this being an industry standard, they come awfully close to suggesting that the submitter did something unusual or unreasonable in rejecting their bounty terms. When someone lets you know about a vulnerability, they're giving you an opportunity to have the issue fixed before the public knows about it. They've done something they didn't need to do - they could have just publicly disclosed it immediately, causing significant damage to your reputation and potentially putting your customers at risk. They could potentially have sold the information to a third party. But they didn't - they came to you first. If you want to offer them money in order to encourage them (and others) to do the same in future, then that's great. If you want to tie strings to that money, that's a choice you can make - but there's no reason for them to agree to those strings, and if they choose not to then you don't get to complain about that afterwards. And if they make it clear at the time of submission that they intend to publicly disclose the issue after 90 days, then they're acting in accordance with widely accepted norms. If you're not able to fix an issue within 90 days, that's very much your problem.

If your bug bounty requires people sign an NDA, you should think about why. If it's so you can control disclosure and delay things beyond 90 days (and potentially never disclose at all), look at whether the amount of money you're offering for that is anywhere near commensurate with the value the submitter could otherwise gain from the information and compare that to the reputational damage you'll take from people deciding that it's not worth it and just disclosing unilaterally. And, seriously, never ask for an NDA before you're committing to a specific $ amount - it's never reasonable to ask that someone sign away their rights without knowing exactly what they're getting in return.

tl;dr - a bug bounty should only be one component of your vulnerability reporting process. You need to be prepared for people to decline any restrictions you wish to place on them, and you need to be prepared for them to disclose on the date they initially proposed. If they give you 90 days, that's entirely within industry norms. Remember that a bargain is being struck here - you offering money isn't being generous, it's you attempting to provide an incentive for people to help you improve your security. If you're asking people to give up more than you're offering in return, don't be surprised if they say no.

comments

One more API in this mini-series! g_assert_finalize_object(), which is available in GLib 2.61.2, which was released today.

This one’s useful when writing tests (and only when writing tests). It’s been put together by Simon McVittie to implement the common pattern needed in tests, where you want to unref a GObject and assert that you just dropped the final reference to the object — i.e., check that no references to the object have been leaked in the test.

Use it in place of g_object_unref(). If G_DISABLE_ASSERT is defined, it will actually just be a call to g_object_unref().

Here’s an example usage of it, straight out of the GLib unit test for it:

static void test_assert_finalize_object (void) { GObject *obj = g_object_new (G_TYPE_OBJECT, NULL); /* do some things with the obj here */ g_assert_finalize_object (obj); }

After a lot of Merge Requests related with MBIDS and AcoustID, finally I started working on acoustid plugin.

Before the logic was to return the recording with most sources. Now, we need to return multiple results. We need to retrieve first release belonging to each release group of each recording which matched with the given chromaprint.

Each chromaprint can get matched with multiple recordings and each recording can belong to many different release groups. Now each release group can have many releases, but releases within a release group only differ in track_count, dates or release country, so this difference between releases is not useful for a single song. Hence we can pick any release, so I went ahead with the first release.
Now the doubt continued, after implementing browse operation we realized, query operation would make more sense, because browse operation needs a top level container which can be browsed.

I also started with first version of tag editor. UI is not polished yet, but I could browse through various suggestions.

So in the last month, I’ve been working on MusicBrainz IDs retrieval and mappings. After that I spent some time trying to figure out how to start working on metadata retrieval. There’s so many pieces involved, after finally figuring out a rough plan, I started with the first piece of the puzzle. i.e AcoustID
AcoustID is a unique identifier which points to a specific entry in the AcoustID database. There are mainly two ways of accessing AcoustID database which are relevant to us, through fingerprint or through identifier. Former is more common, considering not all song files will have AcoustID stored. One good thing about my project is, once we retrieve chromaprint fingerprint or AcoustID or MBIDs, we are gonna make sure we writeback them to files, so that same calculations don’t have to be repeated.
Similar to how I extracted MBIDs from files based on mappings given by MusicBrainz, I had to extract AcoustID using mp3, vorbis and gstreamer extractors. Once again, GStreamer didn’t have support for GST tags for acoustid. So I had to update my old mr on GStreamer. Finally after few days, that MR got merged! It is quite an achievement!
I also discussed whether a new Grilo property is required for chromaprint fingerprint with Toso and Jean. They suggested that it’s better to keep chromaprint fingerprint within chromaprint plugin only and anyways, we can indirectly access them anywhere, there’s no need for them to be in core.
Other than this I also started working on browse operation to be able to return multiple results. I am not yet sure if browse or query operation makes more sense. Once I have first draft of browse operation, I will also start working on Tag Editor.

Most machine learning algorithms have been developed to perform classification or regression. However, in clinical research we often want to estimate the time to and event, such as death or recurrence of cancer, which leads to a special type of learning task that is distinct from classification and regression. This task is termed survival analysis, but is also referred to as time-to-event analysis or reliability analysis. Many machine learning algorithms have been adopted to perform survival analysis: Support Vector Machines, Random Forest, or Boosting. It has only been recently that survival analysis entered the era of deep learning, which is the focus of this post.

You will learn how to train a convolutional neural network to predict time to a (generated) event from MNIST images, using a loss function specific to survival analysis. The first part, will cover some basic terms and quantities used in survival analysis (feel free to skip this part if you are already familiar). In the second part, we will generate synthetic survival data from MNIST images and visualize it. In the third part, we will briefly revisit the most popular survival model of them all and learn how it can be used as a loss function for training a neural network. Finally, we put all the pieces together and train a convolutional neural network on MNIST and predict survival functions on the test data.

There is a Pango 1.44 release now. It contains all the changes I outlined recently. We also managed to sneak in a few features and fixes for longstanding bugs. That is the topic of this post.

Line breaking

One area for improvements in this release is line breaking.

Hyphenation

We don’t have TeX-style automatic hyphenation yet (although it may happen eventually). But at least, Pango inserts hyphens now when it breaks a line in the middle of a word (for example, at a soft hyphen character).

Example with soft hyphens

This is something i have wanted to do for a very long time, so I am quite happy that switching to harfbuzz for shaping on all platforms has finally enabled us to do this without too much effort.

Better line breaks

Pango follows Unicode UAX14 and UAX29 for finding word boundaries and line break opportunities.  The algorithm described in there is language-independent, but allows for language-specific tweaks. The Unicode standard calls this tailoring.

While Pango has had implementations for both the language-independent and -dependent parts before, we didn’t have them clearly separated in the API, until now.

In 1.44, we introduce a new pango_tailor_break() function which applies language-specific tweaks to a segment of text that has a uniform language. It is meant to be called after pango_default_break().

Line break control

Since my focus was on line-breaking already, I’ve added support for a text attribute to control line breaking. You can now say:

Don't break <span allow_break="false">here!</span>

in Pango markup, and Pango will obey.

In the hyphenation example above, the words showing possible hyphenation points (like im‧peachment) are marked up in this way.

Placement

Another area with significant changes is placement, both of lines and of individual glyphs.

Line height

Up to now, Pango has been placing the lines of a paragraph directly below each other, possibly with a fixed amount of spacing between them. While this works ok most of the time, a more typographically correct way to go about this is to control the baseline-to-baseline distance between lines.

Fonts contain a recommended value for this distance, so the first step was to make this value available with a new pango_font_metrics_get_height() API.

To make use of it, we added a new parameter to PangoLayout that tells it to place lines according to baseline-to-baseline distance. Once we had this, it was very easy to turn the parameter into a floating point number and allow things like double-spaced lines, by saying

pango_layout_set_line_spacing (layout, 2.0) Line spacing 1, 1.5, and 2

You can still use the old way of spacing if you set line-spacing to 0.

Subpixel positions

Pango no longer rounds glyph positions and font metrics to integral pixel numbers. This lets consumers of the formatted glyphs (basically, implementations of PangoRenderer) decide for themselves if they want to place glyphs at subpixel positions or pixel-aligned.

Non-integral extents

The cairo renderer in libpangocairo will do subpixel positioning, but you need cairo master for best results. GTK master will soon have the necessary changes to take advantage of it for its GL and Vulkan renderers too.

This is likely one of the more controversial changes in this release—any change to font rendering causes strong reactions. One of the reasons for doing the release now is that it gives us enough time to make sure it works ok for all users of Pango before going out in the next round of upstream and distro releases in the fall.

Visualization

Finally, I spent some time implementing  some long-requested features around missing glyphs, and their rendering as hex boxes. These are also known as tofu (which is the origin of the name for the Noto fonts – ‘no tofu’).

Invisible space

Some fonts don’t have a glyph for the space character – after all, there is nothing to draw. In the past, Pango would sometimes draw a hex box in this case. This is entirely unnecessary – we can just leave a gap of the right size and pretend that nothing happened.  Pango 1.44 will do just that: no more hex boxes for space.

Visible space

On the other hand, sometimes you do want to see where spaces and other whitespace characters such as tabs, are. We’ve added an attribute that lets you request visible rendering of whitespace:

<span show="spaces">Some space here</span> Visible space

This is implemented in the cairo backend, so you will need to use pangocairo to see it.

Special characters

In the same vein, sometimes it is helpful to see special characters such as left-to-right controls in the output.  Unicode calls these characters default-ignorable.

The show attribute also lets you make default-ignorables visible:

<span show=”ignorables”>Hidden treasures</span>

Visible default-ignorable characters

As you can see, we use nicknames for ignorables.

Font information

Pango has been shipping a simple tool called pango-list for a while. It produces a list of all the fonts Pango can find.  This can be very helpful in tracking down changes between systems that are caused by differences in the available fonts.

In 1.44, pango-list can optionally show font metrics and variation axes as well. This may be a little obsure, but it has helped me fix the CI tests for Pango.

Summary

This release contains a significant amount of change; I’ve closed a good number of ‘teenage’ bugs while working on it. Please let us know if you see problems or unexpected changes with it!

This release of scikit-survival adds support for scikit-learn 0.21 and pandas 0.24, among a couple of other smaller fixes. Please see the release notes for a full list of changes. If you are using scikit-survival in your research, you can now cite it using an Digital Object Identifier (DOI).