You are here

Planet Debian

Subscribe to Feed Planet Debian
Planet Debian - https://planet.debian.org/
Përditësimi: 7 months 34 min më parë

Michal Čihař: translation-finder 1.1

Mër, 20/03/2019 - 2:45md

The translation-finder module has been released in version 1.1. It is used by Weblate to detect translatable files in the repository making setup of translation components in Weblate much easier. This release brings lot of improvements based on feedback from our users, making the detection more reliable and accurate.

Full list of changes:

  • Improved detection of translation with full language code.
  • Improved detection of language code in directory and file name.
  • Improved detection of language code separated by full stop.
  • Added detection for app store metadata files.
  • Added detection for JSON files.
  • Ignore symlinks during discovery.
  • Improved detection of matching pot files in several corner cases.
  • Improved detection of monolingual Gettext.

Filed under: Debian English SUSE Weblate

Reproducible builds folks: Reproducible Builds: Weekly report #203

Mër, 20/03/2019 - 1:51md

Here’s what happened in the Reproducible Builds effort between Sunday March 10 and Saturday March 16 2019:

Don’t forget that Reproducible Builds is part of May/August 2019 round of Outreachy which offers paid internships to work on free software. Internships are open to applicants around the world and are paid a stipend for the three month internship with an additional travel stipend to attend conferences. So far, we received more than ten initial requests from candidates and the closing date for applicants is April 2nd. More information is available on the application page.

Packages reviewed and fixed, and bugs filed strip-nondeterminism

strip-nondeterminism is our tool that post-processes files to remove known non-deterministic output. This week, Chris Lamb:

Test framework development

We operate a comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This week, the following changes were made:

  • Alexander Couzens (OpenWrt support):
    • Correct the arguments for the reproducible_openwrt_package_parser script. []
    • Copy over Package-* files when building. []
    • Fix the Packages.manifest parser. [] []
  • Mattia Rizzolo:

This week’s edition was written by Arnout Engelen, Bernhard M. Wiedemann, Chris Lamb, Holger Levsen & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Jonathan Dowland: First successful Amiga disk-dumping session

Mër, 20/03/2019 - 12:04md

This is the seventh part in a series of blog posts. The previous post was Learning new things about my old Amiga A500.

X-COPY User Interface

Totoro Soot Sprites?

"Cyberpunk" party invitation

My childhood home

HeroQuest board game guide

I've finally dumped some of my Amiga floppies, and started to recover some old files! The approach I'm taking is to use the real Amiga to read the floppies (in the external floppy disk drive) and then copy them onto a virtual floppy disk image on the Gotek Floppy Emulator. I use X-COPY to perform the copy (much as I would have done back in 1992).

FlashFloppy's default mode of operation is to scan over the filesystem on the attached USB and assign a number to every disk image that it discovers (including those in sub-folders). If your Gotek device has the OLED display, then it reports the path to the disk image to you; but I have the simpler model that simply displays the currently selected disk slot number.

For the way I'm using it, its more basic "indexed" mode fits better: you name files in the root of the USB's filesystem using a sequential scheme starting at DSKA0000.ADF (which corresponds to slot 0) and it's then clear which image is active at any given time. I set up the banks with Workbench, X-COPY and a series of blank floppy disk images to receive the real contents, which I was able to generate using FS-UAE (they aren't just full of zeroes).

A few weeks ago I had a day off work and spent an hour in the morning dumping floppies. I managed to dump around 20 floppies successfully, with only a couple of unreadable disks (from my collection of 200). I've prioritised home-made disks, in particular ones that are likely to contain user-made content rather than just copies of commercial disks. But in some cases it's hard to know for sure what's on a disk, and sometimes I've made copies of e.g. Deluxe Paint and subsequently added home-made drawings on top.

Back on my laptop, FS-UAE can quite happily read the resulting disk images, and Deluxe Paint IV via FS-UAE can happily open the drawings that I've found (and it was a lot of fun to fire up DPaint for the first time in over 20 years. This was a really nice piece of software. I must have spent days of my youth exploring it).

I tried a handful of user-mode tools for reading the disk images (OFS format) but they all had problems. In the end I just used the Linux kernel's AFFS driver and loop-back mounts. (I could have looked at libguestfs instead).

To read Deluxe Paint image files on a modern Linux system one can use ImageMagick (via netpbm back-end) or ffmpeg. ffmpeg can also handle Deluxe Paint animation files, but more care is needed with these: It does not appear to correctly convert frame durations, setting the output animations to a constant 60fps. Given the input image format colour depth, it's tempting to output to animated GIF, rather than a lossy video compression format, but from limited experimentation it seems some nuances of the way that palettes are used in the source files are not handled optimally in the output either. More investigation here is required.

Enjoy a selection of my childhood drawings…

Jonathan Dowland: WadC 3.0

Mër, 20/03/2019 - 11:55pd

blockmap.wl being reloaded (click for animation)

A couple of weeks ago I release version 3.0 of Wad Compiler, a lazy functional programming language and IDE for the construction of Doom maps.

3.0 introduces more flexible randomness with rand; two new test maps (blockmap and bsp) that demonstrate approaches to random dungeon generation; some useful data structures in the library; better Hexen support and a bunch of other improvements.

Check the release notes for the full details, and check out the gallery of examples to see the kind of things you can do.

Version 3.0 of WadC is dedicated to Lu (1972-2019). RIP.

Neil McGovern: GNOME ED Update – February

Mar, 19/03/2019 - 11:43md

Another update is now due from what we’ve been doing at the Foundation, and we’ve been busy!

As you may have seen, we’ve hired three excellent people over the past couple of months. Kristi Progri has joined us as Program Coordinator, Bartłomiej Piorski as a devops sysadmin, and Emmanuele Bassi as our GTK Core developer. I hope to announce another new hire soon, so watch this space…

There’s been quite a lot of discussion around the Google API access, and GNOME Online Accounts. The latest update is that I submitted the application to Google to get GOA verified, and we’ve got a couple of things we’re working through to get this sorted.

Events all round!

Although the new year’s conference season is just kicking off, it’s been a busy one for GNOME already. We were at FOSDEM in Brussels where we had a large booth, selling t-shirts, hoodies and of course, the famous GNOME socks. I held a meeting of the Advisory Board, and we had a great GNOME Beers event – kindly sponsored by Codethink.

We also had a very successful GTK Hackfest – moving us one step closer to GTK 4.0.

Coming up, we’ll have a GNOME booth at:

  • SCALEx17 – Pasadena, California (7th – 10th March)
  • LibrePlanet – Boston Massachusetts (23rd – 24th March)
  • FOSS North – Gothenburg, Sweden (8th – 9th April)
  • Linux Fest North West – Bellingham, Washington (26th – 28th April)

If you’re at any of these, please come along and say hi! We’re also planning out events for the rest of the year. If anyone has any particularly exciting conferences we may not have heard of, please let us know.

Discourse

It hasn’t yet been announced, but we’re trialling an instance of Discourse for the GTK and Engagement teams. It’s hopeful that this may replace mailman, but we’re being quite careful to make sure that email integration continues to work. Expect more information about this in the coming month. If you want to go have a look, the instance is available at discourse.gnome.org

Keith Packard: metro-snek

Mar, 19/03/2019 - 9:08md
MetroSnek — snek on Metro M0 Express

When I first mentioned Snek a few months ago, Phillip Torrone from Adafruit pointed me at their Metro M0 board, which uses an Arduino-compatible layout but replaces the ATMega 328P with a SAMD21G18A. This chip is an ARM Cortex M0 part with 256kB of flash and 32kB of RAM. Such space!

Even though there is already a usable MicroPython port for this board, called CircuitPython, I figured it would be fun to get Snek running as well. The CircuitPython build nearly fills the chip, so the Circuit Python boards all include an off-chip flash part for storing applications. With Snek, there will be plenty of space inside the chip itself for source code, so one could build a cheaper/smaller version without the extra part.

UF2 Boot loader

I decided to leave the existing boot loader in place instead of replacing it with the AltOS version. This makes it easy to swap back to CircuitPython without needing any custom AltOS tools.

The Metro M0 Express boot loader is reached by pressing the reset button twice; it's pretty sweet in exposing a virtual storage device with a magic file, CURRENT.UF2, into which you write the ROM image. You write a UF2 formatted file to this name and the firmware extracts the data on the fly and updates the flash in the device. Very slick.

To make this work with AltOS, I had to adjust the start location of the operating system to 0x2000 and leave a bit of space at the end of ROM and RAM clear for the boot loader to use.

Porting AltOS

I already have an embedded operating system that works on Cortex M0 parts, AltOS, which I've been developing for nearly 10 years for use in rocketry and satellite applications. It's also what powers [ChaosKey])(http://altusmetrum.org/ChaosKey/).

Getting AltOS running on another Cortex M0 part is a simple matter of getting clocks running and writing drivers.

What I haven't really settled on is whether to leave this code as a part of AltOS, or to pull the necessary bits into the Snek repository and doing a bare-metal implementation.

I've set up the Snek distribution to make integrating it into another operating system simple; that's how the NuttX port works, for instance. It does make the build process more complicated as you have to build and install Snek, then build AltOS for the target device.

SAMD21 Clocks

Every SoC has a different way of configuring and wiring clocks within the system. Most that I've used have a complex clock-tree that you plug various configuration values into to generate clocks for the processor and peripherals.

The SAMD21 is simpler in offering a set of general-purpose clock controllers that can source a variety of clock signals and divide them by an integer. The processor uses clock controller 0; all of the other peripherals can be configured to use any clock controller you like.

The Metro M0 express and Feather M0 express have only a 32.768kHz crystal; they don't have a nice even-MHz crystal connected to the high-speed oscillator. As a result, to generate a '48MHz' clock for the processor and USB controller, I ended up multiplying the 32.768kHz frequency by 1464 using a PLL to generate a 47.972352MHz signal, which is about 0.06% low. Close enough for USB to work.

At first, I typo'd a register value leaving the PLL un-locked. The processor still ran fine, but when I looked at the clock with my oscilloscope, it was very ragged with a mean frequency around 30MHz. It took a few hours to track down the incorrect value, at which point the clock stabilized at about 48MHz.

SAMD21 USART

Next on the agenda was getting a USART to work; nothing terribly complicated there, aside from the clock problem mentioned above which generated a baud rate of around 6000 instead of 9600.

I like getting a USART working because it's usually (always?) easier than USB, plus demonstrates that clocking is working as expected. I can debug serial data with a simple logic analyzer. This time, the logic analyzer is how I discovered the clocking issue -- a bit time of 166µs does not equal 9600 baud.

SAMD21 USB

While I like having USB on-chip in the abstract, the concrete adventure of implementing USB for a new chip is always fraught with peril. In this case, the chip documentation was missing a couple of key details that I had to discover experimentally.

I'm still trying to come up with an abstraction for writing USB drivers for small systems; every one is different enough that I keep using copy&paste instead of building a driver core on top of hardware-specific primitives. In this case, the USB driver is 883 lines of code; the second shortest in AltOS with the ATMega32u4 driver being slightly smaller.

TODO

The only hardware that works today is one USARTs and USB. I also go Snek compiled and running. Left to do:

  • Digital GPIO controls. I've got basic GPIO functionality available in the underlying operating system, but it isn't exposed through Snek yet.

  • Analog outputs. This will involve hooking timers to outputs so that we can PWM them.

  • Analog inputs. That requires getting an ADC driver written and then hooking that into Snek.

  • On-board source storage. I think the ATMega model of storing just one set of source code on the device and reading that at boot time is clean and simple, so I want to do the same here. I think it will be simpler to use the on-chip flash instead of the external flash part. That means reserving a specific chunk of that for source code.

  • Figure out whether this code is part of AltOS, or part of Snek.

Links

Steinar H. Gunderson: When your profiler fools you

Mar, 19/03/2019 - 8:42pd

If you've been following my blog, you'll know about Nageru, my live video mixer, and Futatabi, my instant replay program with slow motion. Nageru and Futatabi both work on the principle that the GPU should be responsible for all the pixel pushing—it's just so much better suited than the CPU—but to do that, the GPU first needs to get at the data.

Thus, in Nageru, pushing the data from the video card to the GPU is one of the main CPU drivers. (The CPU also runs the UI, does audio processing, runs an embedded copy of Chromium if needed—we don't have full GPU acceleration there yet—and not the least encodes the finished video with x264 if you don't want to use Quick Sync for that.) It's a simple task; take two pre-generated OpenGL textures (luma and chroma) with an associated PBO, take the frame that the video capture card has DMAed into system RAM, and copy it while splitting luma from chroma. It goes about as fast as memory bandwidth will allow.

However, when you also want to run Futatabi, it runs on a different machine and also needs to get at the video somehow. Nageru solves this by asking Quick Sync (through VA-API) to encode it to JPEG, and then sending it over the network. Assuming your main GPU is an NVIDIA one (which gives oodles of more headroom for complicated things than the embedded Intel GPU), this means you now need an extra copy, and that's when things started to get hairy for me.

To simplify a long and confusing search, what I found was that with five inputs (three 1080p, two 720p), Nageru would be around two cores (200% CPU in top), and with six (an aditional 1080p), it would go to 300%. (This is on a six-core machine.) This made no sense, so I pulled up “perf record”, which said… nothing. The same routines and instructions were hot (mostly the memcpy into the NVIDIA GPU's buffers); there was no indication of anything new taking time. How could it be?

Eventually I looked at top instead of perf, and saw that some of the existing threads went from 10% to 30% CPU. In other words, everything just became uniformly slower. I had a hunch, and tried “perf stat -ddd“ instead. One value stood out:

3 587 503 LLC-load-misses # 94,72% of all LL-cache hits (71,91%)

Aha! It was somehow falling off a cliff in L3 cache usage. It's a bit like going out of RAM and starting to swap, just more subtle.

So, what was using so much extra cache? I'm sure someone extremely well-versed in perf or Intel VTune would figure out a way to look at all the memory traffic with PEBS or something, but I'm not a perf expert, so I went for the simpler choice: Remove things. E.g., if you change a memcpy to a memset(dst, 0, len), the GPU will be still be encoding a frame, and you'll still be writing things to memory, but you won't be reading anymore. I noticed a really confusing thing: Occasionally, when I did less work, CPU usage would be going up. E.g., have a system at 2.4 cores used, remove some stuff, go up to 3.1 cores! Even worse, occasionally after start, it would be at e.g. 2.0 cores, then ten seconds later, it would be at 3.0. I suspected overheating, but even stopping and waiting ten minutes would not be enough to get it reliably down to 2.0 again.

It eventually turned out (again through perf stat) that power saving was getting the better of me. I hadn't expected it to kick in, but seemingly it did, and top doesn't compensate in any way. Change from the default powersave governor to performance (I have electric heating anyway, and it's still winter up here), and tada... My original case was now down from 3.0 to 2.6 cores.

However, this still wasn't good enough, so I had to do some real (if simple) optimization. Some rearchitecting saved a copy; instead of the capture card receiver thread memcpy-ing into a buffer and the MJPEG encoder thread memcpy-ing it from there to the GPU's memory-mapped buffers, I let the capture card receiver memcpy it directly over instead. (This wasn't done originally due to some complications around locking and such; it took a while to find a reasonable architecture. Of course, when you look at the resulting patch, it doesn't look hard at all.)

This took me down from 3.0 to 2.1 cores for the six-camera case. Success! But still too much; the cliff was still there. It was pretty obvious that it was stalling on something in the six-camera case:

8 552 429 486 cycle_activity.stalls_mem_any # 5 inputs 17 082 914 261 cycle_activity.stalls_mem_any # 6 inputs

I didn't get a whole lot of more L3 misses, though; they seemed to just get much slower (occasionally as much as 1200 cycles on average, which is about four times as much as normal). Unfortunately, you have zero insight in what the Intel GPU is doing to your memory bus, so it's not easy to know if it's messing up your memory or what. Turning off the actual encoding didn't help much, though; it was really the memcpy into them that hurt. And since it was into uncached (write-combined) memory, doing things like non-temporal writes didn't help at all. Neither did combining the two copies into one loop (read once, write twice). Or really anything else I could think of.

Eventually someone on ##intel-media came up with a solution; instead of copying directly into the memory-mapped buffers (vaDeriveImage), copy into a VAImage in system RAM, and then let the GPU do the copy. For whatever reason, this helped a lot; down from 2.1 to 1.4 cores! And 0.6 of those cores were non-VA-API related things. So more patch and happiness all around.

At this point, I wanted to see how far I could go, so I borrowed more capture inputs and set up my not-so-trusty SDI matrix to create an eight-headed capture monster:

Seemingly after a little more tuning of freelist sizes and such, it could sustain eight 1080p59.94 MJPEG inputs, or 480 frames per second if you wanted to—at around three cores again. Now the profile was starting to look pretty different, too, so there were more optimization opportunities, resulting in this pull request (helping ~15% of a core). Also, setting up the command buffers for the GPU copy seemingly takes ~10% of a core now, but I couldn't find a good way of improving it. Most of the time now is spent in the original memcpy to NVIDIA buffers, and I don't think I can do much better than that without getting the capture card to peer-to-peer DMA directly into the GPU buffers (which is a premium feature you'll need to buy Quadro cards for, it seems). In any case, my original six-camera case now is a walk in the park (leaving CPU for a high-quality x264 encode), which was the goal of the exercise to begin with.

So, lesson learned: Sometimes, you need to look at the absolutes, because the relative times (which is what you usually want) can fool you.

Jonathan Carter: Running for DPL

Mar, 19/03/2019 - 4:30pd

I am running for Debian Project Leader, my official platform is published on the Debian website (currently looks a bit weird, but a fix is pending publication), with a more readable version available on my website as well as a plain-text version.

Shortly after I finished writing the first version of my platform page, I discovered an old talk from Ian Murdock at Microsoft Research where he said something that resonated well with me, and I think also my platform. Paraphrasing him:

You don’t want design by committee, but you want to tap in to the wisdom of the crowd. Lay out a vision for solving the problem, but be flexible enough to understand that you don’t have all the answers yourself, that other people are equally if not more intelligent than you are, and collectively, the crowd is the most intelligent of all.

– paraphrasing of Ian Murdock during a talk at Microsoft Research.

I’m feeling oddly calm about all of this and I like it. It has been a good exercise taking the time to read what everyone has said across all the mediums and trying to piece together all the pertinent parts and form a workable set of ideas.

I’m also glad that we have multiple candidates, I hope that it will inspire some thought, ideas and actions around the topics that occupy our collective head space.

The campaign period remains open until 2019-04-06. During this period, you can can ask me or any of the other candidates about how they envision their term as DPL.

There are 5 candidates in total, here’s the full list of candidates (in order of self-nomination with links to platforms as available):

Laura Arjona Reina: A weekend for the Debian website and friends

Hën, 18/03/2019 - 9:05md

Last weekend (15-17 March 2019) some members of the Debian web team have met at my place in Madrid to advance work together in what we call a Debian Sprint. A report will be published in the following days, but I want to say thank you to everybody that made possible this meeting happen.

We have shared 2-3 very nice days, we have agreed in many topics and started to design an new homepage focused in newcomers (since a Debianite usually just go to the subpage they need) and showing that Debian the operating system is made by a community of people. We are committed to simplify the content of and the structure of www.debian.org, we have fixed some bugs already, and talked about many other aspects. We shared some time to know each other, and I think all of us became motivated to continue working on the website (because there is still a lot of work to do!) and make easy for new contributors to get involved too.

For me, a person who rarely finds the way to attend Debian events, it was amazing to meet in person with people who I have shared work and mails and IRC chats since years in some cases, and to offer my place for the meeting. I have enjoyed a lot preparing all the logistics and I’m very happy that it went very well. Now I walk through my neighbourhood for my daily commute and every place is full of small memories of these days. Thanks, friends!

Dirk Eddelbuettel: RQuantLib 0.4.8: Small updates

Dje, 17/03/2019 - 8:12md

A new version 0.4.8 of RQuantLib reached CRAN and Debian. This release was triggered by a CRAN request for an update to the configure.ac script which was easy enough (and which, as it happens, did not result in changes in the configure script produced). I also belatedly updated the internals of RQuantLib to follow suit to an upstream change in QuantLib. We now seamlessly switch between shared_ptr<> from Boost and from C++11 – Luigi wrote about the how and why in an excellent blog post that is part of a larger (and also excellent) series of posts on QuantLib internals.

QuantLib is a very comprehensice free/open-source library for quantitative finance, and RQuantLib connects it to the R environment and language.

In other news, we finally have a macOS binary package on CRAN. After several rather frustrating months of inaction on the pull request put together to enable this, it finally happened last week. Yay. So CRAN currently has an 0.4.7 macOS binary and should get one based on this release shortly. With Windows restored with the 0.4.7 release, we are in the best shape we have been in years. Yay and three cheers for Open Source and open collaboration models!

The complete set of changes is listed below:

Changes in RQuantLib version 0.4.8 (2019-03-17)
  • Changes in RQuantLib code:

    • Source code supports Boost shared_ptr and C+11 shared_ptr via QuantLib::ext namespace like upstream.
  • Changes in RQuantLib build system:

    • The configure.ac file no longer upsets R CMD check; the change does not actually change configure.

Courtesy of CRANberries, there is also a diffstat report for the this release. As always, more detailed information is on the RQuantLib page. Questions, comments etc should go to the new rquantlib-devel mailing list. Issue tickets can be filed at the GitHub repo.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Emmanuel Kasper: Splitting a large mp3 / flac / ogg by detecting silence gaps

Dje, 17/03/2019 - 7:50md
If you have a large audio file coming for instance from a whole music album, the excellent mp3splt can do this for you:

mp3splt -o @n-@f -s my_long_file.mp3

will autodetect the silences, and create a list of tracks based on the large file.

mp3splt is  available in the Debian / Ubuntu archive.

Dirk Eddelbuettel: Rcpp 1.0.1: Updates

Dje, 17/03/2019 - 2:49md

Following up on the 10th anniversary and the 1.0.0. release, we excited to share the news of the first update release 1.0.1 of Rcpp. package turned ten on Monday—and we used to opportunity to mark the current version as 1.0.0! It arrived at CRAN overnight, Windows binaries have already been built and I will follow up shortly with the Debian binary.

We had four years of regular bi-monthly release leading up to 1.0.0, and having now taken four months since the big 1.0.0 one. Maybe three (or even just two) releases a year will establish itself a natural cadence. Time will tell.

Rcpp has become the most popular way of enhancing GNU R with C or C++ code. As of today, 1598 packages on CRAN depend on Rcpp for making analytical code go faster and further, along with 152 in BioConductor release 3.8. Per the (partial) logs of CRAN downloads, we currently average 921,000 downloads a month.

This release feature a number of different pull requests detailed below.

Changes in Rcpp version 1.0.1 (2019-03-17)
  • Changes in Rcpp API:

    • Subsetting is no longer limited by an integer range (William Nolan in #920 fixing #919).

    • Error messages from subsetting are now more informative (Qiang and Dirk).

    • Shelter increases count only on non-null objects (Dirk in #940 as suggested by Stepan Sindelar in #935).

    • AttributeProxy::set() and a few related setters get Shield<> to ensure rchk is happy (Romain in #947 fixing #946).

  • Changes in Rcpp Attributes:

    • A new plugin was added for C++20 (Dirk in #927)

    • Fixed an issue where 'stale' symbols could become registered in RcppExports.cpp, leading to linker errors and other related issues (Kevin in #939 fixing #733 and #934).

    • The wrapper macro gets an UNPROTECT to ensure rchk is happy (Romain in #949) fixing #948).

  • Changes in Rcpp Documentation:

    • Three small corrections were added in the 'Rcpp Quickref' vignette (Zhuoer Dong in #933 fixing #932).

    • The Rcpp-modules vignette now has documentation for .factory (Ralf Stubner in #938 fixing #937).

  • Changes in Rcpp Deployment:

    • Travis CI again reports to CodeCov.io (Dirk and Ralf Stubner in #942 fixing #941).

Thanks to CRANberries, you can also look at a diff to the previous release. Questions, comments etc should go to the rcpp-devel mailing list off the R-Forge page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

Jo Shields: Too many cores

Mër, 13/03/2019 - 3:48md
Arming yourself

ARM is important for us. It’s important for IOT scenarios, and it provides a reasonable proxy for phone platforms when it comes to developing runtime features.

We have big beefy ARM systems on-site at Microsoft labs, for building and testing Mono – previously 16 Softiron Overdrive 3000 systems with 8-core AMD Opteron A1170 CPUs, and our newest system in provisional production, 4 Huawei Taishan XR320 blades with 2×32-core HiSilicon Hi1616 CPUs.

The HiSilicon chips are, in our testing, a fair bit faster per-core than the AMD chips – a good 25-50%. Which begged the question “why are our Raspbian builds so much slower?”

Blowing a raspberry

Raspbian is the de-facto main OS for Raspberry Pi. It’s basically Debian hard-float ARM, rebuilt with compiler flags better suited to ARM11 76JZF-S (more precisely, the ARMv6 architecture, whereas Debian targets ARMv7). The Raspberry Pi is hugely popular, and it is important for us to be able to offer packages optimized for use on Raspberry Pi.

But the Pi hardware is also slow and horrible to use for continuous integration (especially the SD-card storage, which can be burned through very quickly, causing maintenance headaches), so we do our Raspbian builds on our big beefy ARM64 rack-mount servers, in chroots. You can easily do this yourself – just grab the raspbian-archive-keyring package from the Raspbian archive, and pass the Raspbian mirror to debootstrap/pbuilder/cowbuilder instead of the Debian mirror.

These builds have always been much slower than all our Debian/Ubuntu ARM builds (v5 soft float, v7 hard float, aarch64), but on the new Huawei machines, the difference became much more stark – the same commit, on the same server, took 1h17 to build .debs for Ubuntu 16.04 armhf, and 9h24 for Raspbian 9. On the old Softiron hardware, Raspbian builds would rarely exceed 6h (which is still outrageously slow, but less so). Why would the new servers be worse, but only for Raspbian? Something to do with handwavey optimizations in Raspbian? No, actually.

When is a superset not a superset

Common wisdom says ARM architecture versions add new instructions, but can still run code for older versions. This is, broadly, true. However, there are a few cases where deprecated instructions become missing instructions, and continuity demands those instructions be caught by the kernel, and emulated. Specifically, three things are missing in ARMv8 hardware – SWP (swap data between registers and memory), SETEND (set the endianness bit in the CPSR), and CP15 memory barriers (a feature of a long-gone control co-processor). You can turn these features on via abi.cp15_barrier, abi.setend, and abi.swp sysctl flags, whereupon the kernel fakes those instructions as required (rather than throwing SIGILL).

CP15 memory barrier emulation is slow. My friend Vince Sanders, who helped with some of this analysis, suggested a cost of order 1000 cycles per emulated call. How many was I looking at? According to dmesg, about a million per second.

But it’s worse than that – CP15 memory barriers affect the whole system. Vince’s proposal was that the HiSilicon chips were performing so much worse than the AMD ones, because I had 64 cores not 8 – and that I could improve performance by running a VM, with only one core in it (so CP15 calls inside that environment would only affect the entire VM, not the rest of the computer).

Escape from the Pie Folk

I already had libvirtd running on all my ARM machines, from a previous fit of “hey one day this might be useful” – and as it happened, it was. I had to grab a qemu-efi-aarch64 package, containing a firmware, but otherwise I was easily able to connect to the system via virt-manager on my desktop, and get to work setting up a VM. virt-manager has vastly improved its support for non-x86 since I last used it (once upon a time it just wouldn’t boot systems without a graphics card), but I was easily able to boot an Ubuntu 18.04 arm64 install CD and interact with it over serial just as easily as via emulated GPU.

Because I’m an idiot, I then wasted my time making a Raspbian stock image bootable in this environment (Debian kernel, grub-efi-arm64, battling file-size constraints with the tiny /boot, etc) – stuff I would not repeat. Since in the end I just wanted to be as near to our “real” environment as possible, meaning using pbuilder, this simply wasn’t a needed step. The VM’s host OS didn’t need to be Raspbian.

Point is, though, I got my 1-core VM going, and fed a Mono source package to it.

Time taken? 3h40 – whereas the same commit on the 64-core host took over 9 hours. The “use a single core” hypothesis more than proven.

Next steps

The gains here are obvious enough that I need to look at deploying the solution non-experimentally as soon as possible. The best approach to doing so is the bit I haven’t worked out yet. Raspbian workloads are probably at the pivot point between “I should find some amazing way to automate this” and “automation is a waste of time, it’s quicker to set it up by hand”

Many thanks to the #debian-uk community for their curiosity and suggestions with this experiment!

Reproducible builds folks: Reproducible Builds: Weekly report #202

Mër, 13/03/2019 - 3:24md

Here’s what happened in the Reproducible Builds effort between Sunday March 3 and Saturday March 9 2019:

diffoscope development

diffoscope is our in-depth “diff-on-steroids” utility which helps us diagnose reproducibility issues in packages. This week:

Chris Lamb uploaded version 113 to Debian unstable fixing a long list of issues. It included contributions already covered in previous weeks as well as new ones by Chris, including:

  • Provide explicit help when the libarchive system package is missing or “incomplete”. (#50)
  • Explicitly mention when the guestfs module is missing at runtime and we are falling back to a binary diff. (#45)

Vagrant Cascadian made the corresponding update to GNU Guix. []

Packages reviewed and fixed, and bugs filed Test framework development

We operate a comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This week, Holger Levsen made the following improvements:

  • Analyse node maintenance job runs to determine whether to mark nodes offline. []
  • Detect hanging health check runs, not just failed ones. []
  • Allow members of the jenkins UNIX group to sudo(8) to the jenkins user [] and simplify adding users to said group [].
  • Improve the “SHA1 checker” script to deal with packages with more than one version [] and to re-download buildinfo.debian.net’s files if they are older than two weeks. []
  • Node maintenance. [][][][]
  • In the version checker, correctly deal with a rare situation when several, say, diffoscope versions are available in one Debian suite at the same time. []

In addition, Alexander “lynxis” Couzens, made a number of changes to our OpenWrt support, including:

  • Add OpenWrt support to our database. []
  • Adding a reproducible_openwrt_package_parser.py script. []
  • Strip unreproducible certificates from images. []
Outreachy

Don’t forget that Reproducible Builds is part of May/August 2019 round of Outreachy. Outreachy provides internships to work free software. Internships are open to applicants around the world, working remotely and are not required to move. Interns are paid a stipend of $5,500 for the three month internship and have an additional $500 travel stipend to attend conferences/events.

So far, we received more than ten initial requests from candidates. The closing date for applicants is April 2nd. More information is available on the application page.

This week’s edition was written by Bernhard M. Wiedemann, Chris Lamb, Holger Levsen, Vagrant Cascadian & reviewed by a bunch of Reproducible Builds folks on IRC & the mailing lists.

Kees Cook: security things in Linux v5.0

Mër, 13/03/2019 - 12:04pd

Previously: v4.20.

Linux kernel v5.0 was released last week! Looking through the changes, here are some security-related things I found interesting:

read-only linear mapping, arm64
While x86 has had a read-only linear mapping (or “Low Kernel Mapping” as shown in /sys/kernel/debug/page_tables/kernel under CONFIG_X86_PTDUMP=y) for a while, Ard Biesheuvel has added them to arm64 now. This means that ranges in the linear mapping that contain executable code (e.g. modules, JIT, etc), are not directly writable any more by attackers. On arm64, this is visible as “Linear mapping” in /sys/kernel/debug/kernel_page_tables under CONFIG_ARM64_PTDUMP=y, where you can now see the page-level granularity:

---[ Linear mapping ]--- ... 0xffffb07cfc402000-0xffffb07cfc403000 4K PTE ro NX SHD AF NG UXN MEM/NORMAL 0xffffb07cfc403000-0xffffb07cfc4d0000 820K PTE RW NX SHD AF NG UXN MEM/NORMAL 0xffffb07cfc4d0000-0xffffb07cfc4d1000 4K PTE ro NX SHD AF NG UXN MEM/NORMAL 0xffffb07cfc4d1000-0xffffb07cfc79d000 2864K PTE RW NX SHD AF NG UXN MEM/NORMAL

per-task stack canary, arm
ARM has supported stack buffer overflow protection for a long time (currently via the compiler’s -fstack-protector-strong option). However, on ARM, the compiler uses a global variable for comparing the canary value, __stack_chk_guard. This meant that everywhere in the kernel needed to use the same canary value. If an attacker could expose a canary value in one task, it could be spoofed during a buffer overflow in another task. On x86, the canary is in Thread Local Storage (TLS, defined as %gs:20 on 32-bit and %gs:40 on 64-bit), which means it’s possible to have a different canary for every task since the %gs segment points to per-task structures. To solve this for ARM, Ard Biesheuvel built a GCC plugin to replace the global canary checking code with a per-task relative reference to a new canary in struct thread_info. As he describes in his blog post, the plugin results in replacing:

8010fad8: e30c4488 movw r4, #50312 ; 0xc488 8010fadc: e34840d0 movt r4, #32976 ; 0x80d0 ... 8010fb1c: e51b2030 ldr r2, [fp, #-48] ; 0xffffffd0 8010fb20: e5943000 ldr r3, [r4] 8010fb24: e1520003 cmp r2, r3 8010fb28: 1a000020 bne 8010fbb0 ... 8010fbb0: eb006738 bl 80129898 <__stack_chk_fail>

with:

8010fc18: e1a0300d mov r3, sp 8010fc1c: e3c34d7f bic r4, r3, #8128 ; 0x1fc0 ... 8010fc60: e51b2030 ldr r2, [fp, #-48] ; 0xffffffd0 8010fc64: e5943018 ldr r3, [r4, #24] 8010fc68: e1520003 cmp r2, r3 8010fc6c: 1a000020 bne 8010fcf4 ... 8010fcf4: eb006757 bl 80129a58 <__stack_chk_fail>

r2 holds the canary saved on the stack and r3 the known-good canary to check against. In the former, r3 is loaded through r4 at a fixed address (0x80d0c488, which “readelf -s vmlinux” confirms is the global __stack_chk_guard). In the latter, it’s coming from offset 0x24 in struct thread_info (which “pahole -C thread_info vmlinux” confirms is the “stack_canary” field).

per-task stack canary, arm64
The lack of per-task canary existed on arm64 too. Ard Biesheuvel solved this differently by coordinating with GCC developer Ramana Radhakrishnan to add support for a register-based offset option (specifically “-mstack-protector-guard=sysreg -mstack-protector-guard-reg=sp_el0 -mstack-protector-guard-offset=...“). With this feature, the canary can be found relative to sp_el0, since that register holds the pointer to the struct task_struct, which contains the canary. I’m hoping there will be a workable Clang solution soon too (for this and 32-bit ARM). (And it’s also worth noting that, unfortunately, this support isn’t yet in a released version of GCC. It’s expected for 9.0, likely this coming May.)

top-byte-ignore, arm64
Andrey Konovalov has been laying the groundwork with his Top Byte Ignore (TBI) series which will also help support ARMv8.3’s Pointer Authentication (PAC) and ARMv8.5’s Memory Tagging (MTE). While TBI technically conflicts with PAC, both rely on using “non-VA-space” (Virtual Address) bits in memory addresses, and getting the kernel ready to deal with ignoring non-VA bits. PAC stores signatures for checking things like return addresses on the stack or stored function pointers on heap, both to stop overwrites of control flow information. MTE stores a “tag” (or, depending on your dialect, a “color” or “version”) to mark separate memory allocation regions to stop use-after-tree and linear overflows. For either of these to work, the CPU has to be put into some form of the TBI addressing mode (though for MTE, it’ll be a “check the tag” mode), otherwise the addresses would resolve into totally the wrong place in memory. Even without PAC and MTE, this byte can be used to store bits that can be checked by software (which is what the rest of Andrey’s series does: adding this logic to speed up KASan).

ongoing: implicit fall-through removal
An area of active work in the kernel is the removal of all implicit fall-through in switch statements. While the C language has a statement to indicate the end of a switch case (“break“), it doesn’t have a statement to indicate that execution should fall through to the next case statement (just the lack of a “break” is used to indicate it should fall through — but this is not always the case), and such “implicit fall-through” may lead to bugs. Gustavo Silva has been the driving force behind fixing these since at least v4.14, with well over 300 patches on the topic alone (and over 20 missing break statements found and fixed as a result of the work). The goal is to be able to add -Wimplicit-fallthrough to the build so that the kernel will stay entirely free of this class of bug going forward. From roughly 2300 warnings, the kernel is now down to about 200. It’s also worth noting that with Stephen Rothwell’s help, this bug has been kept out of linux-next by him sending warning emails to any tree maintainers where a new instance is introduced (for example, here’s a bug introduced on Feb 20th and fixed on Feb 21st).

ongoing: refcount_t conversions
There also continues to be work converting reference counters from atomic_t to refcount_t so they can gain overflow protections. There have been 18 more conversions since v4.15 from Elena Reshetova, Trond Myklebust, Kirill Tkhai, Eric Biggers, and Björn Töpel. While there are more complex cases, the minimum goal is to reduce the Coccinelle warnings from scripts/coccinelle/api/atomic_as_refcounter.cocci to zero. As of v5.0, there are 131 warnings, with the bulk of the remaining areas in fs/ (49), drivers/ (41), and kernel/ (21).

userspace PAC, arm64
Mark Rutland and Kristina Martsenko enabled kernel support for ARMv8.3 PAC in userspace. As mentioned earlier about PAC, this will give userspace the ability to block a wide variety of function pointer overwrites by “signing” function pointers before storing them to memory. The kernel manages the keys (i.e. selects random keys and sets them up), but it’s up to userspace to detect and use the new CPU instructions. The “paca” and “pacg” flags will be visible in /proc/cpuinfo for CPUs that support it.

platform keyring
Nayna Jain introduced the trusted platform keyring, which cannot be updated by userspace. This can be used to verify platform or boot-time things like firmware, initramfs, or kexec kernel signatures, etc.

Edit: added userspace PAC and platform keyring, suggested by Alexander Popov
Edit: tried to clarify TBI vs PAC vs MTE

That’s it for now; please let me know if I missed anything. The v5.1 merge window is open, so off we go! :)

© 2019, Kees Cook. This work is licensed under a Creative Commons Attribution-ShareAlike 3.0 License.

Daniel Lange: Wiping harddisks in 2019

Mar, 12/03/2019 - 7:53md

Wiping hard disks is part of my company's policy when returning servers. No exceptions.

Good providers will wipe what they have received back from a customer, but we don't trust that as the hosting / cloud business is under constant budget-pressure and cutting corners (wipefs) is a likely consequence.

With modern SSDs there is "security erase" (man hdparm or see the - as always well maintained - Arch wiki) which is useful if the device is encrypt-by-default. These devices basically "forget" the encryption key but it also means trusting the devices' implementation security. Which doesn't seem warranted. Still after wiping and trimming, a secure erase can't be a bad idea .

Still there are three things to be aware of when wiping modern hard disks:

  1. Don't forget to add bs=4096 (blocksize) to dd as it will still default to 512 bytes and that makes writing even zeros less than half the maximum possible speed. SSDs may benefit from larger block sizes matched to their flash page structure. These are usually 128kB, 256kB, 512kB, 1MB, 2MB and 4MB these days.1
  2. All disks can usually be written to in parallel. screen is your friend.
  3. The write speed varies greatly by disk region, so use 2 hours per TB and wipe pass as a conservative estimate. This is better than extrapolating what you see initially in the fastest region of a spinning disk.
  4. The disks have become huge (we run 12TB disks in production now) but the write speed is still somewhere 100 MB/s ... 300 MB/s. So wiping servers on the last day before returning is not possible anymore with disks larger than 4 TB each (and three passes). Or 12 TB and one pass (where e.g. fully encrypted content allows to just do a final zero-wipe).

hard disk size one pass three passes 1 TB2 h6 h 2 TB4 h12 h 3 TB6 h18 h 4 TB8 h24 h (one day) 5 TB10 h30 h 6 TB12 h36 h 8 TB16 h48 h (two days) 10 TB20 h60 h 12 TB24 h72 h (three days) 14 TB28 h84 h 16 TB32 h96 h (four days) 18 TB36 h108 h 20 TB40 h120 h (five days)

  1. As Douglas pointed out correctly in the comment below, these are IT Kilobytes and Megabytes, so 210 Bytes and 220 Bytes. So Kibibytes and Mebibytes for those firmly in SI territory. 

Bits from Debian: New Debian Developers and Maintainers (January and February 2019)

Mar, 12/03/2019 - 1:00md

The following contributors got their Debian Developer accounts in the last two months:

  • Paulo Henrique de Lima Santana (phls)
  • Unit 193 (unit193)
  • Marcio de Souza Oliveira (marciosouza)
  • Ross Vandegrift (rvandegrift)

The following contributors were added as Debian Maintainers in the last two months:

  • Romain Perier
  • Felix Yan

Congratulations!

John Goerzen: Goodbye to a 15-year-old Debian server

Mar, 12/03/2019 - 10:18pd

It was October of 2003 that the server I’ve called “glockenspiel” was born. It was the early days of Linux-based VM hosting, using a VPS provider called memset, running under, of all things, User Mode Linux. Over the years, it has been migrated around, sometimes running on the metal and sometimes in a VM. The operating system has been upgraded in-place using standard Debian upgrades over the years, and is now happily current on stretch (albeit with a 32-bit userland). But it has never been reinstalled. When I’d migrate hosting providers, I’d use tar or rsync to stream glockenspiel across the Internet to its new home.

A lot of people reinstall an OS when a new version comes out. I’ve been doing Debian upgrades with apt for ages, and this one is a case in point. It lingers.

Root’s .profile was last modified in November 2004, and its .bashrc was last modified in December 2004. My own home directory still has a .pinerc, .gopherrc, and .arch-params file. I last edited my .vimrc in 2003 and my .emacs dates back to 2002 (having been copied over from a pre-glockenspiel FreeBSD server).

drwxr-xr-x 3 jgoerzen jgoerzen 4096 Dec 3 2003 irclogs -rw-r--r-- 1 jgoerzen jgoerzen 373 Dec 3 2003 .vimrc -rw-r--r-- 1 jgoerzen jgoerzen 651 Nov 27 2003 .reportbugrc drwx------ 3 jgoerzen jgoerzen 4096 Sep 2 2003 .arch-params -rw-r--r-- 1 jgoerzen jgoerzen 1115 Aug 23 2003 .gopherrc drwxr-xr-x 3 jgoerzen jgoerzen 4096 Jul 18 2003 .subversion -rw-r--r-- 1 jgoerzen jgoerzen 15317 Jun 21 2003 .pinerc

Poking around /etc on glockenspiel is like a trip back in time. Various apache sites still have configuration files around, but have long since been disabled. Over the years, glockenspiel has hosted source code repositories using Subversion, arch, tla, darcs, mercurial and git. It’s hosted websites using Drupal, WordPress, Serendipity, and so forth. It’s hosted gopher sites, websites or mailing lists for various Free Software projects (such as Freeciv), and any number of local charitable organizations. Remnants of an FTP configuration still exist, when people used web design software to build websites for those organizations on their PCs and then upload them to glockenspiel.

-rw-r--r-- 1 root root 268 Dec 25 2005 libnet.cfg -rw-r----- 1 root root 1305 Nov 11 2004 mrtg.cfg -rw-r--r-- 1 root root 552 Jul 31 2004 pam.conf

All this has been replaced by a set of Docker containers running my docker-debian-base software. They’re all in git, I can rebuild one of the containers in a few seconds or a few minutes by typing “make”, and there is no cruft from 2002. There are a lot of benefits to this.

And yet, there is a part of me that feels it’s all so… cold. Servers having “personalities” was always a distinctly dubious thing, but these days as we work through more and more layers of virtualization and indirection and become more distant from the hardware, we lose an appreciation for what we have and the many shoulders of giants upon which we stand.

And, so with that, the final farewell to this server that’s been running since 2003:

glockenspiel:/etc# shutdown -P now Shared connection to glockenspiel.complete.org closed.

Lucas Nussbaum: On Debian frustrations

Mar, 12/03/2019 - 6:51pd

Michael Stapelberg writes about his frustrations with Debian, resulting in him reducing his involvement in the project. That’s sad: over the years, Michael has made a lot of great contributions to Debian, addressing hard problems in interesting, disruptive ways.

He makes a lot of good points about Debian, with which I’m generally in agreement. An interesting exercise would be to rank those issues: what are, today, the biggest issues to solve in Debian? I’m nowadays not following Debian closely enough to be able to do that exercise, but I would love to read others’ thoughts (bonus points if it’s in a DPL platform, given that it seems that we have a pretty quiet DPL election this year!)

Most of Michael’s points are about the need for modernization of Debian’s infrastructure and workflows, and I agree that it’s sad that we have made little progress in that area over the last decade. And I think that it’s important to realize that providing alternatives to developers have a cost, and that when a large proportion of developers or packages have switched to doing something (using git, using dh, not using 1.0-based patch systems such as dpatch, …), there are huge advantages with standardizing and pushing this on everybody.

There are a few reasons why this is harder than it sounds, though.

First, there’s Debian culture of stability and technical excellence. “Above all, do not harm” could also apply to the mindset of many Debian Developers. On one hand, that’s great, because this focus on not breaking things probably contributes a lot to our ability to produce something that works as well as Debian. But on the other hand, it means that we often seek solutions that limit short-term damage or disruption, but are far from optimal on the long term.
An example is our packaging software stack. I wrote most of the introduction to Debian packaging found in the packaging-tutorial package (which is translated in six languages now), but am still amazed by all the unjustified complexity. We tend to fix problems by adding additional layers of software on top of existing layers, rather than by fixing/refactoring the existing layers. For example, the standard way to package software today is using dh. However, dh stands on dh_* commands (even if it does not call them directly, contrary to what CDBS did), and all the documentation on dh is still structured around those commands: if you want to install an additional file in a package, probably the simplest way to do that is to add it to debian/packagename.install, but this is documented in the manpage for dh_install, which your are not going to actually call because dh abstracts that away for you! I realize that this could be better explained in packaging-tutorial… (patch welcomed)

There’s also the fact that Debian is very large, very diverse, and hard to test. It’s very easy to break things silently in Debian, because many  of our packages are niche packages, or don’t have proper test suites (because not everything can be easily tested automatically). I don’t see how the workflows for large-scale changes that Michael describes could work in Debian without first getting much better at detecting regressions.

Still, there’s a lot of innovation going on inside packaging teams, with the development of language-specific packaging helpers (listed on the AutomaticPackagingTools wiki page). However, this silo-ed organization tends to fragment the expertise of the project about what works and what doesn’t: because packaging teams don’t talk much together, they often solve the same problems in slightly different ways. We probably need more ways to discuss interesting stuff going on in teams, and consolidating what can be shared between teams. The fact that many people have stopped following debian-devel@ nowadays is probably not helping…

The addition of salsa.debian.org is probably the best thing that happened to Debian recently. How much this ends up being used for improving our workflows remain to be seen:

  • We could use Gitlab merge requests to track patches, rather than attachments in the BTS. Some tooling to provide an overview of open MRs in various dashboards is probably needed (and unfortunately GitLab’s API is very slow when dealing with large number of projects).
  • We could probably have a way to move the package upload to a gitlab-ci job (for example, by committing the signed changes file in a specific branch, similar to what pristine-tar does, but there might be a better way)
  • I would love to see a team experiment with a monorepo approach (instead of the “one git repo per package + mr to track them all” approach). For teams with lots of small packages there are probably a lot of things to win with such an organization.

 

Shirish Agarwal: Processing Insanity

Mar, 12/03/2019 - 6:08pd

This blog post starts from where it ended a few days ago. I am fortunate and to an extent even blessed that I have usually had honest advise but sometimes even advice falls short when you see some harsh realities. There were three people who replied, you can read mark’s and frode’s reply as they have shared in the blog post.

I even shared it with a newly-found acquaintaince in the hopes that there may be some ways, some techniques or something which would make more sense as this is something I have never heard within the social circles I have been part of so was feeling more than a bit ill-prepared. When I shared with Paramitra (gentleman whom I engaged as part of another socio-techno probable intervention meetup I am hoping to meet soon) , he also shared some sound advice which helped me mentally prepare as well –

So, if you’re serious about what you can do with/for this friend of yours and his family, I do have several suggestions. 

1. To the best of my knowledge, and I have some exposure, no one goes ‘insane’ just like that. There has to be a diagnosis. Please find out from his family if he’s been taken to a psychiatrist. If not, that’s the first thing you can convince his family to do. Be with them, help them with that task.

2. If he’s been diagnosed, find out what that is. Most psychiatric disorders can be brought to a manageable level with proper medications and care. But any suggestions I can offer on that depends on the diagnosis.

3. However, definitely inform his family that tying him up, keeping him locked etc will only worsen the situation. He needs medical and family care – not incarceration, unless the doctor prescribes institutionalized treatment.

Hope this helps. Please be a friend to him and his family at this hour of crisis. As a nation, our understanding of mental health per se is poor, to say the least.

Paramita Banerjee

So armed with what I thought was sufficient knowledge I went to my friend’s home. The person whom I met could not be the same person whom I knew as a friend in college. During college, he had a reputation of a toughie and he looked and acted the part. So, in many ways it was the unlikeliest of friendships. I shared with him tips of Accountancy, Economics etc. and he was my back. He was also very quick on the re-partees so we used to have quite a fun time exchanging those. For the remainder of the exchange I will call my friend ‘Amar’ and his sister ‘Kritika’ as they have been the names I like.

The person whom I met was a mere shadow of the person I knew. Amar had no memory of who I was. He had trouble comprehending written words and mostly mumbled. Amar did say something interesting and fresh once in a while but it was like talking mostly to a statue. He stank and was mostly asleep even when he was awake. Amar couldn’t look straight at me and he had that if I touched him or he touched me he would infect me. He had long nails as well. Kritika told me that he does have baths once every few days but takes 3-4 hours to take a bath, sleeps in there as well. The same happens when he goes for shitting as well. The saving grace is they have their own toilet and bathroom within the house. I have no comprehension how they might be adjusting, all in that small space.

I learned from Kritika what I hadn’t known about him and the family over the last ten odd years. His mum died in the same room where he was and he had no comprehension that she had died, this had happened just a few weeks back. He was one of three children, the middle child, the elder daughter, who is now a widow and has three daughters who are living with them. Amar, his father and the youngest sister who is trying desperately to keep it altogether but I don’t know how and what she will be able to do. 7 mouths to feed and 6 people who all have their own needs and wants apart from basic existence. They are from a low-income group. The elder sister does have lot of body pains although I was not able to ask what from. I do know nursing is a demanding profession and from my hospital stay, at times they have to around the clock 24×7 doing things no normal person can do.

Two of the nieces are nearing teenage years and was told of sexually suggestive remarks to the nieces by one of the neighbors. The father is a drunk, the brother-in-law who died was a drunk and the brother, Amar had consumed lots of cannabis seeds. Apparently, during the final year exams where we were given different centers he went to Bombay/Mumbai to try his hands at movies, then went to Delhi and was into selling some sort of chemicals from company to the other.

Maybe it was ‘bad company’ as her mother on the phone had told me, maybe it was the work he was doing which he was not happy with which led him to cannabis addiction. I have no way of knowing anything of his past. I did ask Kritika if she can dig out any visiting cards or something. I do have enough friends in Delhi so it’s possible I can know about how things came to be this bad.

There was a small incident which also left me a bit shaken. The place where they are is a place called Pavana Nagar. This is on back of Pimpri-Chinchwad industrial township so most of the water that the town/village people consume has lot of chemical effluents and this the local councillor (called nagar sevak) knows but either can’t or won’t do anything about it. There are lot of diseases due to the pollutants in the water. The grains they buy or purchase, Kritika suspects or/and knows also use the same water but she is helpless to do anything about it.

The incident is a small one but I wanted to share a fuller picture before sharing that. I had left my bag, a sort of sling bag where I was sitting in the room . After Kritika took me to another building to show me the surrounding areas (as I was new here and had evinced interest to know the area) , when we came back, my bag was not to be found. While after searching for a while, I got the bag, there was no money in it ( I usually keep INR 100-200 in case money gets stolen from on me. I also keep some goodies (sweet and sour both) just in case I feel hungry and there’s nothing around. Both were missing. The father pretended, he had put the bag away by mistake. I didn’t say anything because it would have been loss of face for the younger sister although it’s possible that she knows or had some suspicions. With the younger kids around, it would have been awkward to say that and I didn’t really wanna make a scene. It wasn’t much, but just something I didn’t expect.

Also later I came to know that whenever the father drinks, he creates lot of drama and says whatever comes to his mind. It is usually dirty, nasty and hurtful from what I could gather.

Due to my extended stay in hopsital due to Epilepsy had come to know of couple of medical schemes which were meant for weaker sections of the society. I did share what I knew of the schemes. While I hope to talk with Kritika more, I don’t see a way out of the current mess they are in. The sense I got from her is that she is fighting too many battles and I don’t how she can win them all. I also told her about NMRI I have no clue where to go from here. Also don’t wanna generalize but there might be possibilities of many Amars and Kritikas in our midst or around us whose story we don’t know. If they could just have some decent water, no mosquitoes it probably would enhance their lives quite a bit and maybe have a bit more agency about themselves. There is one thing that Kritika shared which was also interesting. She had experience of working back-office for some IT company but now looking after the family she just couldn’t do the same thing.

Note and disclaimer – The names ‘Amar’ and ‘Kritika’ are just some names I chose. The names have been given to –

a. Give privacy to the people involved.
b. To embody sustance to the people and the experience so they are not nameless people.


Faqet