Free software activities in March 2020

Here is my monthly update covering what I have been doing in the free software world during March 2020 (previous month):

• Patched the browserify-lite Javascript compiler to ensure that module dependency maps are rendered in a deterministic order. (#14)

• Pushed a change to the "pmemkv" key-value data store to make their documentation build reproducibly. (#615)

• As part of my duties of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet as well as the usual internal discussions regarding logistics, licensing, policy and liaising with the ClearlyDefined project and so on.

  

  Congratulations to Megan Sanicki and Josh Simmons who were elected and re-elected respectively to the OSI Board's two individual member seats and to Italo Vignoli who was elected to the affiliate member seat. These newly-elected Directors will took their seats on March 20, 2020.

• Opened a pull request against the onionbalance load balancer for hidden Tor services to fix test failures under Pytest 3.x. (#75)

• Filed an issue against IMAP Spam Begone (isbg), a script by Louis-Philippe Véronneau (pollo) that makes it easy to process an email inbox using SpamAssassin, to report that a duplicate documentation entry includes a nondeterministic value taken from the value of the XDG cache directory. (#151)

• As part of being on being on the judging panel of the OpenUK Awards I am pleased to announce that after some discussion nominations are now open until 15th June in five different categories.

• Merged a number of contributions to my django-cache-toolbox "non-magical" caching library for Django web applications, including caching negative relation lookups locally (#14) and to include the README file in the package long description (#17).

• Made some small changes to my tickle-me-email library which implements Gettings Things Done (GTD)-like behaviours in IMAP inboxes to support to optionally limiting the number of messages in the send-later functionality. [...]

In addition, I did even more hacking on the Lintian static analysis tool for Debian packages, including:

• New features:

  • Check for py3versions -i in autopkgtests and debian/rules files. (#954763)
  • Warn when py3versions -s is used without a python3-all dependency. (#954763)
  • Expand possible-missing-colon-in-closes to also check for semicolons used in place of colons. (#954484)
  • Check for new packages that use a date-based versioning scheme (eg. YYYYMMDD-1) without a 0~ suffix. (#953036)

• Improvements:

  • Don't emit duplicate-short-description or duplicate-long-description if they contain substitution variables. (#947168)
  • Don't report about using a newer Standards-Version when uploading to the backports repository. [...]
  • Don't check the length of Checksums-Sha256 fields. (#954798)
  • Don't check for DEBIAN/md5sums control files in udebs. (#954803)
  • Apply a patch from gregoa to remove some <!nocheck> annotations; some packages are required to build successfully. (#954338)

• Misc:

  • Correct reference to build dependencies in the long description of the debian-rules-uses-installed-python-versions tag. [...]
  • Make some cosmetic improvements to the CONTRIBUTING.md file. [...]
  • Correct reference to a bug in a previous debian/changelog entry. [...]
  • Avoid indenting approximately 150 lines by returning early from a subroutine and other code improvements. [...]

Reproducible builds

One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom.

Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:

• Filed an issue against IMAP Spam Begone — a script by Louis-Philippe Véronneau (pollo) that makes it easy to process an email inbox using SpamAssassin — to report that a (duplicate) documentation entry includes nondeterministic value taken from the value of the XDG cache directory (#151) and filed an upstream pull requests against the pmemkv key-value data store to make their documentation build reproducibly (#615).

• Further refined my merge request against the debian-installer component to allow all arguments from sources.list files (such as [check-valid-until=no]) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure. (#13)

• Submitted two following patches to fix reproducibility-related toolchain issues within Debian:

  • node-browserify-lite: Please make the output reproducible. (#954409)

  • pdb2pqr: Please make the aconf.py file reproducible. (#955287)

• Submitted eight patches to fix specific reproducibility issues in beep (caused by a variation between /bin/dash and /bin/bash), cloudkitty (due to a default value being taken from the number of CPUs on the build machine), font-manager (embedding the value of @abs_top_srcdir@ into the resulting binary), gucharmap (due to embedding the absolute build path when generating a comment in a header file), infernal (timestamps are injected into a Python example, which should not be shipped anyway), ndisc6 (embeds the value of CFLAGS into the binary without sanitising any absolute build paths), node-nodedbi (embedded timestamp in binary) & pmemkv (does not respect SOURCE_DATE_EPOCH when populating a YEAR variable).

• Kept isdebianreproducibleyet.com up to date. [...]

• Continued collaborative work on an academic paper to be published within the next few months.

• Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.

• Drafted, published and publicised our monthly report.

• Improved our website, including correcting the syntax of some CSS class formatting [...], improved some "filed against" copy a little better [...] and corrected a reference to calendar.monthrange Python method. [...]

In our tooling, I also made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading version 138 to Debian:

• Improvements:

  • Don't allow errors with R script deserialisation cause the entire operation to fail, for example if an external library cannot be loaded. (#91)
  • Experiment with memoising output from expensive external commands, eg. readelf. (#93)
  • Use dumppdf from the python3-pdfminer if we do not see any other differences from pdftext, etc. (#92)
  • Prevent a traceback when comparing two R .rdx files directly as the get_member method will return a file even if the file is missing. [...]

• Reporting:

  • Display the supported file formats into the package long description. (#90)
  • Print a potentially-helpful message if the PyPDF2 module is not installed. [...]
  • Remove any duplicate comparator descriptions when formatting in the --help output or in the package long description. [...]
  • Weaken "Install the X package to get a better output." message to "... may produce a better output." as the former is not guaranteed. [...]

• Misc:

  • Ensure we only parse the recommended packages from --list-debian-substvars when we want them for debian/tests/control generation. [...]
  • Add upstream metadata file [...] and add a Lintian override for upstream-metadata-in-native-source as "we" are upstream. [...]
  • Inline the RequiredToolNotFound.get_package method's functionality as it is only used once. [...]
  • Drop the deprecated "py36 = [..]" argument in the pyproject.toml file. [...]

The Reproducible Builds project also operates a fully-featured and comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, I reworked the web-based package rescheduling tool to:

• Require a HTTP POST method in the web-based scheduler as not only should HTTP GET requests be idempotent but this will allow many future improvements in the user interface. [...][...][...]

• Improve the authentication error message in the rescheduler to suggest that the developer's SSL certificate may have expired. [...]

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 8 hours on its sister Extended LTS project.

• Investigated and triaged glibc (CVE-2020-1751), jackson-databind, libbsd (CVE-2019-20367), libvirt (CVE-2019-20485), netkit-telnet & netkit-telnet-ssl (CVE-2020-10188), pdfresurrect (CVE-2020-9549) & shiro (CVE-2020-1957), etc.

• In the script that reserves a unique advisory number don't warn about potential duplicate work when issuing a regression in order to avoid this message being missed when it does apply. [...]

• Frontdesk duties, responding to user/developer questions, reviewing others' packages, participating in mailing list discussions, etc.

• xtrlock versions 2.8+deb9u1 (#949112) and 2.8+deb10u1 (#949113) was accepted to the Debian stretch and buster distributions.

• Issued DLA 2115-2 to correct a regression in a previous fix (a use-after-free vulnerability) in the ProFTPD FTP server.

• Issued DLA 2132-1 to fix an issue where incorrect default permissions on a HTTP cookie store could have allowed local attackers to read private credentials in libzypp, the library underpinning package management tools such as YaST, zypper and the openSUSE/SLE implementation of PackageKit.

• Issued DLA 2134-1 to patch an out-of-bounds write vulnerability in pdfresurrect, a tool for extracting or scrubbing versioning data from PDF documents.

• Issued DLA 2136-1, addressing an out-of-bounds buffer read vulnerability in libvpx, a library implementing the VP8 & VP9 video codecs.

• Issued DLA 2142-1. It was discovered that there was a buffer overflow vulnerability in slirp, a SLIP/PPP emulator for using a dial up shell account. This was caused by the incorrect usage of return values from snprintf(3).

• Issued DLA 2145-1 and DLA 2145-2 for twisted to prevent a large number of HTTP request splitting vulnerabilities in Twisted, a Python event-based framework for building various types of internet applications.

• Issued ELA-219-1 to address an out-of-bounds read vulnerability during string comparisons in libbsd, a library of functions commonly available on BSD systems but not on others such as GNU.

You can find out more about the Debian LTS project via the following video:

Debian Uploads

• python-django (2.2.11-1 & 3.0.4-1) — New upstream security releases. (#953102)

• redis:

  • 6.0~rc1-3 — Install OpenSSL when running the testsuite as it is required for generating test certificates.
  • 6.0~rc2-1 — New upstream beta release.

• xtrlock (2.13) — Add a warning that X11 does not support grabbing events under the Wayland display server protocol. (#953319)

• gunicorn (20.0.4-4) — Ensure that the Python 3.x gunicorn binary package replaces the now-legacy gunicorn3 that was removed in 19.9.0-2 upon installation. I also backported this version to buster-backports. (#953883)

• hiredis (0.14.1-1) — New upstream release.

• libfiu (1.00-7) — Fix build failures under Python 3.8. (#954287)

• memcached:

  • 1.6.0-1 — New upstream release.
  • 1.6.1-1 — New upstream release.
  • 1.6.2-1 — New upstream security release. (#954808)

• bfs (1.6-2, 1.6-3 & 1.6-4) — Fix various build failures on kFreeBSD architectures.

• mtools (4.0.24-1) — New upstream release.

• xcb (2.4-6) — Remove a vague "pigeon holes" metaphor from package description.

For the Debian Privacy Maintainers team I requested that the pyptlib package be removed from the archive (#953429) as well as uploading onionbalance (0.1.8-6) to fix test failures under Pytest 3.x (#953535) and a new upstream release of nautilus-wipe.

Finally, I sponsored an upload of bilibop (0.6.1) on behalf of Yann Amar.

You can subscribe to new posts via email or RSS.