Free software activities in January 2020

Here is my monthly update covering what I have been doing in the free software world during January 2020 (previous month):

• Attended Sustain 2020 in Brussels, Belgium prior to FOSDEM. Thanks to all the organisers and attendees for making this happen.

• Merged a pull request from Danil Kozyatnikov to my django-autologin library for the Django web-development framework to add support for custom User models. [...]

• As part of my duties of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions regarding license policy, liaising with the ClearlyDefined project to recieve their annual report to the OSI board, etc. etc.

• Opened pull requests upstream to make the build reproducible in the Shotcut cross-platform video editor [...] and pikepdf, a Python PDF library [...].

• More conversations for the next interation of the OpenUK awards to be presented in June.

• I officially left the Debian FTP-master team in order to prioritise my work elsewhere in the distribution. [...]

• Published a new version of my Strava Enhancement Suite after incorporating some fixes from Tom Chapman. [...]

• More hacking on the Lintian static analysis tool for Debian packages, including:

  • New features:

    • Recognise 4.5.0 as the latest Standards-Version to match the latest Debian Policy. [...]
    • Check for ListenStream configuration keys in systemd .socket files that refer to /var/run. (#948478)

  • Bug fixes:

    • Create a .cache directory for the GitLab CI if it does not exist. [...]
    • Ensure that Lintian itself is "Lintian clean" by not warning about our Yapp::Parser test fixture. [...]
    • Prevent false positives in missing-build-dependency-for-dh_-command by adding entries for dh-sequence-ada and dh-sequence-sphinxdoc. (#947836)

  • Misc:

    • Adjust the "certainty" of the no-dh-sequencer tag to "possible". (#948376)
    • Replace a potentially loaded term in the name of the check for suspiciously-long lines in source files. [...][...]
    • Rename systemd-service-file-pidfile-refers-to-var-run to systemd-service-file-refers-to-var-run. [...][...]
    • Refresh "private" Debhelper data. [...]

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws almost all software is distributed pre-compiled to end users. The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom.

Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:

• Filed upstream pull requests for Shotcut cross-platform video editor [...] and pikepdf, a Python PDF library [...].

• Attended Sustain 2020 in Brussels, Belgium just prior to FOSDEM.

• In Debian:

  • Submitted 8 patches to fix specific reproducibility issues in bochs, mcomix, pcbasic, pikepdf, python-gmusicapi, python-pysam, shotcut & vmatch.

  • Submitted a patch for pkg-js-tools to exclude Makefile from automatic installation if not explicitely set in debian/nodejs/files. (#949580)

  • Kept isdebianreproducibleyet.com up to date. [...]

  • Fixed a bug against node-npmrc for failing to build (and/or is unreproducible) if $HOME/.npmrcs already exists. #949579

• Categorised a huge number of packages and issues in the Reproducible Builds "notes" repository, including identifying a number of new generalised issues.

• disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. This month, I fixed an issue by ignoring the return values of fsyncdir to ensure (for example) dpkg(1) can "flush" /var/lib/dpkg correctly. [...]

• I spent a few moments on our website this month as well including in the Python SOURCE_DATE_EPOCH documentation, clarifying that the second example generates a Python str-type, not a datetime.datetime [...], correcting word omissions in the report template [...], linking to our mailing list overview page (and not the archives) [...], applying the Black source code reformatter to the draft generation script [...], moving the continuous tests heading level to <h1> to match the other pages [...], calculating the report posts' authors dynamically [...], etc.

• Drafted, published and publicised our monthly report.

I also made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues:

• New features:

  • Support external difference tools such as Meld, etc. similar to git-difftool(1). (#87
  • Extract resources.arsc files as well as classes.dex from Android .apk files to ensure that we show the differences there. (#27)
  • Fallback to the regular .zip container format for .apk files if apktool is not available. [...][...][...][...]
  • Drop --max-report-size-child and --max-diff-block-lines-parent; scheduled for removal in January 2018. [...]
  • Append a comment to a difference if we fallback to a less-informative container format but we are missing a tool. [...][...]

• Bug fixes:

  • No longer raise a KeyError exception if we request an invalid member from a directory container. [...]

• Documentation/workflow improvements:

  • Clarify that "install X" in various outputs actually refers to system-level packages. [...]
  • Add a note to the Contributing documentation to suggest enable concurrency when running the tests locally. [...]
  • Include the CONTRIBUTING.md file in the PyPI.org release. [...][...]

• Logging improvements:

  • Log a debug-level message if we cannot open a file as container due to a missing tool to assist in diagnosing issues. [...]
  • Correct a debug message related to compare_meta calls to quote the arguments correctly. [...]
  • Add the current PATH environment variable to the Normalising locale... debug-level message. [...]
  • Print the Starting diffoscope $VERSION line as the first line of the log as we are, well, starting diffoscope. [...]
  • If we don't know the HTML output name, don't emit an enigmatically truncated HTML output for debug message. [...]

• Tests:

  • Don't exhaustively output the entire HTML report when testing the regression for #875281; parsing the JSON and pruning the tree should be enough. (#84)
  • Refresh and update the fixtures for the .ico tests to match the latest version of Imagemagick in Debian unstable. [...]

• Code improvements:

  • Add a .git-blame-ignore-revs file to improve the output of git-blame(1) by ignoring large changes when introducing the Black source code reformatter reformatter and update the CONTRIBUTING.md guide on how to optionally use it locally. [...]
  • Add a noqa line to avoid a false-positive Flake8 "unused import" warning. [...]
  • Move logo.svg to under the doc/ directory [...] and make setup.py executable [...].
  • Tidy diffoscope.main's configure method. [...][...][...][...]
  • Drop an assertion that is guaranteed by parallel if conditional [...] and an unused "Difference" import from the APK comparator. [...]
  • Turn down the "volume" for a recommendation in a comment. [...]
  • Rename the diffoscope.locale module to diffoscope.environ as we are modifying things beyond just the locale (eg. calling tzset, etc.) [...]
  • Factor-out the generation of foo not available in path comment messages into the exception that raises them [...] and factor out running all of our many zipinfo into a new method [...].

• trydiffoscope is the web-based version of diffoscope. This month, I fixed the PyPI.org release by adding the trydiffoscope script itself to the MANIFEST file and performing another release cycle. [...]

Debian

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

• Frontdesk duties, responding to user/developer questions, reviewing others' packages, participating in mailing list discussions, etc.

• Investigated and triaged apt-cacher-ng (CVE-2020-5202), cacti, dnsmasq (CVE-2019-14834), e2fsprogs (CVE-2019-5188), firefox-esr, glib2.0 (CVE-2020-6750), intel-microcode (CVE-2020-0549, etc.), ldm (CVE-2019-20373), libbsd (CVE-2019-20367), libxml2 (CVE-2019-20388, etc.), nethack (CVE-2020-5211, etc.), nginx (CVE-2019-20372), openjdk-7 (CVE-2020-2604, etc.), openjpeg2 (CVE-2020-6851), otrs2 (CVE-2020-1765, CVE-2020-1766 & CVE-2020-1767), phpmyadmin (CVE-2020-5504), pillow (CVE-2020-5313, etc.), pure-ftpd (CVE-2019-20176), python-pysaml2 (CVE-2020-5390), python3.4 (CVE-2020-8492), suricata (CVE-2019-18792, etc.), systemd (CVE-2019-20386), tomcat7 (CVE-2019-17563, etc.), waitress (CVE-2019-16792), wireshark (CVE-2020-7045, etc.) & xerces-c.

• Uploaded versions xtrlock 2.8+deb9u1 (#949112) and 2.8+deb10u1 (#949113) to the jessie and buster distributions.

• Issued DLA 1931-2 for libgcrypt20 correcting a regression in the former handling of an ECDSA timing attack.

• Issued DLA 2056-1 for a HTTP smuggling attack in waitress, a pure-Python WSGI server whereby if a proxy server is used in front of waitress, an invalid request may be sent by an attacker that bypasses the front-end and is parsed differently by the server leading to potential "smuggled" request.

• Issued DLA 2057-1 to address three issues in Pillow, the de-facto imaging library for the Python programming language.

• ldm, the display manager for the Linux Terminal Server Project, incorrectly parsed responses from an SSH server which could result in local root privilege escalation. This resulted in the issuing of DLA 2064-1.

• Issued DLA 2069-1 for cacti to fix a number of cross-site scripting vulnerabilities in this web interface for monitoring systems.

• Issued DLA 2083-1 in hiredis to address a large number of NULL pointer dereferences due to unchecked return values from malloc(3) and friends.

You can find out more about the project via the following video:

Debian bugs filed

• node-npmrc: Fails to build (and/or is unreproducible) if $HOME/.npmrcs exists. #949579

• firmware-atheros: Please package new "upstream" firmware version. (#947980)

Uploads

• installation-birthday (14) — Add a --puppet option to output the date as a Puppet "fact" after a suggestion from Antoine Beaupre (anarcat). (#948838)

• python-django (3.0.2-1) — New upstream release & add python3-selenium to test dependencies and to a "runtime" Suggests. (#947549)

• memcached (1.5.21-1) — New upstream release.

• libfiu (1.00-6) — Drop Build-Depends on libpython-all-dev for Python 2.x removal. (#936856)

• hiredis (0.14.0-5) — CVE-2020-7105: Prevent a large number of NULL pointer redeferences due to unchecked return values from malloc(3). (#949995)

• bfs (1.5.2-1) — New upstream release.

You can subscribe to new posts via email or RSS.