Free software activities in July 2020

Here is my monthly update covering what I have been doing in the free and open source software world during July 2020 (previous month):

• Opened a pull request to make the build reproducible in PyERFA, a set of Python bindings for various astronomy-related utilities (#45), as well as one for PeachPy assembler to make the output of codecode/x86_64.py reproducible (#108).

• As part of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions regarding logistics and policy etc. This month, it was SPI's Annual General Meeting and the OSI has been running a number of remote strategy sessions for the board.

• Fixed an issue in my tickle-me-email library that implements Getting Things Done (GTD)-like behaviours in IMAP inboxes to ensure that all messages have a unique Message-Id header. [...]

• Reviewed and merged even more changes by Pavel Dolecek into my Strava Enhancement Suite, a Chrome extension to improve the user experience on the Strava athletic tracker.

• Updated travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub, to use the Travis CI continuous integration platform) to fix a compatibility issue with the latest version of mk-build-deps. [...][...]

For Lintian, the static analysis tool for Debian packages:

• Update the regular expression to search for all the released versions in a .changes file. [...]

• Avoid false-positives when matching sensible-utils utilities such as i3-sensible-pager. (#966022)

• Rename the send-patch tag to patch-not-forwarded-upstream. [...]

• Drop reminders from 26 tags that false-positives should be reported to Lintian as this is implicit in all our tags. [...]

Reproducible Builds

One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The project is proud to be a member project of the Software Freedom Conservancy. Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:

• Opened pull requests to make the build reproducible in pyerfa (#45) as well as one for PeachPy to make the output of codecode/x86_64.py reproducible (#108).

• In Debian:

  • Submitted the following patches to fix reproducibility-related toolchain issues within Debian:

    • dh-fortran-mod: Please make the output reproducible. (#965255)

    • gem2deb: should not install mkmf.log files. (#964772)

    • nmap: Please make the obfuscated nmap_service.exe file reproducible. (#964369)

  • I also submitted 13 patches to fix specific reproducibility issues in flit, gmap, jskeus, libtpms, logilab-common, mrbayes, numpydoc, pyerfa, python-cooler, python-peachpy, python-pyxs, sratom & weather-util.

  • When investigating packages, I discovered and reported that dolfinx was running its test suite even when the nocheck build profile was set. (#965946)

  • Kept isdebianreproducibleyet.com up to date. [...]

• Categorised a large number of packages and issues in the Reproducible Builds 'notes' repository.

• Drafted, published and publicised our monthly report.

• Updated the main Reproducible Builds website and documentation to drop a number of unused Javascript files [...][...][...] and added unminified versions of Bootstrap and jQuery [...] and fixed a number of broken URLs [...][...].

• strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month, I made sure that we did not install the internal handler documentation generated from Perl POD documents [...] and fixed a trivial typo [...].

diffoscope

Elsewhere in our tooling, I made the following changes to diffoscope, including preparing and uploading versions 150, 151, 152, 153 & 154 to Debian:

• New features:

  • Add support for flash-optimised F2FS filesystems. (#207)
  • Don't require zipnote(1) to determine differences in a .zip file as we can use libarchive. [...]
  • Allow --profile as a synonym for --profile=-, ie. write profiling data to standard output. [...]
  • Increase the minimum length of the output of strings(1) to eight characters to avoid unnecessary diff noise. [...]
  • Drop some legacy argument styles: --exclude-directory-metadata and --no-exclude-directory-metadata have been replaced with --exclude-directory-metadata={yes,no}. [...]

• Bug fixes:

  • Pass the absolute path when extracting members from SquashFS images as we run the command with working directory in a temporary directory. (#189)
  • Correct adding a comment when we cannot extract a filesystem due to missing libguestfs module. [...]
  • Don't crash when listing entries in archives if they don't have a listed size such as hardlinks in ISO images. (#188)

• Output improvements:

  • Strip off the file offset prefix from xxd(1) and show bytes in groups of 4. [...]
  • Don't emit javap not found in path if it is available in the path but it did not result in an actual difference. [...]
  • Fix ... not available in path messages when looking for Java decompilers that used the Python class name instead of the command. [...]

• Logging improvements:

  • Add a bit more debugging info when launching libguestfs. [...]
  • Reduce the --debug log noise by truncating the has_some_content messages. [...]
  • Fix the compare_files log message when the file does not have a literal name. [...]

• Codebase improvements:

  • Rewrite and rename exit_if_paths_do_not_exist to not check files multiple times. [...][...]
  • Add an add_comment helper method; don't mess with our internal list directly. [...]
  • Replace some simple usages of str.format with Python 'f-strings' [...] and make it easier to navigate to the main.py entry point [...].
  • In the RData comparator, always explicitly return None in the failure case as we return a non-None value in the success one. [...]
  • Tidy some imports [...][...][...] and don't alias a variable when we don't end up using. [...]
  • Clarify the use of a separate NullChanges quasi-file to represent missing data in the Debian package comparator [...] and clarify use of a 'null' diff in order to remember an exit code. [...]

• Misc:

  • Profile the launch of libguestfs filesystems. [...]
  • Clarify and correct our contributing info. [...][...][...][...][...][...]

Debian

In Debian, I made the following uploads this month:

• Redis (6.0.6-1) — New upstream release.

• Django:

  • 2.2.14-1 — New upstream bugfix release.
  • 3.1~rc1-1 — New upstream release candidate.

• lastpass-cli (1.3.3-4) — Fix a build failure under GCC 10. (#957416)

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 for the Extended LTS project. This included:

• Frontdesk duties, including responding to user/developer questions, reviewing packages, participating in mailing list discussions, offering comments on our survey and attended our IRC contributor meeting. I also made some changes to tools after the move from the Debian version supported [...].

• Investigated and triaged atril, ceph, cimg, ffmpeg, golang-github-seccomp-libseccomp-golang, gosa, http-parser, jruby, json-c, jupyter-notebook, keystone, ksh, libjpeg-turbo, libopenmpt, libpam-radius-auth, log4net, mailman, milkytracker, mupdf, pillow, poppler, puma, python-rtslib-fb, roundcube, ruby-zip, salt, slirp, sqlite3, transmission & wordpress.

• Issued DLA 2273-1 for shiro, first to fix a path-traversal issue where a specially-crafted request could cause an authentication bypass (CVE-2020-1957) and then to fix an encoding issue introduced in the handling of the previous CVE-2020-1957 path-traversal issue which itself could have also caused an authentication bypass (CVE-2020-11989).

• Issued DLA 2274-1 to fix a possible signature verification issue in the fwupd firmware update tool as the return value of gpgme_op_verify_result was not being checked.

• Issued DLA 2299-1 and ELA 252-1 in order to prevent a privilege escalation vulnerability discovered in Net-SNMP, a set of tools for collecting and organising information about devices on computer networks.

• Issued DLA 2300-1 as it was discovered that there was an issue where kdepim-runtime would default to using unencrypted POP3 communication despite the user interface indicating that encryption was in use.

You can find out more about the project via the following video:

You can subscribe to new posts via email or RSS.