Free software activities in August 2021

This is my monthly update covering what I have been doing in the free software world throughout August 2021 (view report for July).

• Although my term on Board directors of the Open Source Initiative (OSI) was extended earlier in the year due to exceptional circumstances surrounding the election, I finally had my meeting to formally vote in my replacement and the other directors this month. The last few years have involved a lot of change for the OSI, open source as well as the world as a whole. But compared to when I joined the Board back in 2018, the organisation is in a much better position to respond to changes, and I look forward to seeing what the organisation gets up to in the future, although now slightly from afar.

• I opened a pull request to make the build reproducible in pytsk, a project which provides Python bindings for the SleuthKit set of digital forensic tools. It turns out that the pytsk3.c file was not being generated in a deterministic fashion due to iterating over a nondeterministic Python data structure. [...]

• Reviewed and merged a pull request to my django-slack library which provides a convenient wrapper between projects using the Django web development framework and the Slack chat platform. Specifically, the PR added support for the unfurl_links and unfurl_media parameters. [...]

Reproducible Builds

The motivation behind Reproducible Builds is to ensure no flaws have been introduced during compilation processes by promising identical results are always generated from a given source. This allows multiple third-parties to come to a consensus on whether a build was compromised or not.

This month, I:

• Helped publicise a number of talks related to reproducible builds at DebConf21. This included Holger's talk on "Reproducible Buster, Bullseye & Bookworm" as well as Vagrant's talk called "Looking Forward to Reproducible Builds".

• Uploaded python-libarchive-c version 3.1-1 to Debian experimental for the new 3.x branch. (python-libarchive-c is used by diffoscope.

• In Debian:

  • Kept isdebianreproducibleyet.com up to date. [...]

  • Submitted the following patches to fix reproducibility-related toolchain issues within Debian:

  • I also submitted 10 patches to fix specific reproducibility issues in mapcache, numcodecs, pytsk, rust-coreutils, samtools, spatialindex, spirv-cross, surgescript, translate and tty-solitaire.

  • I filed a bug against libopenobex with a patch to report a missing optional build-dependency on graphviz which was uncovered during reproducibility testing. (#991938)

• Opened a pull request to make the build reproducible in pytsk, a project which provides Python bindings for the SleuthKit set of digital forensic tools. The pytsk3.c file was not being generated in a deterministic fashion. [...]

• Spent more time collecting data and other input for a long-running task to create an "ecosystem map" for the Reproducible Builds project.

• Helped publicise and then attend our latest monthly IRC meeting.

• Contributed to a thread on our mailing list regarding Android "desugaring" and its effects on reproducible builds.

• Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.

• Drafted, published and publicised our monthly report for July 2021.

I also made the following changes to diffoscope, including uploading versions 180, 181 and 182 to Debian.

• New features:

  • Add support for extracting Android V2 signing keys. [...]
  • If we specify a suffix for a temporary file or directory within the code, ensure it starts with an underscore (ie. "_") to make the generated filenames more human-readable. [...]
  • Don't include short GCC lines that differ on a single prefix byte either. These are distracting, not very useful and are simply the strings(1) command's idea of the build ID, which is displayed elsewhere in the diff. [...][...]
  • Don't include specific .debug-like lines in the ELF-related output, as it is invariably a duplicate of the debug ID that exists better in the readelf(1) differences for this file. [...]

• Bug fixes:

  • Add a special case to SquashFS image extraction to not fail if we aren't the superuser. [...]
  • Only use java -jar /path/to/apksigner.jar if we have an apksigner.jar as newer versions of apksigner in Debian use a shell wrapper script which will be rejected if passed directly to the JVM. [...]
  • Reduce the maximum line length for calculating Wagner-Fischer, improving the speed of output generation a lot. [...]
  • Don't require apksigner in order to compare .apk files using apktool. [...]
  • Update calls (and tests) for the new version of odt2txt. [...]

• Output improvements:

  • Mention in the output if the apksigner tool is missing. [...]
  • Profile diffoscope.diff.linediff and specialize. [...][...]

• Logging improvements:

  • Format debug-level messages related to ELF sections using the diffoscope.utils.format_class. [...]
  • Print the size of generated reports in the logs (if possible). [...]
  • Include profiling information in --debug output if --profile is not set. [...]

• Codebase improvements:

  • Clarify a comment about the HUGE_TOOLS Python dictionary. [...]
  • We can pass -f to apktool to avoid creating a strangely-named subdirectory. [...]
  • Drop an unused File import. [...]
  • Update the supported & minimum version of Black. [...]
  • We don't use the logging variable in a specific place, so alias it to an underscore (ie. "_") instead. [...]
  • Update some various copyright years. [...]
  • Clarify a comment. [...]

• Test improvements:

  • Update a test to check specific contents of SquashFS listings, otherwise it fails depending on the test systems user ID to username passwd(5) mapping. [...]
  • Assign "seen" and "expected" values to local variables to improve contextual information in failed tests. [...]
  • Don't print an orphan newline when the source code formatting test passes. [...]

Debian

• python-django (3.2.6-1) — New upstream bugfix release.

• memcached:

  • 1.6.10+dfsg-1 — New upstream release.
  • 1.6.10+dfsg-2 — Restore systemd sandboxing as the Debian version of systemd supports these features. Thanks to Robie Basak for the bug and patch. (#892922)

• redis:

  • 6.2.5-2 — Explicitly specify USE_JEMALLOC to override upstream's detection of ARM architecture systems. This was affecting reproducibility as the aarch64 kernel flavour was using Jemalloc (whilst armv7l was not).
  • 6.2.5-3 — Skip OOM-related tests on incompatible platforms. (#982122)
  • 6.2.5-4 — Use /run instead of /var/run for PID and UNIX socket files. Thanks to @MichaIng-guest for the patch.

• python-libarchive-c (3.1-1) — Non-maintainer upload to Debian experimental for the 3.x branch. (python-libarchive-c is used by diffoscope.

I also sponsored an upload of adminer version 4.8.1-1 for Alexandre Rossi.

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

This month, I:

• Investigated and triaged: asterisk (CVE-2021-32686), gitit (CVE-2021-38711), glances (CVE-2021-23418), gradle (CVE-2021-32751), hoteldruid (CVE-2021-37832 & CVE-2021-37833), libapache2-mod-auth-mellon (CVE-2021-3639), pjproject (CVE-2021-32686), postgresql-9.1 and `transfig (CVE-2020-21675).

• Frontdesk duties, responding to user/developer questions, reviewing others' packages, participating in mailing list discussions etc. in addition to attending our monthly meeting as well as our public meeting at DebConf21.

• Issued DLA 2727-1 as it was discovered that there was a code injection issue in PyXDG a library used to locate freedesktop.org configuration/cache directories.

• Issued DLA 2728-1 to address four issues in the VideoLAN/VLC media player caused by buffer overflows and NULL-pointer deferences.

• Issued DLA 2729-1 for the Asterisk telephony system to address an issue where if a particular driver received a packet that contained an unsupported media format, a crash could have occurred.

• Issued DLA 2736-1 for lynx (and ELA-472-1 for lynx-cur) to prevent a remote credential leak in the Lynx text-based web browser. After this update, it now correctly handles authentication subcomponents in URIs (eg. the user:pass substring in https://user:pass@example.com) to avoid remote attackers discovering cleartext credentials in SSL connection data.

• Issued DLA 2717-2 to address a regression in the previous update for Redis. A test was not correctly backported from the latest upstream release which meant that binaries were not available on all LTS platforms. The Redis server code itself was unaffected.

You can find out more about the LTS and ELTS projects via the following video:

You can subscribe to new posts via email or RSS.