Free software activities in February 2019

Here is my monthly update covering what I have been doing in the free software world during February 2019 (previous month):

• My activities as the current Debian Project Leader are covered in my "Bits from the DPL" email to the debian-devel-announce mailing list.

• Gave talks at FOSDEM and CopyLeftConf on the "Commons Clause" license amendment, represented Debian on GNOME's Advisory Board at a bi-annual meeting and participated in the Open Source Initiative's monthly board meeting. I also gave a talk on reproducible builds at Speck&Tech #31.

• Opened a pull request for the Redis key-value database to not assume the pointer size on __x86_64__ to avoid warnings when building on the x32 architecture. [...]

• Fastmail recently updated their user interface which broke my Fastmail Enhancement Suite Chrome browser extension, requiring some attention. [...][...]

• Opened pull requests for the python-octaviaclient component of the Openstack component [...] and against the Sphinx Python documentation generator to ensure that Graphviz filenames are reproducible [...].

• Corrected a {% load switch %} reference in the README of my simple switch tag my for Django projects. [...]

• Even more hacking on the Lintian static analysis tool for Debian packages:

  • New features:

    • Emit an experimental warning for packages shipping cron scripts without an equivalent systemd .timer file. (#922862)
    • Add /lib/runit/invoke/run as a known interpreter. (#923232)
    • Make orig-tarball-missing-upstream-signature a dsc check so it appears when running against non-.changes files. (#922557)
    • Detect .git.git (etc.) in Vcs-Git headers. (#921084)

  • Bug fixes:

    • Prevent false positives in pkg-config-references-unknown-shared-library by also tracking static libraries (#921872), ignore entries such as -lfoo{install_suffix} as they are interpolated at runtime by Autoconf (#922511), create an exception list and populate it with shared objects shipped by libc6-dev (#922402), also including the libraries shipped in the libgcc1 package and finally add gcc as a manual exception. [...]
    • Don't emit orig-tarball-missing-upstream-signature if the package uses opts=mode=git,pgpmode=gittag. (#920763)
    • Ignore spelling errors in patch author names. (#922233)
    • Correct reference to the mips64el architecture to ensure we emit binary-from-other-architecture. (#921573)
    • Don't emit source-contains-prebuilt-java-object or build-depends-on-obsolete-package for Lintian itself. [...][...]
    • Don't emit source-nmu-has-incorrect-version-number for uploads to backports. (#923060)
    • Use the source package name (not the section!) when checking uses-dpkg-database-directly. (#922530)
    • Check all combinations of processables and binary package names to avoid false-positives or unused override warnings in spelling checks. (#921814)
    • Assume that license files themselves do not require DEP-5 copyright file coverage. (#921752)

• In Debian, I contributed the following patches:

  • Enable verbose GCC output in cpio's build process with --disable-silent-rules. (#922056)

  • libdatetime-format-strptime-perl: Correct "safe" typo in Changes & debian/NEWS. (#922034)

  • Reference the texlive-latex-base package in the install pdflatex error message in pandoc. (#922418)

• I also made the following uploads to Debian:

  • python-django (2.2~beta1-1 & 1.11.20-1) — New upstream releases.

  • bfs (1.3.3-1) — New upstream release.

  • python-httplib2 (0.11.3-2) — Add Breaks on python{,3}-pysimplesoap (#921882).

  • lastpass-cli (1.3.1-7) — Pass VERBOSE=1 for non-quiet build logs in order to expose them on the Buildd Log Scanner.

  • python-redis:

    • 3.1.0-1 — New upstream release.
    • 3.1.0-2 — Attempt to fix autopkgtests. (#922327)
    • 3.2.0-1 — New upstream release.
  • c-graph 2.0.1-2, connman-gtk 1.1.1+git20180626.b72c6ab-1 & python-css-parser 1.0.4-1 (sponsored uploads).

• As a Debian FTP assistant, I ACCEPTed 51 packages: cpptest, daps, eye, filemanager-actions, fonts-b612, fonts-roadgeek, gnome-books, golang-github-cactus-go-statsd-client, golang-github-codahale-hdrhistogram, golang-github-crossdock-crossdock-go, golang-github-go-xorm-core, golang-github-hetznercloud-hcloud-go, golang-github-teris-io-shortid, golang-github-thcyron-uiprogress, golang-github-uber-go-atomic, golang-github-vividcortex-mysqlerr, golang-github-yudai-golcs, hcloud-cli, isospec, libappimage, libdist-inkt-role-git-perl, libgit-sub-perl, libjs-bootbox, libsystem-sub-perl, libusrsctp, node-base64url, node-node-rsa, node-solid-keychain, node-trust-json-document, node-trust-jwa, node-trust-webcrypto, nrepl-clojure, ontospy, open-infrastructure-compute-tools, open-infrastructure-service-tools, open-infrastructure-storage-tools, open-infrastructure-system-tools, pollen, pollinate, progress-linux-metapackages, puppet-module-panko, python-apptools, ruby-blade, ruby-blade-qunit-adapter, ruby-blade-sauce-labs-plugin, ruby-faye, ruby-faye-websocket, ruby-sprockets-export, rust-grep-searcher, spf-engine & x2gobroker. I also filed 3 RC bugs against packages that had potentially-incomplete debian/copyright files (against filemanager-actions, gnome-books & ruby-blade-qunit-adapter).

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

• Investigated and triaged gsoap (CVE-2019-7659), kde4libs, python-django (CVE-2019-6975), spice-xpi (CVE-2010-2792), etc.

• Frontdesk duties, responding to user/developer questions, reviewing others' packages, etc.

• Issued DLA 1664-1 as it was discovered that there was a denial of service vulnerability (or even the ability to conduct private key recovery) within the elliptic curve cryptography handling in the Go programming language libraries.

• Issued DLA 1667-1 for dovecot where a flaw in the TLS username handling could lead to an attacker being able to log in as anyone else in the system.

• Issued DLA 1672-1 for curl, correcting three heap/stack-based vulnerabilities.

• Issued DLA 1681-1 to address denial of service vulnerability in gsoap, a C/C++ language binding used for SOAP-based web services.

• Issued DLA 1660-2 for rssh as it was discovered that the fix for the security vulnerability released for rssh in 2.3.4-4+deb8u2 introduced a regression that blocked scp(1) of multiple files.

• Issued ELA 82-1 to address two vulnerabilities in the libarchive multi-format compression library.

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws almost all software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom.

Conservancy acts as a corporate umbrella, allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:

• Made more progress towards making the Debian Installer images reproducible, performing some further testing of the generated images resulting in two patches to ensure that builds were reproducible regardless of both the user's umask(2) (#920631) and even the underlying ordering of files on disk (#920676).

• Presented at Speck&Tech #31 entitled Open Security in Trento, Italy.

• Implemented a check in the Lintian static analysis tool that performs automated checks against Debian packages in order to add a check for .sass-cache directories. As as they contain non-deterministic subdirectories they implicitly cause an unreproducible build. (#920593)

• disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into filesystems for easy and reliable testing. In February I fixed an issue this week in the handling of the fsyncdir system call to ensure dpkg(1) can "flush" /var/lib/dpkg correctly [...].

• strip-nondeterminism is our tool that post-processes files to remove known non-deterministic output. This month I adjusted its behaviour to de-duplicate hardlinks via stat(2) before processing to avoid issues when handling files in parallel; as the per-filetype handlers are yet currently guaranteed to be atomic, one process could temporarily truncate a file which can cause errors in other processes operating on the "same" file under a different pathname. This was thus causing package build failures in packages that de-duplicate hardlinks in their build process such as the Debian Administrator's Handbook. (#922168)

• diffoscope is our in-depth "diff-on-steroids" utility which helps us diagnose reproducibility issues in packages. This month, I made the following changes:

  • Improve the displayed comment when falling back to a binary diff to include the file's type. (#49)
  • Add a --exclude-directory-metadata=recursive option to support ignoring timestamp (etc.) differences within nested containers. (#907600).
  • Add support for comparing .crx Chrome browser extensions. (#41)
  • Adjust the behaviour to not look for adjacent -dbgsym Debian package files automatically anymore to align better with users' expectations. The existing behaviour can be re-enabled by specifying the new --use-dbgsym flag (#44 / #920701).
  • Add support for comparing MP3 and files with similar metadata. (#43)
  • Replace the literal xxd(1) output in tests/data/hello.wasm with its binary equivalent (#47) and ensure both WebAssembly test data files are actually unique. (#42)
  • Catch tracebacks when mounting invalid filesystem images under guestfs. [...]
  • Fix tests when using Ghostscript 9.20 vs 9.26 for the Debian stable distribution and for stable with the security repositories enabled. [...][...]
  • Compare .asc PGP signatures as text, not as a hexdump of the text. (#908991).
  • Replace over 8 MB of Android boot ROM test suite fixtures with 14 KB equivalents to reduce the size of the release tarball. (#894334).
  • Additionally compare pgpdump(1) output when comparing PGP signatures. [...]
  • --help output improvements:
    • Include links to the diffoscope homepage and bug tracker. [...]
    • Refer to the Debian package names when indicating how to obtain the tlsh and argcomplete Python modules. [...]
    • Indent and wrap the list of supported file formats. [...]
  • Adopt the Black code formatter:
    • Run against the existing source code. [...].
    • Add an initial black setup in a PEP 518 pyproject.toml file [...], updating MANIFEST.in to include it in future release tarballs. [...]
    • Add a test to ensure future source code satisfies the formatter. [...]
    • Allow GitLab CI failures in stable-bpo due to new dependency. [...]
  • Drop a DOS/MBR "source string" test. [...]
  • Drop ubuntu-devel from internal test matrix due to a linux-firmware package installation issue. [...]
  • Uploaded version 112 to Debian unstable, dropped an errant </ul> from the diffoscope.org website [...] and also applied the "black" code formatter to the try.diffoscope.org client [...].

• Updated the SSL certificate for try.diffoscope.org to ensure validation after the deprecation of TLS-SNI-01 validation in LetsEncrypt, increased the diskspace and memory available for buildinfo.debian.net and kept isdebianreproducibleyet.com up to date [...].

• Filed upstream pull requests for the python-octaviaclient component of the Openstack framework [...] and the Sphinx Python documentation generator [...] to make their

• Submitted seven patches to fix specific reproducibility issues in heudiconv, libiio, lmfit-py, node-lunr, python-octaviaclient, sphinx & x2gobroker.

• Drafted, published and publicised our weekly reports. (#197, #198, #199 & 200) and categorised a huge number of packages and issues in the Reproducible Builds "notes" repository.

You can subscribe to new posts via email or RSS.