Free software activities in January 2019

Here is my monthly update covering what I have been doing in the free software world during January 2019 (previous month):

• My activities as the current Debian Project Leader are covered in my "Bits from the DPL" email to the debian-devel-announce mailing list.

• For the Tails privacy-oriented "live" operating system:

  • Merged/updated the squashfs filesystem tool to ensure that it generates reproducible filesystems in Debian buster, sending the patches upstream to Debian too. [...]

  • Updated the custom APT overlays and related "serials" to ensure that the content injection attack in APT (CVE-2019-3462) applies to both the final images and during the build process. [...]

  • Reviewed/tested a large number of contributions from others. [...][...][...][...][...][...][...][...][...][...][...][...]

• Arranged for a Debian Bug Squashing Party to take place as part of the foss-north.se conference in Gothenburg, Sweden during April. [...].

• Represented Debian (and the free software community) in general as part of my duties of being on the board of directors of the Open Source Initiative at our monthy board meeting and in/around various licensing discussions occurring on the internet.

• Proposed two pull requests for systemd to add support for Purism Librem 13 v4 keyboards [...] and to correct some spelling errors [...].

• As part of preparing a new version of GNU mtools disk utilities for Debian, I sent patches upstream to fix a race condition in the install target [...] and to correct number of grammatical errors in the documentation [...].

• Continued to maintain a set of module repositories forked from prior to Redis Labs relicensing a number of AGPL-licensed Redis modules with the "Commons Clause" amendment.

• Updated travis.debian.net (my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform) to restore the Dockerfile within the Docker build process itself fixing some rare build failures. [...]

• Created a pull request against the shadow tool that manages, inter alia, the /etc/passwd and /etc/shadow file. This is was after Johannes Schauer noticed that the sp_lastchg field in the latter was no longer reproducible in Debian bug #917773. In addition, I created a number of pull requests to make upstream build proceses reproducible, including Ansible, SatPy, VIT, splitpatch, etc.

• Curiously, my first ever contribution to the Django web development framework was removed/reverted.

• Even more hacking on the Lintian static analysis tool for Debian packages, including:

  • New features:

    • Check for packages maintainer scripts supporting legacy versions of Debian. (#917566)
    • Group warnings that only differ on the architecture in the HTML reports. (#919162)
    • Check for packages that use dh_runit without specifying a Breaks on the ${runit:Breaks} substvar. (#920299)
    • Add a check for empty executable files in $PATH. (#919458)
    • Check for packages that ship sass-cache directories. (#920593)
    • Locate and process .buildinfo mentioned in a .changes. (#920228)
    • Check for pkg-config files that reference unknown shared objects via (for example) Libs: -lfoo. (#920699)
    • Check for inconsistencies between debian/copyright and the information in AppStream metadata files. (#907072)
    • Add brightness to the list of initscripts that do not need a corresponding service. (#918459)
    • Check for packages that ship headers in /usr/include/python3.x. (#919979)
    • Check for manual calls to dpkg-maintscript-helper(1). (#917567)

  • Bug fixes:

    • Prevent false-positives in the command-with-path-in-maintainer-script for automatically-added sections. (#920568)
    • Ignore "duplicate" .buildinfo files generated using mergechanges(1). (#920469)
    • Require that build paths are longer than / when checking file-references-package-build-path. [...]
    • Rewrite "old" version calculation to prevent false-positives in the maintainer-script-supports-ancient-package-version tag. (#920638)
    • Ignore quilt(1) .pc/ directories when checking inconsistent-appstream-metadata-license. (#920647)
    • Allow comments in debian/tests/control files. (#917964)
    • Permit -b branch specifications in Mercurial Vcs-Hg control fields. (#920355)
    • If a changelog entry is missing a maintainer (eg. "-- "), don't assume that it is a non-maintainer upload. (#920184)
    • Prevent false-positives for leading directory entries when checking for files installed "outside" of /usr. [...]
    • Prevent a variety of false-positives when checking "new style" init scripts that use "#!/usr/bin/env /lib/init/init-d-script" as a shebang. (#919604)
    • Fix symbols-file-missing-build-depends-package-field when a package contains more than one library. (#918473)
    • Prevent false-positives in maintainer-script-should-not-use-dpkg-maintscript-helper around automatically added sections. (#917567)
    • Don't emit a pedantic warning for Debhelper "compat" level 12 until Debian bullseye. (#918809)

  • Reporting:

    • Fix FTBFS by avoiding "self" false-positives when checking for file-references-package-build-path in Lintian's own test suite. (#920536)
    • Guile object files do not objdump(1) correctly so exclude them from a number of tests. (#918444)
    • Update Debian Policy and other manual references since these documents have migrated to Sphinx or newer versions of DocBook (#918963)
    • Print Carp tracebacks in --debug mode. [...]
    • Add a reference to the official specification for unnecessary-source-date-epoch-assignment. [...]
    • Include the upload and release dates in the output of the maintainer-script-supports-ancient-package-version tag. [...]
    • Clarify that spelling-* tags also correct various grammatical errors too.

  • Misc:

    • Drop the versioned-dependency-satisfied-by-perl tag. (#917967)
    • Split wrong-path-for-interpreter into "wrong" and "incorrect" variants. (#917790)

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws almost all software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom.

Conservancy acts as a corporate umbrella, allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month:

• Made considerable progress towards making the Debian Installer images reproducible with a number of rounds of code review, a subsequent merge of my merge request and the closing of the corresponding bug report.

  After further testing, I filed #920676 with a patch to ensure the build is reproducible regardless of the underlying filesystem ordering and also filed #920631 correcting a corresponding issue regarding the user's umask(2).

• Posted a historical summary and a request for action on Fontconfig's mailing list in order that a solution may be found and included in Debian buster. This has resulted in considerable rounds of discussion and progress.

• Presented at Université de Rennes, France on reproducible builds and how they can prevent developers from becoming targets of various attacks as well as many of the tools and processes that team uses.

• Created a pull request against the shadow tool that manages, inter alia, the /etc/passwd and /etc/shadow file. This is was after Johannes Schauer noticed that the sp_lastchg field in the latter was no longer reproducible in Debian bug #917773.

• Reproducibility patches for GNU mtools finally entered the Debian unstable distribution via my upload of the mtools 4.0.23-1 package via the new "salvaging" process.

• Fixed disorderfs, our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues. to ignore the return values to fsyncdir to ensure dpkg(1) (for example) can flush /var/lib/dpkg without error. [...]

• Made a huge number of updates to our reproducible-builds.org project website, including:

  • Move our homepage, contribute, tools, resources and events pages to new visual style. [...][...][...][...][...][...][...]

  • "Markdown-ise" the Paris summit event documentation. [...]

  • Tidy and highlight the display of our sponsors [...][...] and generally improve a number of styles, such as blockquotes, ensure linked headings don't inherit link styling, etc. [...][...][...].

  • Update the support mechanisms for the weekly reports, such as dropping the migrate-blog-posts script [...] as well as fixing some title handling code [...] [...].

  • Split, tidy and expand footer [...][...][...] and link the main heading element of blog posts "back" to themselves [...].

  • Set a temporary logo for Christmas. [...][...]

• Opened Debian bug #919207 requesting that the squashfs-tools package (which creates and manipulates read-only compressed file systems) applies a patch to remove non-deterministic data introduced by a "fragmentation deflator" thread. This was the final patch required for reproducible images for (at least) Tails.

• Updated diffoscope (our in-depth "diff-on-steroids" utility which helps us diagnose reproducibility issues in packages) to:

  • Add a note to the "Files similar despite different names" message to clarify that a lower score is more similar" [...] and also prefer to emit a comment that files are "identical" rather than having a "fuzziness score of zero". [...].

  • Avoid crashing if we were unable to successfully extract a guestfs-backed filesystem. (#901982)

  • Fix inverted logic and invalid reference to file in the FreePascal comparator. [...]

  • Re-enable Gnumeric as a Build-Depends. [...] [...]

  • Avoid clumsy profiling title length calculations by switching to Markdown syntax. [...]

  • Drop the printing of dpkg-query(1) output whilst running tests. [...]

• Updated the SSL certificate for try.diffoscope.org to ensure validation after the deprecation of TLS-SNI-01 validation in LetsEncrypt. [...]

• Authored a number of reproducibility-specific patches, including:

  • #918533 filed against gnucap-python.

  • squashfskit (use time(0) instead of time(-1))

  • vit (merged: date --utc) (#918534)

  • #919566 filed against satpy (merged upstream).

  • #920409 filed against splitpatch (forwarded upstream)

  • #920411 filed against mongo-c-driver.

  • #920591 filed against lambda-align2.

  • #920592 filed against roaraudio.

  • #920594 filed against papi.

  • #920595 filed against ukui-themes.

• Kept isdebianreproducibleyet.com up to date. [...]

• Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.

• Drafted, published and publicised our weekly reports. (#192, #193, #194, #195, & 196)

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

• "Frontdesk" duties, triaging CVEs, responding to user questions, testing packages from other maintainers. etc.

• Investigated and triaged CVE-2018-16889, CVE-2018-20677, CVE-2018-20676, CVE-2019-5010, CVE-2019-6130, CVE-2019-6131, CVE-2018-1320, CVE-2019-3462, CVE-2016-10735, CVE-2018-20683, CVE-2018-20030, CVE-2019-6129, CVE-2018-20673 & CVE-2018-20671.

• Backported and issued DSA-4363 for python-django to fix a content-spoofing vulnerability in the default "404" pages. I also ensured this reached users of the stable distribution via a separate backport which was announced as DLA 1629-1.

• Issued ELA-76-1 for apt to address the infamous CVE-2019-3462 content injection attack, backporting the patch from Debian jessie.

Debian uploads

• python-django:

  • 1.11.18-1 — New upstream security release.
  • 2.1.5-1 — New upstream security release,
  • 2.2~alpaha1-1 — New upstream alpha release.

• redis:

  • 5.0.3-3 — Fix FTBFS on hurd-i386 by updating a patch to avoid MAXPATHLEN reference.

  • 5.0.3-4 — Fix a cross build failure when building the Debian-suplied Lua libraries. (#919682)

• mtools (4.0.23-1) — New upstream release, salvaging the package via #916127.

• libfiu (0.98-2) — Honour CPPFLAGS and LDFLAGS when building shared libraries to ensure hardening is applied to generated objects.

• bfs:

  • 1.3.1-1 & 1.3.2-1 — New upstream releases.

  • 1.3.2-2 — Only require libacl1-dev and libcap-dev on systems with the Linux kernel. (#920288)

• redisearch:

  • 1.2.1-2 — Define CLOCK_MONOTONIC_RAW for kFreeBSD.

  • 1.2.1-3 — Check for __FreeBSD_kernel__ over __FreeBSD__ for CLOCK_MONOTONIC_RAW.

  • 1.2.1-4 — Pass -ffile-prefix-map for a reproducible build.

• installation-birthday (12) — New upstream release.

I also performed a sponsored uploads of c-graph, connman-gtk, connman-ui and elpy.

FTP Team

As a Debian FTP assistant I ACCEPTed 85 packages: agg, akira, apt-config-auto-update, beancount, botan, cairosvg, chaosread, corosync-qdevice, deepdiff, desktopfolder, dh-vim-addon, distorm3, exempi, fava, fonts-noto, fonts-quicksand, gcc-9, gnustep-back, gnustep-base, gnustep-gui, heudiconv, ilmbase, kamailio, leaflet, leaflet-image, leaflet-markercluster, libcatmandu-filestore-perl, libgeoip2-perl, libical3, libjs-rtcpeerconnection-shim, libjs-sdp, libjs-webrtc-adapter, libjwt, liblist-utilsby-xs-perl, libmaxmind-db-reader-xs-perl, libnfs, libpillowfight, libqmatrixclient, libwin32-exe-perl, lighttpd, lix, looking-glass, lrslib, musescore-general-soundfont, musescore-general-soundfont-small, netdata, nextcloud-desktop, node-chai, node-domino, node-yarnpkg, omegat, openexr, pacemaker, package-update-indicator, pdfarranger, pkg-js-tools, plinth, pmdk, ptunnel-ng, popper.js, progress-linux, pyninjotiff, pyphen, python-shade, rdkit, ruby-asciidoctor-pdf, ruby-mini-mime, ruby-prawn-icon, ruby-prawn-svg, ruby-voight-kampff, rust-rand-0.5, rust-rand-core-0.2, rust-tokio, silkaj, slirp4netns, spirv-tools, squashfuse, twitter-bootstrap3, uglify-js, use-package, utox, valentina, vulkan-validationlayers, xdg-dbus-proxy & yaz.

I additionally filed 12 RC bugs against packages that had potentially-incomplete debian/copyright files against beancount, fava, libpillowfight, libwin32-exe-perl, netdata, netdata, openexr, pdfarranger, python-shade, rust-rand-0.5, spirv-tools ptunnel-ng & vulkan-validationlayers.

You can subscribe to new posts via email or RSS.