Free software activities in October 2019

Here is my monthly update covering what I have been doing in the free software world during October 2019 (previous month):

• Made some changes to my tickle-me-email library which implements Gettings Things Done-like behaviours in IMAP inboxes including ensuring attached files have their "basename" path as the filename metadata, not the full/absolute one passed to the program [...].

• As part of my duties of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meeting and participated in various licensing and other discussions occurring on the internet, as well as the usual internal discussions regarding logistics and policy etc.

• Opened pull requests to make the build reproducible in:

  • SPIRV-Tools, part of the Khronos 3D graphics processing libraries etc. to ensure a timestamp does not vary with the build timezone. [...]

  • The "stacked" Git stgit tool. [...]

  • The traitlets Python type-checking/enforcement library to make sure that traitlet.Set values are returned in a sorted order. [...]

  • The flask microframework for building Python web applications to make the documentation build reproducibly. [...]

  • The ROS Robot Operating System code generation library for Python to ensure that generated struct constructs are reproducible. [...]

  • khard, a commandline address book utility. [...]

• Even more hacking on the Lintian static analysis tool for Debian packages:

  • New checks/features:

    • Warn about missing ${sphinxdoc:Depends} when --with sphinxdoc or dh_sphinxdoc is used. (#940999, #943711)
    • Warn about packages that use the deprecated $ADTTMP autopkgtest variable. [...]
    • Add 4.4.1 as a known Standards-Version. [...]

  • Bug fixes / false-positive corrections:

    • Skip entirely whitespace lines when checking the override_dh_auto_test-does-not-check-DEB_BUILD_OPTIONS tag. (#943684)
    • Avoid false-positives in non-consecutive-debian-revision by only performing the test when Debian revisions (eg. -2, -3, etc.) are whole integers. (#941395, #942013)
    • Add python2 and python2-minimal as satisfying dependencies for Python packages. (#942658)
    • Bump minimum required version for a debhelper dependency now that version 10 is satisfied in the oldstable distribution. (#942632)
    • Don't emit package-supports-alternative-init-but-no-init.d-script when we have a foo.service and foo.timer pair. (#933109)
    • Don't warn about package-supports-alternative-init-but-no-init.d-script and systemd-service-file-missing-install-key and for units installed in a subdirectory of (for example) /lib/systemd/system. (#941419)

  • Reporting/output:

    • Include more verbose debugging info (eg. the level of concurrency) when running autopkgtests. [...]
    • Correct/improve grammar of the non-consecutive-debian-revision long description. [...]

  • Misc:

    • Don't build Git tags on salsa. [...]
    • Add a trailing ellipsis to the "Preparing X work directories" message to denote processing is occuring in the background. [...]
    • Improve the test package generation logging output to include a current/total progress indicator. [...]
    • Refresh data/fields/perl-provides. [...][...]

Reproducible builds

Whilst anyone can inspect the source code of free software for malicious flaws almost all software is distributed pre-compiled to end users.

The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.

The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom.

Conservancy acts as a corporate umbrella, allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.

This month, I:

• Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.

• Drafted, published and publicised our monthly report.

• Filed pull requests upstream to make the build reproducible in:

  • SPIRV-Tools, part of the Khronos 3D graphics processing libraries etc. to ensure a timestamp does not vary with the build timezone. [...]

  • The "stacked" Git stgit tool. [...]

  • The traitlets Python type-checking/enforcement library to make sure that traitlet.Set values are returned in a sorted order. [...]

  • The flask microframework for building Python web applications to make the documentation build reproducibly. [...]

  • The ROS Robot Operating System code generation library for Python to ensure that generated struct constructs are reproducible. [...]

  • khard, a command-line address book utility. [...]

• In Debian:

  • Kept isdebianreproducibleyet.com up to date. [...]

  • Submitted two following patches to fix specific reproducibility-related toolchain issues. (#943694 & #942342)

  • I also submitted 13 patches to fix specific reproducibility issues in bst-external, checkinstall, cloudkitty, designate, elph, flask, frobby, gobject-introspection, khard, pmemkv, python-oslo.reports, squeak-plugins-scratch & stgit.

• trydiffoscope is my web-based version of the diffoscope in-depth and content-aware diff utility. This month I moved the tool to depend on the python3-docutils package over python-docutils to allow for Python 2.x removal (#943293) as well as updating the packaging to the latest Debian standards and conventions [...][...][...].

• I spent some more time working on our website this month too, including:

  • Improving the formatting of our reports. [...]
  • Adding some missing space. [...]
  • Tidying the new "Testing framework" links. [...]
  • Updating the monthly report template. [...]

• strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build. This month, I dropped the bug_803503.zip test fixture as it is no longer compatible with the latest version of Perl's Archive::Zip. (#940973)

diffoscope

diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues. This month, I made the following changes:

• Disassembling and reporting on files related to the R (programming language):

  • Expose an .rdb file's absolute paths in the semantic/human-readable output, not hidden deep in a hexdump. [...]
  • Rework and refactor the handling of .rdb files with respect to locating the parallel .rdx prior to inspecting the file to ensure that we do not add files to the user's filesystem in the case of directly comparing two .rdb files or — worse — overwriting a file in is place. [...]
  • Query the container for the full path of the parallel .rdx file to the .rdb file as well as looking in the same directory. This ensures that comparing two Debian packages shows any varying path. [...]
  • Correct the matching of .rds files by also detecting newer versions of this file format. [...]
  • Don't read the site and user environment when comparing .rdx, .rdb or .rds files by using Rscript's --vanilla option. [...][...]
  • Ensure all object names are displayed, including ones beginning with a fullstop (.) [...] and sort package fields when dumping data from .rdb files [...].
  • Mask/hide standard error when processing .rdb files [...] and don't include useless/misleading NULL when dumping data from them. [...]
  • Format package contents as foo = bar rather than using ugly and misleading brackets, etc. [...] and include the object's type [...].
  • Don't pass our long script to parse .rdb files via the command line; use standard input instead. [...]
  • Call thedeparse function to ensure that we do not error out and revert to a binary diff when processing .rdb files with internal "vector" types; they do not automatically coerce to strings. [...]
  • Other misc/cosmetic changes. [...][...][...]

• Output/logging:

  • When printing an error from a command, format the command for the user. [...]
  • Truncate very long command lines when displaying them as an external source of data. [...]
  • When formatting command lines ensure newlines and other metacharacters appear escaped as \n, etc. [...][...]
  • When displaying the standard error from commands, ensure we use the escaped version. [...]
  • Use "exit code" over "return code" terminology when referring to UNIX error codes in displayed differences. [...]

• Internal API:

  • Add ability to pass bytestring input to external commands. [...]
  • Split out command-line formatting into a separate utility function. [...]
  • Add support for easily masking the standard error of commands. [...][...]
  • To match the libarchive container, raise a KeyError exception if we request an invalid member from a directory. [...]
  • Correct string representation output in the traceback when we cannot locate a specific item in a container. [...]

• Misc:

  • Move build-dependency on python-argcomplete to its Python 3 equivalent to facilitate Python 2.x removal. (#942967)
  • Track and report on missing Python modules. (#72)
  • Move from deprecated $ADTTMP to $AUTOPKGTEST_TMP in the autopkgtests. [...]
  • Truncate the tcpdump expected diff to 8KB (from ~600KB). [...]
  • Try and ensure that new test data files are generated dynamically, ie. at least no new ones are added without "good" reasons. [...]
  • Drop unused BASE_DIR global in the tests. [...]

Debian

I filed two patches against the r-base package for not respecting the nocheck and nodoc build profiles respectfully (#942867 & #942870) as well as filing a bug against python3-pluggy for missing a dependency on python3-importlib-metadata (#943320).

Uploads

• python-django:

  • 2.2.6-1 — New upstream bugfix release
  • 3.0~beta1-1 — New upstream beta release

• memcached:

  • 1.5.19-1 — New upstream release
  • 1.5.19-2 — Add iproute2 to the autopkgtest dependencies. (#942183)

• xtrlock (2.12) — Disable multitouch events too, not just regular mouse events. (CVE-2016-10894)

• hiredis (0.14.0-4) — Rework and refresh packaging, dropping some cruft (eg. --dgbsym-migration, old build-dependencies, etc), adding continuous integration tests, etc.

• django-auto-one-to-one (3.3.0-1) — New upstream release.

• python-redis (3.3.11-1) — New upstream release.

• adminer (4.7.4-1) — Sponsored upload on behalf of Alexandre Rossi for a new upstream release, etc.

Debian LTS

This month I have worked 18 hours on Debian Long Term Support (LTS) and 12 hours on its sister Extended LTS project.

• Frontdesk duties, responding to user/developer questions, reviewing others' packages, participating in mailing list discussions, etc.

• Investigated and triaged ansible, graphite-web, haproxy (CVE-2019-18277), jackson-databind (CVE-2019-17267), libapache2-mod-auth-openidc, libsoup (CVE-2019-17266), libtomcrypt, lz4 (CVE-2019-17543), otrs2 (CVE-2019-16375), python-ecdsa (CVE-2019-14853, etc.), python3.4 (CVE-2019-18348), rsyslog, unoconv (CVE-2019-17400), vips (CVE-2019-17534), xen, xtrlock, etc.

• Issued DLA 1944-1 for libapreq2, a library for manipulating HTTP requests.

• Issued DLA 1951-1 to fix a denial of service vulnerability in the libtomcrypt cryptographic library.

• Issued DLA 1952-1 in order to address two vulnerabilities in the rsyslog system/kernel logging daemon in the parsers for AIX and Cisco log messages respectfully

• Issued DLA 1959-1 for the xtrlock screen locking utility to ensure that special multitouch devices were being disabled too.

You can find out more about the Debian LTS project via the following video:

FTP Team

As a Debian FTP assistant I ACCEPTed 25 packages: backintime, celery-batches, eslint, golang-github-containers-image, gtk-d, jsbundle-web-interfaces, networkx, node-eslint-plugin-eslint-plugin, node-eslint-plugin-node, node-eslint-scope, node-eslint-visitor-keys, node-esquery, node-file-entry-cache, node-flatted, node-functional-red-black-tree, node-ignore, node-leche, node-mock-fs, node-proxyquire, numpy, openvswitch, puppet-module-voxpupuli-collectd, pyrsistent, python-dbussy & z3.

I additionally filed 5 RC bugs against packages that had potentially-incomplete debian/copyright files against backintime, celery-batches, networkx, openvswitch & z3.

You can subscribe to new posts via email or RSS.