Here is my monthly update covering what I have been doing in the free software world during March 2020 (previous month):
-
Patched the browserify-lite Javascript compiler to ensure that module dependency maps are rendered in a deterministic order. (#14)
-
Pushed a change to the "pmemkv" key-value data store to make their documentation build reproducibly. (#615)
-
As part of my duties of being on the board of directors of the Open Source Initiative and Software in the Public Interest I attended their respective monthly meetings and participated in various licensing and other discussions occurring on the internet as well as the usual internal discussions regarding logistics, licensing, policy and liaising with the ClearlyDefined project and so on.
Congratulations to Megan Sanicki and Josh Simmons who were elected and re-elected respectively to the OSI Board's two individual member seats and to Italo Vignoli who was elected to the affiliate member seat. These newly-elected Directors will took their seats on March 20, 2020.
-
Opened a pull request against the onionbalance load balancer for hidden Tor services to fix test failures under Pytest 3.x. (#75)
-
Filed an issue against IMAP Spam Begone (isbg), a script by Louis-Philippe Véronneau (pollo) that makes it easy to process an email inbox using SpamAssassin, to report that a duplicate documentation entry includes a nondeterministic value taken from the value of the XDG cache directory. (#151)
-
As part of being on being on the judging panel of the OpenUK Awards I am pleased to announce that after some discussion nominations are now open until 15th June in five different categories.
-
Merged a number of contributions to my django-cache-toolbox "non-magical" caching library for Django web applications, including caching negative relation lookups locally (#14) and to include the
README
file in the package long description (#17). -
Made some small changes to my tickle-me-email library which implements Gettings Things Done (GTD)-like behaviours in IMAP inboxes to support to optionally limiting the number of messages in the
send-later
functionality. [...]
In addition, I did even more hacking on the Lintian static analysis tool for Debian packages, including:
-
New features:
- Check for
py3versions -i
in autopkgtests anddebian/rules
files. (#954763) - Warn when
py3versions -s
is used without apython3-all
dependency. (#954763) - Expand
possible-missing-colon-in-closes
to also check for semicolons used in place of colons. (#954484) - Check for new packages that use a date-based versioning scheme (eg.
YYYYMMDD-1
) without a0~
suffix. (#953036)
- Check for
-
Improvements:
- Don't emit
duplicate-short-description
orduplicate-long-description
if they contain substitution variables. (#947168) - Don't report about using a newer
Standards-Version
when uploading to the backports repository. [...] - Don't check the length of
Checksums-Sha256
fields. (#954798) - Don't check for
DEBIAN/md5sums
control files in udebs. (#954803) - Apply a patch from gregoa to remove some
<!nocheck>
annotations; some packages are required to build successfully. (#954338)
- Don't emit
-
Misc:
- Correct reference to build dependencies in the long description of the
debian-rules-uses-installed-python-versions
tag. [...] - Make some cosmetic improvements to the
CONTRIBUTING.md
file. [...] - Correct reference to a bug in a previous
debian/changelog
entry. [...] - Avoid indenting approximately 150 lines by returning early from a subroutine and other code improvements. [...]
- Correct reference to build dependencies in the long description of the
Reproducible builds
One of the original promises of open source software is that distributed peer review and transparency of process results in enhanced end-user security. However, whilst anyone may inspect the source code of free and open source software for malicious flaws, almost all software today is distributed as pre-compiled binaries. This allows nefarious third-parties to compromise systems by injecting malicious code into ostensibly secure software during the various compilation and distribution processes.
The motivation behind the Reproducible Builds effort is to ensure no flaws have been introduced during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
The initiative is proud to be a member project of the Software Freedom Conservancy, a not-for-profit 501(c)(3) charity focused on ethical technology and user freedom.
Conservancy acts as a corporate umbrella allowing projects to operate as non-profit initiatives without managing their own corporate structure. If you like the work of the Conservancy or the Reproducible Builds project, please consider becoming an official supporter.
This month, I:
-
Filed an issue against IMAP Spam Begone — a script by Louis-Philippe Véronneau (pollo) that makes it easy to process an email inbox using SpamAssassin — to report that a (duplicate) documentation entry includes nondeterministic value taken from the value of the XDG cache directory (#151) and filed an upstream pull requests against the pmemkv key-value data store to make their documentation build reproducibly (#615).
-
Further refined my merge request against the
debian-installer
component to allow all arguments fromsources.list
files (such as[check-valid-until=no]
) in order that we can test the reproducibility of the installer images on the Reproducible Builds own testing infrastructure. (#13) -
Submitted two following patches to fix reproducibility-related toolchain issues within Debian:
-
Submitted eight patches to fix specific reproducibility issues in beep (caused by a variation between
/bin/dash
and/bin/bash
), cloudkitty (due to a default value being taken from the number of CPUs on the build machine), font-manager (embedding the value of@abs_top_srcdir@
into the resulting binary), gucharmap (due to embedding the absolute build path when generating a comment in a header file), infernal (timestamps are injected into a Python example, which should not be shipped anyway), ndisc6 (embeds the value ofCFLAGS
into the binary without sanitising any absolute build paths), node-nodedbi (embedded timestamp in binary) & pmemkv (does not respectSOURCE_DATE_EPOCH
when populating aYEAR
variable). -
Kept isdebianreproducibleyet.com up to date. [...]
-
Continued collaborative work on an academic paper to be published within the next few months.
-
Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
-
Drafted, published and publicised our monthly report.
-
Improved our website, including correcting the syntax of some CSS class formatting [...], improved some "filed against" copy a little better [...] and corrected a reference to
calendar.monthrange
Python method. [...]
In our tooling, I also made the following changes to diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues, including preparing and uploading version 138
to Debian:
-
Improvements:
- Don't allow errors with R script deserialisation cause the entire operation to fail, for example if an external library cannot be loaded. (#91)
- Experiment with memoising output from expensive external commands, eg.
readelf
. (#93) - Use
dumppdf
from thepython3-pdfminer
if we do not see any other differences frompdftext
, etc. (#92) - Prevent a traceback when comparing two R
.rdx
files directly as theget_member
method will return a file even if the file is missing. [...]
-
Reporting:
- Display the supported file formats into the package long description. (#90)
- Print a potentially-helpful message if the PyPDF2 module is not installed. [...]
- Remove any duplicate comparator descriptions when formatting in the
--help
output or in the package long description. [...] - Weaken "Install the X package to get a better output." message to "... may produce a better output." as the former is not guaranteed. [...]
-
Misc:
- Ensure we only parse the recommended packages from
--list-debian-substvars
when we want them fordebian/tests/control
generation. [...] - Add upstream metadata file [...] and add a Lintian override for
upstream-metadata-in-native-source
as "we" are upstream. [...] - Inline the
RequiredToolNotFound.get_package
method's functionality as it is only used once. [...] - Drop the deprecated "
py36 = [..]
" argument in thepyproject.toml
file. [...]
- Ensure we only parse the recommended packages from
The Reproducible Builds project also operates a fully-featured and comprehensive Jenkins-based testing framework that powers tests.reproducible-builds.org. This month, I reworked the web-based package rescheduling tool to:
-
Require a HTTP
POST
method in the web-based scheduler as not only should HTTP GET requests be idempotent but this will allow many future improvements in the user interface. [...][...][...] -
Improve the authentication error message in the rescheduler to suggest that the developer's SSL certificate may have expired. [...]
Debian LTS
This month I have worked 18 hours on Debian Long Term Support (LTS) and 8 hours on its sister Extended LTS project.
-
Investigated and triaged
glibc
(CVE-2020-1751),jackson-databind
,libbsd
(CVE-2019-20367),libvirt
(CVE-2019-20485),netkit-telnet
&netkit-telnet-ssl
(CVE-2020-10188),pdfresurrect
(CVE-2020-9549) &shiro
(CVE-2020-1957), etc. -
In the script that reserves a unique advisory number don't warn about potential duplicate work when issuing a regression in order to avoid this message being missed when it does apply. [...]
-
Frontdesk duties, responding to user/developer questions, reviewing others' packages, participating in mailing list discussions, etc.
-
xtrlock
versions2.8+deb9u1
(#949112) and2.8+deb10u1
(#949113) was accepted to the Debian stretch and buster distributions. -
Issued DLA 2115-2 to correct a regression in a previous fix (a use-after-free vulnerability) in the ProFTPD FTP server.
-
Issued DLA 2132-1 to fix an issue where incorrect default permissions on a HTTP cookie store could have allowed local attackers to read private credentials in libzypp, the library underpinning package management tools such as YaST, zypper and the openSUSE/SLE implementation of PackageKit.
-
Issued DLA 2134-1 to patch an out-of-bounds write vulnerability in pdfresurrect, a tool for extracting or scrubbing versioning data from PDF documents.
-
Issued DLA 2136-1, addressing an out-of-bounds buffer read vulnerability in libvpx, a library implementing the VP8 & VP9 video codecs.
-
Issued DLA 2142-1. It was discovered that there was a buffer overflow vulnerability in slirp, a SLIP/PPP emulator for using a dial up shell account. This was caused by the incorrect usage of return values from
snprintf(3)
. -
Issued DLA 2145-1 and DLA 2145-2 for
twisted
to prevent a large number of HTTP request splitting vulnerabilities in Twisted, a Python event-based framework for building various types of internet applications. -
Issued ELA-219-1 to address an out-of-bounds read vulnerability during string comparisons in
libbsd
, a library of functions commonly available on BSD systems but not on others such as GNU.
You can find out more about the Debian LTS project via the following video:
Debian Uploads
-
python-django (
2.2.11-1
&3.0.4-1
) — New upstream security releases. (#953102) -
6.0~rc1-3
— Install OpenSSL when running the testsuite as it is required for generating test certificates.6.0~rc2-1
— New upstream beta release.
-
xtrlock (
2.13
) — Add a warning that X11 does not support grabbing events under the Wayland display server protocol. (#953319) -
gunicorn (
20.0.4-4
) — Ensure that the Python 3.xgunicorn
binary package replaces the now-legacygunicorn3
that was removed in19.9.0-2
upon installation. I also backported this version to buster-backports. (#953883) -
hiredis (
0.14.1-1
) — New upstream release. -
libfiu (
1.00-7
) — Fix build failures under Python 3.8. (#954287) -
1.6.0-1
— New upstream release.1.6.1-1
— New upstream release.1.6.2-1
— New upstream security release. (#954808)
-
bfs (
1.6-2
,1.6-3
&1.6-4
) — Fix various build failures on kFreeBSD architectures. -
mtools (
4.0.24-1
) — New upstream release. -
xcb (
2.4-6
) — Remove a vague "pigeon holes" metaphor from package description.
For the Debian Privacy Maintainers team I requested that the pyptlib
package be removed from the archive (#953429) as well as uploading onionbalance (0.1.8-6
) to fix test failures under Pytest 3.x (#953535) and a new upstream release of nautilus-wipe.
Finally, I sponsored an upload of bilibop (0.6.1
) on behalf of Yann Amar.