Here is my monthly update covering what I have been doing in the free software world in January 2018 (previous month):
- Updated travis.debian.net, my hosted service for projects that host their Debian packaging on GitHub to use the Travis CI continuous integration platform to test builds:
- Submitted a pull request for the Promise JS framework to make the build reproducible. [...]
- Submitted a number of pull requests to ffs, a set of filesystem API helpers for Python:
- Suggested improvements to Turtlecoin, a private and fast cryptocurrency:
- Updated the Opal web framework for building usable healthcare applications:
- Created a pull request for the RediSearch search engine module for Redis to fix a -Wformat error for the "long unsigned int type. [...]
- Filed a PR against openSUSE's hardware information tool to ensure the CDBISDN_DATE variable ignores timezone in order to make the build reproducible. [...]
- Submitted a pull request to python-stdnum, A Python library to provide functions to handle, parse and validate standard numbers to not rely on stable set ordering to ensure a reproducible build. [...]
- Even more hacking on the Lintian static analysis tool for Debian packages:
- New features:
- Check for files under debian/patches that are not mentioned in any series file. (#887817)
- Check for upstream tarballs that ship examples but none are installed in any binary package. (#539326)
- Ask maintainers to add a header to debian/copyright if their package is in contrib or non-free. (#773562)
- Check for override_dh_auto_test targets that do not check DEB_BUILD_OPTIONS. (#712394)
- Add support for passing .buildinfo files to Lintian. (#853274)
- Check for inconsistencies between Files and Checksums- sections in .changes files. (#658542)
- Check for packages that with dependencies on packages such as cdbs or debhelper. (#700953)
- Warn about packages that specify a Files-Excluded header without a valid Format header. (#745743)
- Check for .jar files that do not match the Debian Java policy. (#791552)
- Emit a pedantic warning for packages using dpatch. (#884500)
- When looking for the source of subdir/foo/bar.min.js, also check src/foo/bar.js etc. (#832027)
- Check for packages that mention planned features in their description. (#782990)
- Improve the description-synopsis-might-not-be-phrased-properly tag also detect multiple sentences. (#778427)
- Warn about Multi-Arch: same packages that ship arch-specific overrides. (#787469)
- Add a warning for comaintained packages not managed in a revision control system. (#884497)
- Warn when a "Files: *" DEP-5 paragraph exists but isn't the first paragraph. (#879235)
- Detect "backports" as an overly generic Python module name. (#888559)
- Allow debian/missing-sources/foo.js directories to represent the source for foo.js. (#836771)
- Add a check for packages that specify dh --with quilt whilst not using the 3.0 (quilt) source format. (#886566)
- Warn about warn packages that use dh_systemd_* overrides whilst using debhelper level 11. (#887899)
- Emit a warning about documentation packages that end with -docs. (#664520)
- Emit an error if packages ship files in /etc/skel. (#887120)
- Detect overly-compressed xz packages. (#829100)
- Warn about packages that ship Python modules but are missing dependencies on an interpreter. (#887083)
- Ensure Name Services Switch modules are placed in the admin section. (#886961)
- Warn about insecure DEP-5 Format: URIs. (#886930)
- Check for packages that should specify Rules-Require-Root. (#886479)
- Emit pedantic warnings for packages that refer to a non-Git VCSs hosted in the Debian infrastructure. (#885974)
- Change ancient-standards-version policy to "a release of Policy from the previous stable release cycle". (#886219)
- Warn for packages that ship (non-reproducible) Python Hypothesis examples. (#886101)
- Warn about orphaned packages not maintained in the Debian infrastucture. (#886057)
- Warn about packages that ship (eg.) test_foo.py files in the global Python module namespace. [...]
- Bug fixes:
- Don't emit new-package-should-not-package-python2-module if the maintainer can justify its inclusion. [...]
- Avoid false positives in spelling-error-in-description where the repetition is part of an acronym expansion. (#883719)
- Ignore Rust .rs files in extra-license-file. (#887715)
- Support the latest binutils when parsing ELF files as this was causing a testsuite failure. (#888456)
- When checking for a Python 3 variant of a Python package, consider any package that declares a binary dependency on ${python3:Depends}. (#886303)
- Fix issue where bad-section-in-changes-file, file-size-mismatch-in-changes-file and checksum-mismatch-in-changes-file were not being checked. [...]
- Don't emit license-problem-php-license when the source comes from pecl.php.net. (#810780)
- Avoid false positives and remove an existing (incorrect) test for apparent brace expansions in config files that do not include a comma. (#888304)
- Avoid a false positive for spelling-error-in-binary that was causing a FTBFS on armhf. (#888074)
- Avoid false positives for missing-notice-file-for-apache-license by looking for files with a .txt extension. (#886343)
- Avoid false-positives when checking Windows PE files. (#886555)
- Ensure xfonts-foo are recognised as part of the x11 section to match the definition on packages.debian.org. (#878609)
- Fix Use of uninitialized value in string ne warnings. (#887428)
- Only run files-multiarch-foreign-files test on amd64. (#886163)
- Don't warn about unknown template type entropy when a package depends on cdebconf. (#677870)
- Ignore TeX \section titles when checking for GFDL licenses. (#863384)
- Only test for packages shipping gschemas.compiled files in specified dirs. (#884142)
- Fix a programming error in the src-orig-index collection script. (#886586)
- Ensure bugs-field-does-not-refer-to-debian-infrastructure can be overridden by not emitting them for -dbgsym packages. (#886426)
- Don't warn about extra license files installed via Sphinx. (#885968)
- Don't warn about django-package-does-not-depend-on-django for -doc packages, etc. [...]
- Skip Objective-C libraries for no-symbols-control-file. (#749202)
- Install files-multiarch-foreign-files tests to a multi-arch directory on all architectures. (#886163)
- Skip comment lines when matching autotools-pkg-config-macro-not-cross-compilation-safe. (#886297)
- Identify both python-foo-dbg and python3-foo-dbg as known debug packages. (#886271)
- Output/reporting improvements:
- Make previously-hidden package anchor links visible. [...]
- Include the offending context and line when emitting brace-expansion-in-debhelper-config-file. [...]
- Include the date the Standards-Version was released in output of ancient-standards-version and out-of-date-standards-version. [...]
- Include the value in the output of debian-rules-should-not-use-DH_EXTRA_ADDONS. [...]
- Append the URI in the output of vcs-deprecated-in-debian-infrastructure. [...]
- Include offending package name in new-package-should-not-package-python2-module output. [...]
- Include Bugs field value in the output of bugs-field-does-not-refer-to-debian-infrastructure. [...]
- Add more context to xz-compression-level-too-high output. [...]
- Severities:
- Upgrade the severity of missing-debian-source-format from wishlist to normal. (#702671)
- Downgrade extra-license-file (#740118), dependency-on-python-version-marked-for-end-of-life (#886259), wrong-section-according-to-package-name (#883772) & newer-standards-version (#886210).
- Documentation:
- Clarify paragraph ordering matters in the description for unused-file-paragraph-in-dep5-copyright. (#762261)
- Remark that new-package-should-not-package-python2-module's appearance on lintian.debian.org can be ignored. (#887124)
- Also mention Recommends and Suggests in the opening paragraph of python-script-but-no-python-dep. (#687141)
- Add missing initials in debian/copyright. (#831729)
- Misc:
- Rename debian-watch-does-not-check-gpg-signature to avoid confusion around "may check". (#735040)
- Bump arch-dep-package-has-big-usr-share thresholds. (#648755)
- New features:
Reproducible builds
Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.
The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.
This month I:
- Presented at linux.conf.au on diffoscope, our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
- Filed upstream pull requests for python-stdnum, PromiseJS & hwinfo.
- In Debian:
- Kept isdebianreproducibleyet.com up to date. [...]
- Added a Lintian check for packages that ship (non-reproducible) Python Hypothesis examples. (#886101)
- Added another Lintian check to warn about packages that override dh_fixperms without calling dh_fixperms as this makes the build vary depending on the current umask(2). (#885910)
- I also submitted 21 patches to fix specific reproducibility issues in clanlib, dtkwm, fox1.6, hwinfo, klystrack, kopano-webapp, libmsv, librsvg, mcl, mdds, mstflint, node-deflate-js, node-jquery, node-promise, normaliz, python-hpack, python-pysnmp4, python-stdnum, texlive-extra, todoman & zorp.
- Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
- Worked on publishing our weekly reports. (#140, #142 #143 & #144)
I also made the following changes to our tooling:
diffoscope
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
- New features:
- Compare JSON files using the jsondiff module. (#888112)
- Report differences in extended file attributes when comparing files. (#888401)
- Show extended filesystem metadata when directly comparing two files not just when we specify two directories. (#888402)
- Do some fuzzy parsing to detect JSON files not named .json. [...]
- Bug fixes:
- Misc:
I also fixed an issue in the "trydiffoscope" command-line client that was preventing installation on non-Debian systems (#888882).
disorderfs
disorderfs is our FUSE-based filesystem that deliberately introduces non-determinism into directory system calls in order to flush out reproducibility issues.
- Correct "explicitly" typo in disorderfs.1.txt. [...]
- Bump Standards-Version to 4.1.3. [...]
- Drop trailing whitespace in debian/control. [...]
Debian
My activities as the current Debian Project Leader are covered in my "Bits from the DPL" email to the debian-devel-announce mailing list.
In addition to this, I:
- Published whydoesaptnotusehttps.com, an overview of why APT does not rely solely on SSL for validation of downloaded packages as I noticed it was being asked a lot on support forums.
- Reported a number of issues for the mentors.debian.net review service.
Patches contributed
- dput: Suggest --force if package has already been uploaded. (#886829)
- linux: Add link to the Firmware page on the wiki to failed to load log messages. (#888405)
- markdown: Make markdown exit with a non-zero exit code if cannot open input file. (#886032)
- spectre-meltdown-checker: Return a sensible exit code. (#887077)
Debian LTS
This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:
- Initial draft of a script to automatically detect when CVEs should be assigned to multiple source packages in the case of legacy renames, duplicates or embedded code copies.
- Issued DLA 1228-1 for the poppler PDF library to fix an overflow vulnerability.
- Issued DLA 1229-1 for imagemagick correcting two potential denial-of-service attacks.
- Issued DLA 1233-1 for gifsicle — a command-line tool for manipulating GIF images — to fix a use-after-free vulnerability.
- Issued DLA 1234-1 to fix multiple integer overflows in the GTK gdk-pixbuf graphics library.
- Issued DLA 1247-1 for rsync, fixing a command-injection vulnerability.
- Issued DLA 1248-1 for libgd2 to prevent a potential infinite loop caused by signedness confusion.
- Issued DLA 1249-1 for smarty3 fixing an arbitrary code execution vulnerability.
- "Frontdesk" duties, triaging CVEs, etc.
Uploads
- adminer (4.5.0-1) — New upstream release.
- bfs (1.2-1) — New upstream release.
- dbus-cpp (5.0.0+18.04.20171031-1) — Initial upload to Debian.
- installation-birthday (7) — Add e2fsprogfs to Depends so it can drop Essential: yes. (#887275
- process-cpp:
- 3.0.1-1 — Initial upload to Debian.
- 3.0.1-2 — Fix FTBFS due to symbol versioning.
- python-django (1:1.11.9-1 & 2:2.0.1-1) — New upstream releases.
- python-gflags (1.5.1-4) — Always use SOURCE_DATE_EPOCH from the environment.
- redis:
- redisearch (1.0.3-1, 1.0.4-1 & 1.0.5-1) — New upstream releases.
- trydiffoscope (67.0.0) — New upstream release.
I also sponsored the following uploads:
- check-manifest (0.36-1)
- dict-devil (1.0-13)
- nose2 (0.7.3-2)
- pytest-httpbin (0.3.0-1)
- python-blessed (1.14.2-3)
- twodict (1.2-1)
Debian bugs filed
- gdebi: Invalid gnome-mime-application-x-deb icon in AppStream metadata. (#887056)
- git-buildpackage: Please make gbp clone not quieten the output by default. (#886992)
- git-buildpackage: Please word-wrap generated changelog lines. (#887055)
- isort: Don't install test_isort.py to global Python namespace. (#887816)
- restrictedpython: Please add Homepage. (#888759)
- xcal: Missing patches due to 00List != 00list. (#888542)
I also filed 4 bugs against packages missing patches due to incomplete quilt conversions against cernlib geant321, mclibs & paw.
RC bugs
- gnome-shell-extension-tilix-shortcut: Invalid date in debian/changelog. (#886950)
- python-qrencode: Missing PIL dependencies due to use of Python 2 substvars in Python 3 package. (#887811)
I also filed 7 FTBFS bugs against lintian, netsniff-ng, node-coveralls, node-macaddress, node-timed-out, python-pyocr & sleepyhead.
FTP Team
As a Debian FTP assistant I ACCEPTed 173 packages: appmenu-gtk-module, atlas-cpp, canid, check-manifest, cider, citation-style-language-locales, citation-style-language-styles, cloudkitty, coreapi, coreschema, cypari2, dablin, dconf, debian-dad, deepin-icon-theme, dh-dlang, django-js-reverse, flask-security, fpylll, gcc-8, gcc-8-cross, gdbm, gitlint, gnome-tweaks, gnupg-pkcs11-scd, gnustep-back, golang-github-juju-ansiterm, golang-github-juju-httprequest, golang-github-juju-schema, golang-github-juju-testing, golang-github-juju-webbrowser, golang-github-posener-complete, golang-gopkg-juju-environschema.v1, golang-gopkg-macaroon-bakery.v2, golang-gopkg-macaroon.v2, harmony, hellfire, hoel, iem-plugin-suite, ignore-me, itypes, json-tricks, jstimezonedetect.js, libcdio, libfuture-asyncawait-perl, libgig, libjs-cssrelpreload, liblxi, libmail-box-imap4-perl, libmail-box-pop3-perl, libmail-message-perl, libmatekbd, libmoosex-traitfor-meta-class-betteranonclassnames-perl, libmoosex-util-perl, libpath-iter-perl, libplacebo, librecaptcha, libsyntax-keyword-try-perl, libt3highlight, libt3key, libt3widget, libtree-r-perl, liburcu, linux, mali-midgard-driver, mate-panel, memleax, movit, mpfr4, mstch, multitime, mwclient, network-manager-fortisslvpn, node-babel-preset-airbnb, node-babel-preset-env, node-boxen, node-browserslist, node-caniuse-lite, node-cli-boxes, node-clone-deep, node-d3-axis, node-d3-brush, node-d3-dsv, node-d3-force, node-d3-hierarchy, node-d3-request, node-d3-scale, node-d3-transition, node-d3-zoom, node-fbjs, node-fetch, node-grunt-webpack, node-gulp-flatten, node-gulp-rename, node-handlebars, node-ip, node-is-npm, node-isomorphic-fetch, node-js-beautify, node-js-cookie, node-jschardet, node-json-buffer, node-json3, node-latest-version, node-npm-bundled, node-plugin-error, node-postcss, node-postcss-value-parser, node-preact, node-prop-types, node-qw, node-sellside-emitter, node-stream-to-observable, node-strict-uri-encode, node-vue-template-compiler, ntl, olivetti-mode, org-mode-doc, otb, othman, papirus-icon-theme, pgq-node, php7.2, piu-piu, prometheus-sql-exporter, py-radix, pyparted, pytest-salt, pytest-tempdir, python-backports.tempfile, python-backports.weakref, python-certbot, python-certbot-apache, python-certbot-nginx, python-cloudkittyclient, python-josepy, python-jsondiff, python-magic, python-nose-random, python-pygerrit2, python-static3, r-cran-broom, r-cran-cli, r-cran-dbplyr, r-cran-devtools, r-cran-dt, r-cran-ggvis, r-cran-git2r, r-cran-pillar, r-cran-plotly, r-cran-psych, r-cran-rhandsontable, r-cran-rlist, r-cran-shinydashboard, r-cran-utf8, r-cran-whisker, r-cran-wordcloud, recoll, restrictedpython, rkt, rtklib, ruby-handlebars-assets, sasmodels, spectre-meltdown-checker, sphinx-gallery, stepic, tilde, togl, ums2net, vala-panel, vprerex, wafw00f & wireguard.
I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against: fpylll, gnome-tweaks, org-mode-doc & py-radix.