Here is my monthly update covering what I have been doing in the free software world in December 2017 (previous month):
- Released a new version of python-gfshare, my Python library that implements Shamir’s method for secret sharing fixing parts of the documentation as well as fixing two warnings via contributions by Kevin Ji [...] [...].
- Opened a PR against vim-pizza (a plugin to order pizza from within the Vim text editor) to use xdg-open or sensible-browser under Debian and derivatives. [...]
- Created two pull requests for the RediSearch search engine module for Redis, first to un-ignore the /debian dir in .gitignore to aid packaging [...] and second to inherit CFLAGS/LDFLAGS from the outside environment to enable hardening support [...].
- Even more hacking on the Lintian static analysis tool for Debian packages:
- New features:
- Support Standards-Version 4.1.3.
- Warn when files specified in Files-Excluded exist in the source tree. (#871454)
- Check Microsoft Windows Portable Executable (PE) files missing hardening features. (#837548)
- Warn about Python 2.x packages using ${python3:Depends} and Python 3.x packages using ${python:Depends}. (#884676)
- Check changelog entries with incorrectly formatted dates. (#793406)
- Check override_dh_fixperms targets missing calls to dh_fixperms. (#885910)
- Ensure PAM modules are in the admin, preventing a false positive for libpam-krb5. (#885899)
- Check Python packages installing modules called site, docs, examples etc. into the global namespace. (#769365)
- Check packages that invoke AC_PATH_PROG without considering cross-compilation. (#884798)
- Emit a warning for packages that mismatch version control systems in Vcs-* headers. (#884503)
- Warn when packages specify a Bugs field in debian/control that does not refer to official Debian infrastructure. (#741071)
- Warn for packages shipping pkg-config files under /usr/lib/pkgconfig. (#885096)
- Warn about packages that ship non-reproducible Python .doctree files. (#885327)
- Bump the recommended Debhelper compat level to 11. (#884699)
- Warn about Python 3 packages that depend on Python 2 packages (and vice versa). (#782277)
- Check for override_dh_clean targets missing calls to dh_clean. (#884817)
- Check Apache 2.0-licensed packages that do not distribute their accompanying NOTICE files. (#885042)
- Detect embedded jQuery libraries with version number in their filenames. (#833613)
- Also emit embedded-javascript-library for Twitter Bootstrap and Mustache.
- Check development packages that ship ELF binaries in $PATH. (#794295)
- Warn about library packages with excessive priority. (#834290)
- Warn about Multi-Arch: foreign packages that ship CMake, pkg-config or static libraries in public, architecture-dependent search paths. (#882684)
- Test for packages shipping gschemas.compiled files. (#884142)
- Warn if a package ships compiled font files. (#884165)
- Detect invalid debian/po/POTFILES.in. (#883653)
- Warn for packages that modify the epoch yet there's no comment about the change in the changelog.
- Bug fixes:
- Refactor django-package-does-not-depend-on-django check to correctly check Django packages called python2-django-foo.
- Match python2.7:any (etc.) when checking dependency-on-python-version-marked-for-end-of-life, not just python2.7. (#883053)
- Correct parsing of jobs=42 in .lintianrc. If specified, it would be coerced to a boolean resulting in a value of 1.
- Avoid false positives in apache2-deprecated-auth-config where the offending lines are wrapped in IfModule or IfVersion. (#788991, #710656)
- Prevent false positives when checking for Python 2 documentation, development common and tools packages that depend on Python 3 (etc.) packages. (#885693)
- Prevent false positives in missing-python-build-dependency. (#750537)
- Don't emit init.d-script-needs-depends-on-lsb-base if the package ships a Systemd service file. (#864999)
- Ignore commented-out lines in debian/watch to avoid false-positives referencing old values. (#806237)
- Prevent a false positive in possibly-insecure-handling-of-tmp-files-in-maintainer-script by detecting XXX-like mktemp(1) templates. (#601323)
- Use the list of files in the orig tarball to prevent false positives when checking for the source-includes-file-in-files-excluded. (#884848)
- Fix various issues in the src-orig-index collection script.
- dh_scour is provided by python3-scour, not python-scour. (#885106)
- Avoid a embedded-php-library false positive for streams.php. (#637473)
- Add a regression test for a potential false positive for vcs-field-has-unexpected-spaces. (#884870)
- Don't hardcode architecture triplet to fix FTBFS on non-AMD64 architectures. (#884683)
- Prevent a false-positive in missing-dep-for-interpreter by matching ABI-versioned virtual packages for Erlang. (#810204)
- Don't match, for example, FB.login() when used as a documentation example. (#884296)
- Correct operator precedence in epoch-change-without-comment to prevent a false positive when an epoch is present but is unchanged between versions.
- Correct false positives in source-includes-file-in-files-excluded where the maintainer has removed upstream's debian/ directory and then we would trigger it on the maintainer's replacement files.
- Reporting improvements:
- Include the offending filename and line number in the output of apache2-deprecated-auth-config & apache2-unparsable-dependency.
- Include the offending/unknown shebang in the output of various interpreter-related tags. (#673734)
- Avoid misleading tag descriptions when emitting timewarp-standards-version if the date parts are identical. (#884785)
- Add links from each maintainer page to the corresponding package on the full report. (#884572)
- Documentation:
- Suggest using /usr/share/dpkg/architecture.mk for debian-rules-sets-dpkg-architecture-variable.
- Standardise on capital-L Lintian in package descriptions.
- Also note that unused-override can be triggered if Lintian adds/modifies supplementary tag metadata.
- Add debian/changelog to the file-contains-trailing-whitespace example.
- Update description of python-script-but-no-python-dep to refer to ${python3:Depends}. (#660718)
- Clarify that new-package-should-not-package-python2-module triggers when there is a single changelog entry & provide general guidance where upstreams have not ported to Python 3 yet.
- Correct reference to dh_elpa(1) manpage. (#883356)
- Raise the severity of the dependency-on-python-version-marked-for-end-of-life and python-foo-but-no-python3-foo Python 2.x deprecation tags to regular warnings. (#883581)
- Miscellaneous:
- Add a vendor profile for Purism's PureOS. (#884408)
- Allow the tag display limit to be configured via --tag-display-limit. (#813525)
- Tag build-dependencies with <!nocheck> in debian/control.
- Make -v imply --no-tag-display-limit. (#812756)
- Remove russian → Russian corrections as they are covered by data/spelling/corrections-case. (#883041)
- New features:
- Suggested an improvement to the "lack of entropy" error message in the TLSH (Trend Micro Locality Sensitive Hash) fuzzy matching algorithm. [...]
- I also blogged about simple media cachebusting when using GitHub Pages.
Reproducible builds
Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.
The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.
This month I:
- Created pull requests upstream for cypari2, parso, sonic-pi, pydoctor & pylint.
- In Debian:
- Kept isdebianreproducibleyet.com up to date. [...]
- Submitted 20 patches to fix specific reproducibility issues in at-spi2-core, bibledit, cairomm, cypari2, designate, flask-peewee, golang-github-tjfoc-gmsm, multipath-tools, nanoc, p4vasp, parso, properties-cpp, psychtoolbox-3, pydoctor, pylint, pysph, python-h2, sasview, sonic-pi & simavr.
- Made a huge number of formatting improvements to our website, including typo fixes, capitalisation & addding section headings. [...]
- Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
- Worked on publishing our weekly reports. (#136, #137, #138 & #139)
I also made the following changes to our tooling:
diffoscope
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
- Support Android ROM boot.img introspection. (#884557)
- Handle case where a file to be "fuzzy" matched does not contain enough entropy despite being over 512 bytes. (#882981)
- Ensure the cleanup of symlink placeholders is idempotent. [...]
trydiffoscope
trydiffoscope is a web-based version of the diffoscope in-depth and content-aware diff utility. Continued thanks to Bytemark for sponsoring the hardware.
- Parse dpkg-parsechangeloga in setup.py instead of hardcoding version. [...]
- Flake8 the main file. [...]
buildinfo.debian.net
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
- Don't HTTP 500 if no request body. [...]
- Catch TypeError: decode() argument 1 must be string, not None tracebacks. [...]
Debian
My activities as the current Debian Project Leader will be covered in my Bits from the DPL email to the debian-devel-announce mailing list.
Patches contributed
- bitseq: Add missing Build-Depends on python-numpy for documentation generation. (#884677)
- dh-golang: Avoid "uninitialized value" warnings. (#885696)
- marsshooter: Avoid source-includes-file-in-files-excluded Lintian override. (#885732)
- gtranslator: Do not ship .pyo and .pyc files. (#884714)
- media-player-info: Bugs field does not refer to Debian infrastructure. (#885703)
- pydoctor: Add a Homepage field to debian/control. (#884255)
Debian LTS
This month I have been paid to work 14 hours on Debian Long Term Support (LTS). In that time I did the following:
- "Frontdesk" duties, triaging CVEs, etc.
- Updating old notes in data/dla-needed.txt.
- Issued DLA 1204-1 for the evince PDF viewer to fix an arbitrary command injection vulnerability where a specially-crafted embedded DVI filename could be exploited to run commands as the current user when "printing" to PDF.
- Issued DLA 1209-1 to fix a vulnerability in sensible-browser (a utility to start the most suitable web browser based on one's environment or configuration) where remote attackers could conduct argument-injection attacks via specially-crafted URLs.
- Issued DLA 1210-1 for kildclient, a "MUD" multiplayer real-time virtual world game to remedy a command-injection vulnerability.
Uploads
- python-django (2:2.0-1) — Release the new upstream stable release to the experimental suite.
- redis:
- 5:4.0.5-1 — New upstream release & use "metapackage" over "meta-package" in debian/control.
- 5:4.0.6-1 — New upstream bugfix release.
- 5:4.0.6-2 — Replace redis-sentinel's main dependency with redis-tools from redis-server moving the creating/deletion of the redis user, associated data & log directories to redis-tools (#884321), and add stub manpages for redis-sentinel, redis-check-aof & redis-check-rdb.
- 5:4.0.6-1~bpo9+1 — Upload to the stretch-backports repository.
- redisearch:
- 1.0.1-1 — New upstream release.
- 1.0.2-1 — New upstream release, ensure .so file is hardered (upstream patch), update upstream's .gitignore so our changes under debian/ are visible without -f (upstream patch and override no-upstream-changelog in all binary packages.
- installation-birthday (6) — Bump Standards-Version to 4.1.2 and replace Priority: extra with Priority: optional.
Finally, I also made the following miscellaneous uploads:
Debian bugs filed
- binutils: Please fix Vcs-Browser / Vcs-Bzr. (#884378)
- jenkins-job-builder-doc: Documentation is not generated correctly. (#884002)
- r-cran-scatterd3: Please clarify source for embedded D3 Lasso. (#883791)
- I also filed 40 bugs against packages that ship files that should have been removed according to their Files-Excluded header against: alt-ergo, android-platform-libcore, asm, cantata, drupal7, elixir-lang, ezquake, freecol, genometools, golang-github-hashicorp-go-sockaddr, haskell-shake, ice-builder-gradle, icu4j-4.2, ignition-math2, ignition-transport, isdnactivecards, jackd2, kashmir, libgd2, libgdamm5.0, librime, mediaelement, mgltools-vision, opencv, plink, prometheus, pyscanfcs, python-azure, python-openstackdocstheme, python-pyramid, python-social-auth, python-tz, rapidjson, r-cran-scatterd3, scamp, superlu-dist, superlu, testu01, ufo-filters & weightwatcher.
FTP Team
As a Debian FTP assistant I ACCEPTed 106 packages: aodh, autosuspend, binutils, btrfs-compsize, budgie-extras, caja-seahorse, condor, cross-toolchain-base-ports, dde-calendar, deepin-calculator, deepin-shortcut-viewer, dewalls, dh-dlang, django-mailman3, flask-gravatar, flask-mail, flask-migrate, flask-paranoid, flask-peewee, gcc-5-cross-ports, getmail, gitea, gitlab, golang-github-go-kit-kit, golang-github-knqyf263-go-deb-version, golang-github-knqyf263-go-rpm-version, golang-github-mwitkow-go-conntrack, golang-github-parnurzeal-gorequest, golang-github-prometheus-tsdb, haskell-unicode-transforms, haskell-unliftio-core, htslib, hyperkitty, libcbor, libcdio, libcidr, libcloudproviders, libepubgen, libgaminggear, libgitlab-api-v4-perl, libgoocanvas2-perl, libical, libical3, libixion, libjaxp1.3-java, liblog-any-adapter-tap-perl, liborcus, libosmo-netif, libt3config, libtirpc, linux-show-player, mailman-hyperkitty, mailman-suite, mailmanclient, muchsync, node-browser-stdout, node-crc32, node-deflate-js, node-get-func-name, node-ip-regex, node-json-parse-better-errors, node-katex, node-locate-path, node-uglifyjs-webpack-plugin, nq, nvidia-cuda-toolkit, openstack-meta-packages, osmo-ggsn, osmo-hlr, osmo-libasn1c, osmo-mgw, osmo-pcu, patman, peewee, postorius, pyasn1, pymediainfo, pyprind, pysmi, python-colour, python-defaults, python-django-channels, python-django-x509, python-ldap, python-quamash, python-ratelimiter, python-rebulk, python-trezor, python3-defaults, python3-stdlib-extensions, python3.6, python3.7, qscintilla2, range-v3, rawkit, remmina, reprotest, ruby-gettext-i18n-rails-js, ruby-webpack-rails, sacjava, sphinxcontrib-pecanwsme, unicode-cldr-core, wolfssl, writerperfect, xrdp & yoshimi.
I additionally filed 4 RC bugs against packages that had incomplete debian/copyright files against: libtirpc, python-ldap, python-trezor & sphinxcontrib-pecanwsme.