Here is my monthly update covering what I have been doing in the free software world in October 2017 (previous month):
- Wrote and released python-gfshare, a Python library that implements Shamir’s method for secret sharing, a cryptography technique to split a file into multiple parts. I blogged about it earlier in the month.
- Improved the new-style URL handling in the 2.x branch of the Django web framework to check for incorrectly migrated calls from the old system. [...]
- Added support for Python 3, Django 1.9 & Django 2.0 to django-auto-one-one-to-one, my library to cleanly manage related model instances.
- Proposed two small pull requests for Redisearch, a search engine module for the Redis key-value storage database:
- In Peter Bengtsson's new django-cache-memoize library, suggested an addition to the store_result documentation. [...]
- Made two pull requests for django-auditlog, a Django app that maintains an audit trail of changes in your application:
- Yet more Lintian hacking (previously):
- New features:
- Warn when packages unnecessararily set dpkg-architecture(1) variables. (#793554)
- Check for files that use content from the /etc/init.d/skeleton template. (#879152)
- Warn about certain files under debian/* that contain trailing whitespace characters. (#748405)
- Ignore embedded jQuery libraries for Doxygen. (#736360)
- Warn about Django libraries that do not depend on Django itself. (#877292)
- Check for Python modules with overly generic names such as tests or test. (#875964)
- Warn if packages set CFLAGS if the value of DEB_BUILD_OPTIONS contains noopt. (#718640)
- Ignore embedded jQuery libraries for Doxygen. (#736360)
- Warn about empty fields in debian/control. (#744388)
- Add bionic as known Ubuntu distribution. (#880115)
- Warn if native systemd .service files only wrap the existing SysV/LSB init scripts. (#870704)
- Emit new empty-section-field tag instead of uninitialized value warnings on an empty Section: field. (#878515)
- Warn about debian/watch files using insecure URIs, similar to vcs-field-uses-insecure-uri. (#849515)
- Include the offending URI in debian-watch-uses-insecure-uri output, not the line number.
- Allow empty md5sums files. (#781372)
- Move latest-debian-changelog-entry-without-new-date tag into a new check of type source. (#873612)
- Add cwl-runner to the list of known interpreters. (#851126)
- Also match packages named python2-* as relating to the Python 2.x migration.
- Bug fixes:
- Don't error out when AppStream metadata is invalid; emit appstream-metadata-invalid instead. (#879661)
- Actually check for a dependency on sensible-utils before emitting script-needs-depends-on-sensible-utils. (#877439)
- Avoid a false positives in debian-control-has-empty-field when the field is wrapped onto a new line. (#879977)
- Drop README.source from files to check against file-contains-trailing-whitespace as it can include literal quotes from upstreams that would be ideally left intact.
- Lower the severity of package-installs-java-bytecode from E: to W:. (#879862)
- Do not trigger package-installs-java-bytecode if the path contains WEB-INF, demo, doc etc. (#879860)
- Verify files triggering package-installs-java-bytecode files really are Java .class files. (#879861)
- Check the Recommends field as well when testing scripts for script-needs-depends-on-sensible-utils. (#879953)
- Ignore the "magic" http://sf.net/ redirector URI for the debian-watch-uses-insecure-uri tag. (#879206)
- Revert addition of "none were" -> "none was" multiword spelling correction as it is "acceptable beyond serious criticism". (#878457)
- Drop copyright-year-in-future; too error prone and time-consuming to maintain given the severity of the issues it can find. (#877766)
- Exempt debian/copyright from the license-problem-non-free-RFC tag to avoid false-positives "meta" references. (#877999)
- Ignore privacy breach violations within HTML/Javascript comments. (#877421)
- Avoid warning for init.d-script-not-marked-as-conffile when the init.d script does not exist as we will already be alerted via the init.d-script-not-included-in-package tag.
- Do not emit python-foo-but-no-python3-foo for packages ending with -common.
- Add missing example-script-uses-deprecated-nodejs-location tag. (#877142)
- Correct invalid link to upgrading-checklist in Standards-Version checks. (#878184)
- Documentation:
- Add a note to orig-tarball-missing-upstream-signature regarding support in pristine-tar and git-buildpackage.
- Add example on how to remove trailing whitespace with sed.
- Tests:
- Split out checks for debconf-config-not-executable into a separate test protected by a Test-Requires now that dpkg ≥ 1.19.0 will bail out on that condition.
- Correct Depends of python2.7 → python3 in a Python 3 test package.
- Add test for ignoring python-foo-doc packages for the Python 3 migration and correct short descriptions of test packages.
- New features:
- Finally, I was interviewed by HostingAdvice about my role as Debian Project Leader.
Reproducible builds
Whilst anyone can inspect the source code of free software for malicious flaws, most software is distributed pre-compiled to end users.
The motivation behind the Reproducible Builds effort is to allow verification that no flaws have been introduced — either maliciously or accidentally — during this compilation process by promising identical results are always generated from a given source, thus allowing multiple third-parties to come to a consensus on whether a build was compromised.
I have generously been awarded a grant from the Core Infrastructure Initiative to fund my work in this area.
This month I:
- Presented at All Things Open, 2017 in Raleigh, NC, United States.
- Attended the third Reproducible Builds summit in Berlin, Germany.
- Presented at the inaugural Freenode #live conference in Bristol, United Kingdom.
- In Debian, I:
- Kept isdebianreproducibleyet.com up to date.
- Submitted 10 patches to fix specific reproducibility issues in argagg, cadvisor, fmtlib, geographiclib, live-build, plr, polygen, python-amqp, rcs & sdlgfx.
- Made the following non-maintainer uploads (NMUs) to Debian to fix reproducibility issues:
- Updated the try.diffoscope.org SSL key.
- Categorised a large number of packages and issues in the Reproducible Builds "notes" repository.
- Worked on publishing our weekly reports. (#128, #129, #130 & #131)
I also made the following changes to our tooling:
diffoscope
diffoscope is our in-depth and content-aware diff utility that can locate and diagnose reproducibility issues.
- Improve names in output of "internal" binwalk members. (#877525).
- Don't crash on malformed md5sums files. (#877473).
- Omit misleading "any of" prefix when only complaining about a single module on import. [...]
- Adjust tests as ps2ascii now varies its output on timezone. [...]
strip-nondeterminism
strip-nondeterminism is our tool to remove specific non-deterministic results from a completed build.
- Clojure considers .class file to be stale if it shares the same timestamp of the .clj. We thus adjust the timestamps of the .clj to always be younger. (#877418).
- Print a message in --verbose mode if no canonical time was specified. [...]
buildinfo.debian.net
buildinfo.debian.net is my experiment into how to process, store and distribute .buildinfo files after the Debian archive software has processed them.
- Always show SHA-256 checksums, regardless of the browser viewport size. [...]
- Add an API endpoint to fetch specific .buildinfo files for a certain package/version/architecture. [...]
Debian
My activities as the current Debian Project Leader are covered in my "Bits from the DPL" email to the debian-devel-announce mailing list.
Patches contributed
Debian LTS
This month I have been paid to work 18 hours on Debian Long Term Support (LTS). In that time I did the following:
- "Frontdesk" duties, triaging CVEs, etc.
- Followed up on a large number of upstream "pings" that have been left dormant.
- Issued DLA 1121-1 to fix an out-of-bounds read vulnerability in curl where a malicious FTP server could abuse this to prevent clients from interacting with it.
- Issued DLA 1123-1 for the "Go" programming language where an attacker could generate a MIME request such that the server ran out of file descriptors.
- Issued DLA 1126-1 for the libxfont font selection and rasterisation library, correcting two vulnerabilities, both involving the library being tricked into reading invalid/random memory.
- Issued DLA 1134-1 for sdl-image1.2, an image loading library. A maliciously-crafted .xcf file could cause a stack-based buffer overflow resulting in potential code execution.
Uploads
- python-django:
- 2.0~beta1-1 — New upstream 2.x release.
- 1.11.6-1 — New upstream bugfix release.
- gunicorn (19.6.0-10+deb9u1) — Prepared a release for stable to avoid a runtime dependency on a compiler. (#877722)
- redis:
- 4:4.0.2-3:
- Drop the Debian-specific /etc/redis/redis-server.pre-up.d (etc.) hooks and remove them if unchanged.
- Include systemd redis-server@.service and redis-sentinel@.service template files to easily run multiple Redis instances. (#877702)
- Patch redis.conf and sentinel.conf with quilt instead of maintaining our own versions under debian/.
- 4:4.0.2-4:
- Add input validity checking to cluster config slot numbers to fix CVE-2017-15047. (#878076)
- Drop debian/bin/generate-parts now we aren't calling it.
- Correct Bash-ism in NEWS file.
- 4:4.0.2-5: Replace the existing patch for CVE-2017-15047 with an upstream-blessed version that covers another case.
- 4:4.0.2-3:
- redisearch (0.21.3-5) — Initial release.
- docbook2man (2.0.0-40) — Correct spelling mistakes in binaries and other misc packaging tidying.
- python-redis (2.10.6-1) — New upstream release.
- bfs (1.1.3-1) — New upstream release.
FTP Team
As a Debian FTP assistant I ACCEPTed 103 packages: amcheck, argagg, binutils, blockui, bro-pkg, chkservice, citus, django-axes, docker-containerd, doctest, dtkwidget, duktape, feed2exec, fontforge, fonttools, gcc-8, gcc-8-cross, generator-scripting-language, gitgraph.js, haskell-uri-encode, hoel, iniparser, its, jquery-areyousure, kodi, libcatmandu-mods-perl, libcatmandu-template-perl, libcatmandu-xml-perl, libcatmandu-xsd-perl, libcode-tidyall-plugin-sortlines-naturally-perl, libgdamm5.0, libinfinity, libmods-record-perl, libreoffice-dictionaries, libset-intervaltree-perl, libsodium, linux, linux-grsec, ltsp-manager, lxqt-themes, mailman3-core, measurement-kit, mini-buildd, musescore, node-babel, node-babel-eslint, node-babel-loader, node-babel-plugin-add-module-exports, node-babel-plugin-transform-define, node-gulp-newer, node-regenerate-unicode-properties, node-regexpu-core, node-regjsparser, node-unicode-data, node-unicode-loose-match, openjdk-9, orafce, pgaudit, pgsql-ogr-fdw, pk4, postgresql-mysql-fdw, powa-archivist, python-azure-devtools, python-colormap, python-darkslide, python-dotenv, python-karborclient, python-logfury, python-lupa, python-marshmallow, python-murano-pkg-check, python-octaviaclient, python-pathspec, python-pgpy, python-pydub, python-randomize, python-sabyenc, python-searchlightclient, python-stestr, python-subunit2sql, python-twitter, python-utils, python-wsgilog, r-cran-bindr, r-cran-desc, r-cran-hms, r-cran-readstata13, r-cran-rprojroot, r-cran-wikidatar, r-cran-wikipedir, r-cran-wikitaxa, repmgr, requests-file, resteasy3.0, sdl-kitchensink, stardicter, systemd-el, thunderbird, tomcat8.0, uwsgi-plugin-luajit, uwsgi-plugin-mongo, uwsgi-plugin-php & uwsgi-plugin-v8.
I additionally filed 3 RC bugs against packages that had incomplete debian/copyright files against: fonttools, generator-scripting-language & libsodium.