Ensure that make build-wheels builds wheels reproducibly where possible

[eloquence]

Currently, the make build-wheels target and the underlying scripts/build-sync-wheels script does not set SOURCE_DATE_EPOCH or specify a build directory. As a result, it does not create reproducible wheels, which is an important requirement for further build automation.

We should ensure that this target builds wheels reproducibly whenever possible without further modification.

[eloquence]

@redshiftzero currently maintains https://reproduciblewheels.com/ as a useful index of Python projects which can be built as wheels reproducibly (including the wheels from https://pypi.securedrop.org/simple/), with updates to https://github.com/redshiftzero/reproduciblewheels/blob/main/site_data.json being pushed daily via bot. For further comparability, it may be desirable to specify the same build parameters as used for that project. Quote:

To compare your wheel hashes with the hashes of these packages, SOURCE_DATE_EPOCH should be set to 1596163658 and you should pass the --build /tmp/buildwheel option to pip at build time.

[eloquence]

We discussed today that it might be nice to standardize on the timestamp of Aaron's first SD commit. Per git show -s 62bbe590afd77a6af2dcaed46c93da6e0cf40951 --date=unix in securedrop repo that would be 1309379017.

[emkll]

As part of this task, we should add a CI reproducibility check, to ensure changes to the wheel building command are correct.

[conorsch]

Took a look at this today. Used the excellent diffoscope & reprotest tools to examine what was different about the wheels, specifically for the securedrop-client package build. In short, we had the same blockers as already surfaced in https://github.com/redshiftzero/reproduciblewheels:

1. SOURCE_DATE_EPOCH must be set
2. a constant build dir path must be used

Also added a umask of 022 (sane default) to satisfy some other reproducibility checks automatically run by reprotest. In fact, out of all the checks that reprotest provides, our wheels fail only on the time check, because messing with system time breaks HTTPS, so the fetches fail.

When using reprotest, however, the platform for two dependencies of securedrop-client gets improperly set as i686, rather than x86_64: sqlalchemy & markupsafe. We can force a platform via --build-option --plat-name=<foo> but we don't want to do that for the non-platform specific packages, as well. Will have a think on what's our best bet there, but for now it appears to be a side-effect of using reprotest—platform is properly inferred when I run the build-sync-wheels script locally.

So, next steps are:

1. testing with more packages than just securedrop-client, to confirm the results are consistent across the board;
2. figuring out why reprotest sets i686 for platform (optional)
3. updating CI/tests to verify the wheels reproducibility

[eloquence]

It looks to me like this issue is fully resolved, as originally scoped, by #211 (which does add a reproducibility check to CI as well, see https://app.circleci.com/pipelines/github/freedomofpress/securedrop-debian-packaging/834/workflows/cee208e3-34c8-4f65-b8d5-be75bf34c200/jobs/6446), am I missing something or can we close? Any additional work should ideally be scoped into follow-up issues, IMO.

[conorsch]

Agreed! We still have #213 outstanding, which will start using the reproducibly-built wheels, but that's nicely tracked in #212 already.