Improve security and automation of official binary releases

[na--]

As pointed out by this forum post, currently our RPM packages are not signed 😞

I think that during the transition away from AppVeyor/CircleCI (#959), we should also make a substantial improvement (mostly in terms of security) and upgrade in how we build our packages:

• start properly signing them - both the Linux ones and the Windows ones (Sign release binaries and installers for Windows  #1034 - which we've also had other issues with the bintray hosting before...)
• maybe investigate if we can also make the bulds reproducible (as discussed yesterday, though that might have to wait for Go 1.14: cmd/go: build_trimpath test doesn't correctly test executable reproducibility golang/go#35435
• build and deploy the github releases from a CI instead of manually via build-release.sh

[na--]

Something else we should do for every commit, not just for tagged releases: actually build the rpm/deb/msi/etc. packages and test them (in the appropriate OS containers)... The v0.26.0 build of the linux packages broke: https://circleci.com/gh/loadimpact/k6/8360 🤦‍♂️ If we were building these on every commit, or at least once daily, we would have noticed and fixed the issue much sooner.

[imiric]

Since the go-bin-{deb,rpm} tools are giving us issues and they don't seem to be well maintained (last commit was over 2 years ago), we might want to consider switching to https://github.com/jordansissel/fpm .

I haven't used fpm personally, but have heard good things when used with Python projects. Presumably we should be able to use the directory input option for Go binaries, and would have more output options, including macOS .pkg files, so it's worth looking into.

[imiric]

There's another issue with Homebrew packaging on macOS. The current k6 formula actually builds the binary, which currently doesn't have any version information or any custom build flags we use for the GitHub releases. It's also a problem if we change our build system, since it currently uses dep and in v0.28.0 we switched to Go modules. This type of change hopefully won't happen again, but we shouldn't have to maintain two separate builds.

Ideally the binary shouldn't be rebuilt for Homebrew, and the formula should just fetch the *-mac.zip package from our GitHub release, verify the checksum, and bundle the binary for Homebrew. From a quick look, this seems possible but it would be messy to setup if have to use Bintray for this, so not sure, some investigation is needed for this.

[na--]

This is becoming a bit of a forced priority because of https://jfrog.com/blog/into-the-sunset-bintray-jcenter-gocenter-and-chartcenter/

[imiric]

The scope of this issue is a bit fuzzy, but most of the things mentioned have now been addressed:

• We replaced Bintray with our own package repositories at https://dl.k6.io.
• DEB/RPM packages are signed with our GPG key, and MSI packages with our certificate.
• build-release.sh is still used, but is now part of CI and run automatically for new releases.

What's still pending:

• Reproducible builds.
• Building packages more frequently, which is now tracked in Improve reliability of package publishing #1970.
• Homebrew issues...
• Signing of RPM repository metadata, as mentioned in the same forum post. We missed this recently, but should sign the metadata as well and enable repo_gpgcheck. UPDATE: After some investigation, we decided not to sign repo metadata and explicitly set repo_gpgcheck=0 to avoid potential failures. There are known issues with it enabled, and it's not even used for main Fedora repositories.

I suggest closing this issue and opening new ones for specific things. @na-- @mstoykov WDYT?

[na--]

👍 We should close this and open individual issues for anything else we want to do. The important part is that it's now "want", not "need" 🎉 😅

[imiric]

I created #1996 and #1997 for the leftover issues here, so closing this.