Automatic CI/CD multiarch docker builds

[MarkSchmitt]

As discussed in #151 I've been porting my old pull request from the-paperless-project over to this one.

It extends the travis-ci pipeline with three docker builds, one for each x86_64, aarch64 (arm64) and arm32v7 and then creates a multi-arch docker manifest, so it can be used seamlessly on all three platforms.

It makes use of travis-ci's new arm64-graviton2 build machine, so travis-ci.com (not .org) needs to be used. I was able to migrate my account within minutes, so I hope this is not a problem.

The following environment variables and secrets need to be defined in travis:
DOCKER_REPO (f.e. jonaswinkler/paperless-ng)
DOCKER_USERNAME (f.e. jonaswinkler)
DOCKER_PASSWORD (should not be your regular password, but rather a deployment key generated on dockerhub)

Tagging is scripted like so:

• builds from the master branch end up as "latest" images
• builds from a git tag end up with that version as tag, f.e. tag "v1.0-rc1" will end up in jonaswinkler/paperless-ng:v1.0-rc1.
• all builds are always tagged with the git commit hash, so it's easy to test a non-released version

For the qemu build, which is currently only the arm32v7 one, jq is used to parse the docker manifest to get the latest version. This is necessary for qemu to pull the correct base-image. That can be improved a bit .. if the base image changes, that always needs to be adapted in the build script. Not sure if it's worth the efford, so I've left it with this rather simple version.

You can find travis-ci runs on my branch here: https://travis-ci.com/github/MarkSchmitt/paperless

And results here https://hub.docker.com/r/moztr/paperless-travis/tags

As @obbardc pointed out in #151 (comment) that github actions can do the same, but at this point, as there're no aarch64 wheels precompiled, I don't think qemu can be quick enough to handle it (in the tests with travis it was not, with a timeout of 60 minutes for the build). The native arm64 graviton2 machines travis uses from AWS seem to be very fast, so they can handle the amount of compiling necessary to build the wheels from scratch. However, later, when aarch64 wheels are available, I think using github actions might be a good alternative. So I suggest doing it the travis-way first, and then migrate later to a cleaner solution. To get something off the ground, and improve on it later.

[jonaswinkler]

Thank you so much. This will make the entire build process so much easier.

• For some reason the images are quite a bit larger, the current amd64 image used to be around 230MB, thats +130 now, not the end of the world, but somewhat concerning. Maybe I messed up the multi stage build, will have to check.

[jonaswinkler]

I suppose this is required to install some of the python dependencies?

[MarkSchmitt]

Yes, it looks like the arm32v7 base image does not provide this library for some weird reason. I think some dependencies are just a bit different. Though I don't fully understand why O_o

[jonaswinkler]

Let's also say I only want to build images for the branches master, dev and all tags that match ng-*, how would I do that? I don't necessarily need images for every commit, building and tagging dev is usually enough for testing before releases.

[jonaswinkler]

Nevermind regarding the image size, I've forgot to remove pipenv and its cache after dependency installation.

[MarkSchmitt]

Let's also say I only want to build images for the branches master, dev and all tags that match ng-*, how would I do that? I don't necessarily need images for every commit, building and tagging dev is usually enough for testing before releases.

Good idea, I think that can be done with conditions on the build step, I‘ll try to prepare something