Refactoring multiple parts of the dockerfile

[ewjoachim]

• Switched to a 5-staged build:
  • Static, like before (but we now use it instead of Dockerfile.static)
  • Base which has apt deps and main env vars
  • Build with the build-time apt deps, which does the main pip install
  • Dev, which is the developper image (there's no DEVEL arg anymore)
  • App, which is to be the production image
• Remove a lot of heavy & unused things from the docker context through .dockerignore
• Add apt & pip cache, to speed up rebuilds and limit bandwith requirements (with no impact on layer size)
• use set -eux; \ style instead of set -u \ && style
• Address the todo that was in Dockerfile: add a lockfile for pip, setuptools & wheel
• docker-compose uses python image directly for notdatadog & files, should be smaller on disk/memory

This is a good first step toward #9988

@ewdurbin I believe there might be implications on the deployment pipeline, though it might be nothing more than adding --target=app to the docker build. And even without that, I believe it would be the default.

[abitrolly]

Should it be added into GitHub Actions pipeline? I am interested to see how the changes improve the build time.

[ewjoachim]

I am interested to see how the changes improve the build time.

I'm not sure this would speed up CI a lot, because I doubt GitHub CI keeps --mount=type=cache between calls, this would be too dangerous because if users polluted the cache, it would be hard to recover.

For the rest, the layers here should roughly take as long as before, it's refactored for readability and ease of use.

In the end the operations we have are:

• apt install run time deps
• pip install run time deps
• apt install build time deps
• pip install dev
• build statics
  and that's it. We had that before, we have this now, and it takes as long to run.

And even with a layer cache, it reduces bandwidth usage but from an human point of view, downloading 100ko with fiber or reading it from disk takes as long (0s).

[ewdurbin]

This is really incredible work. Thank you @ewjoachim!

Our deployment pipeline doesn't currently have support for setting a target when building, but since the app ends up being default, that's not a problem!

[ewdurbin]

It completely escapes me why we do not use the binary releases for hiredis... but based on the date it seems to be related to redis/hiredis-py#69 (nice) which was seemingly only related to that 0.2.0 release.

Perhaps we should go back to using the wheels provided?

[ewdurbin]

Will address in follow up PR

[ewjoachim]

👍