[manylinux2014] Rework scripts to be more docker cache friendly #879
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
25 tasks
mayeut
force-pushed
the
manylinux2014-multistage-build
branch
7 times, most recently
from
4d5f75b
to
fe9ccda
Compare
mayeut
force-pushed
the
manylinux2014-multistage-build
branch
from
fe9ccda
to
33f1acf
Compare
mayeut
force-pushed
the
manylinux2014-multistage-build
branch
from
33f1acf
to
6c321b6
Compare
Merged
mayeut
force-pushed
the
manylinux2014-multistage-build
branch
from
6c321b6
to
9c2a018
Compare
This won't change travis-ci build time (at least for now) but allows maintainers/contributors to benefit from docker caching for local developments (faster builds at the expense of higher disk usage) Round 1 Use docker multi-stage build to build the manylinux image in 3 steps: 1. runtime_base: this image has all the necessary bits that are common to the build_base image and the manylinux image. This includes system packages required for runtime, the gcc toolchain & some basic tools. 2. build_base: this image builds all the remaining tools required for the manylinux image. Any tool requiring specific system development package not required in the manylinux image shall be built from this image. For tools that used to be installed in /usr/local, they are staged in /manylinux-rootfs for an easy copy in the third stage. 3. manylinux: This image uses the runtime_base image as a base image. Tools built in the 2nd step are copied directly in the final filesystem. A finalization script is then run to install the remaining python tools/dependencies and run checks.
mayeut
force-pushed
the
manylinux2014-multistage-build
branch
from
9c2a018
to
c38b3bc
Compare
|
Is this ready for final review? |
|
This one is ready for review. I merged the previous PR without any reviews in order to get things going forward even though I don't like to do this without reviews... |
This was referenced Jan 30, 2021
This was referenced Jan 30, 2021
|
#965 is next? |
|
@mattip, yes, I'll merge everything up to building on GHA which will take some time for aarch64/s390x/ppc64le |
|
thanks! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This won't change travis-ci build time (at least for now) but allows maintainers/contributors to benefit from docker caching for local developments (faster builds at the expense of higher disk usage)
Round 1
Use docker multi-stage build to build the manylinux image in 3 steps:
runtime_base: this image has all the necessary bits that are common to the build_base image and the manylinux image. This includes system packages required for runtime, the gcc toolchain & some basic tools.
build_base: this image builds all the remaining tools required for the manylinux image. Any tool requiring specific system development package not required in the manylinux image shall be built from this image. For tools that used to be installed in /usr/local, they are staged in /manylinux-rootfs for an easy copy in the third stage.
manylinux: This image uses the runtime_base image as a base image. Tools built in the 2nd step are copied directly in the final filesystem. A finalization script is then run to install the remaining python tools/dependencies and run checks.
This PR builds on top of #876