Improve build-image incremental workflow #561
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
markmandel
added
kind/feature
markmandel
requested review from
XAMPPRocky,
luna-duclos,
rezvaneh,
vallfors and
bombsimon
as code owners
|
CI failure should be fixed by #560 |
XAMPPRocky
approved these changes
Aug 9, 2022
Several improvements to workflows that flow through the build-image to improve build and iteration times. This includes: We don't need this anymore, so no reason to build it for each architecture. I realised that when mixing the `build-image` workflow with local rust tools, that going between them, they would invalidate each other's incremental cache, since it was fingerprinted on absolute directories - drastically expanding build times between iterations. This drastically improves both the incremental build times of images, and every other workflow through `build-image`, such as e2e testing. (`make test-agones` can be used iteratively now). We only use a few files for the Docker image, so let's ignore everything else. Took the docker context down from 1GB+ to < 50MB. On a linux machine all this brings incremental builds down from 10's of minutes to a minute or less, since only what has changed is compiled and pushed as we can now take advantage of rust incremental build and docker image caching. Work on googleforgames#553
|
Ah! This is actually a real issue: I'll go fix. |
|
Build Succeeded 🥳 Build Id: b851b3dc-0e08-46ea-935d-d3bd6b32c7eb To build this version: |
XAMPPRocky
pushed a commit
that referenced
this pull request
Aug 22, 2022
* Improve build-image incremental workflow Several improvements to workflows that flow through the build-image to improve build and iteration times. This includes: We don't need this anymore, so no reason to build it for each architecture. I realised that when mixing the `build-image` workflow with local rust tools, that going between them, they would invalidate each other's incremental cache, since it was fingerprinted on absolute directories - drastically expanding build times between iterations. This drastically improves both the incremental build times of images, and every other workflow through `build-image`, such as e2e testing. (`make test-agones` can be used iteratively now). We only use a few files for the Docker image, so let's ignore everything else. Took the docker context down from 1GB+ to < 50MB. On a linux machine all this brings incremental builds down from 10's of minutes to a minute or less, since only what has changed is compiled and pushed as we can now take advantage of rust incremental build and docker image caching. Work on #553 * Fix issue with site generation.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What type of PR is this?
/kind feature
What this PR does / Why we need it:
Several improvements to workflows that flow through the build-image to improve build and iteration times.
This includes:
Removal of debug images
We don't need this anymore, so no reason to build it for each architecture.
Use
build-imagetarget directoryI realised that when mixing the
build-imageworkflow with local rust tools, that going between them, they would invalidate each other's incremental cache, since it was fingerprinted on absolute directories - drastically expanding build times between iterations.Moving to a specific target directory for
build-imageusage drastically improves both the incremental build times of images, and every other workflow throughbuild-image, such as e2e testing. (make test-agonescan be used iteratively now).More aggressive docker ignore
We only use a few files for the Docker image, so let's ignore everything else. Took the docker context down from 1GB+ to < 50MB.
On a linux machine all this brings incremental builds down from 10's of minutes to a minute or less, since only what has changed is compiled and pushed as we can now take advantage of rust incremental build and docker image caching.
Which issue(s) this PR fixes:
Work on #553
Special notes for your reviewer:
I know what the impact is on a Linux environment, would be good to know how it improves things on a Mac and Windows environment where there is an extra layer of virtualisation.