-
Notifications
You must be signed in to change notification settings - Fork 113
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add caching to CI operations #1696
Merged
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…integration test execution with count=1
…cache hit for this commit when cache is not yet populated
Looks good! |
georgysavva
approved these changes
Sep 22, 2021
BTW, are we done with the elimination of large |
karimodm
approved these changes
Oct 11, 2021
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description of change
This PR improves runtimes of our CI workflows for unit tests and integration tests by leveraging Github Actions Cache.
tester
containerNote: Loading the same cache on multiple jobs in the same workflow causes some issues because the first runner will create the cache and following runners will get a cache hit. However, as long as the first runner did not yet finish its execution the cache will be empty. That's why I opted for an approach to always fall back to another cache with
restore-keys
. This might lead to very frequent cache eviction from Github because of the 5GB max cache size. I guess we have to see how it goes in general.Note 2: experiments showed that caching the Docker image layers with Buildkit is not really worth it as long as exporting the internal Buildkit cache can't be exported. Importing/exporting the cache + rebuilding the binary is simply more expensive than just rebuilding the full image from scratch (see here for more information).
Runtime integration tests with caching of integration tests (tester Go dependencies and build cache) (left) vs without caching (right)
Runtime unit tests with caching of Go modules and build cache (left) vs without caching (right)
Fixes #1689
Type of change