Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci: Use restore-key to continually update ccache storage #1263

Merged
merged 4 commits into from
May 22, 2022

Conversation

paulgessinger
Copy link
Member

We observed a brief period of speedup in the CI after introducing ccache, which degraded again after a few weeks:

citime_Analysis_0
citime_Builds_0
citime_Builds_1
citime_Builds_2
citime_Builds_3

This is due to the fact that GitHub Actions caches do not get updated at the end of their run. This PR introduces a workaround. The CI will

  1. Look for a key like cache_key_$OS_$JOB_$COMMIT_SHA and use that if found. If not it will
  2. Look for the newest key starting with cache_key_$OS_$JOB_, and use that if found. Otherwise it will perform a build from scratch
  3. At the end of the job, if a cache using the restore key (i.e. 2. above) was used, it will create a new cache with the key from 1 (cache_key_$OS_$JOB_$COMMIT_SHA). The next CI run will use that cache as a starting point.

Clearly, this means that the cache size will continually increase. However, ccache has a maximum size setting, which I set to 2GB in this PR. We'll have to see if that results in a reasonably sized, reasonably up-to-date cache and corresponding fast build times.

@paulgessinger paulgessinger added the Infrastructure Changes to build tools, continous integration, ... label May 21, 2022
@paulgessinger paulgessinger added this to the next milestone May 21, 2022
@codecov
Copy link

codecov bot commented May 21, 2022

Codecov Report

Merging #1263 (3324a56) into main (6eb3937) will increase coverage by 0.40%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #1263      +/-   ##
==========================================
+ Coverage   47.31%   47.71%   +0.40%     
==========================================
  Files         375      375              
  Lines       19758    19718      -40     
  Branches     9345     9264      -81     
==========================================
+ Hits         9348     9409      +61     
+ Misses       4021     3920     -101     
  Partials     6389     6389              
Impacted Files Coverage Δ
Core/include/Acts/Seeding/SeedfinderConfig.hpp 0.00% <0.00%> (ø)
Core/include/Acts/Propagator/EigenStepper.ipp 50.73% <0.00%> (+0.37%) ⬆️
Core/src/TrackFitting/GainMatrixSmoother.cpp 10.86% <0.00%> (+0.45%) ⬆️
Core/include/Acts/Propagator/ConstrainedStep.hpp 74.50% <0.00%> (+7.20%) ⬆️
Core/include/Acts/Propagator/AtlasStepper.hpp 71.01% <0.00%> (+8.95%) ⬆️

📣 Codecov can now indicate which changes are the most critical in Pull Requests. Learn more

Copy link
Contributor

@andiwand andiwand left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

great 👍

@kodiakhq kodiakhq bot merged commit 63f33a3 into acts-project:main May 22, 2022
@paulgessinger paulgessinger deleted the ci/ccache-tune branch May 22, 2022 14:25
@paulgessinger paulgessinger modified the milestones: next, v19.1.0 May 25, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
automerge Infrastructure Changes to build tools, continous integration, ...
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants