This repository has been archived by the owner on May 22, 2023. It is now read-only.
Temp fix for the problem with failing jobs - move cache instead of copying #822
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…pying We've seen a number of workflows failing with `No space left on device` error (e.g. https://github.com/keep-network/keep-ecdsa/actions/runs/924123923). The problem was a combination of a big cache size and significant memory allocation after building Docker image. According to moby/buildkit#1850: "At the moment caches are copied over the existing cache so it keeps growing". As a temporary fix (until issue gets fixed by GH), we introduce a step that moves the cache (similarily to how it's described in moby/buildkit#1896). The solution may not keep us from failures forever, as we still may hit the problems with the full memory if the size of the Docker image or cache increases (see this FD discussion: https://www.flowdock.com/app/cardforcoin/tech/threads/Clml1_lI4HYRKrr3N_vcSInDn6A).
|
Given that this is an exact port over from https://github.com/docker/build-push-action/blob/master/docs/advanced/cache.md#github-cache, I think we're good here! |
beaurancourt
approved these changes
Jun 10, 2021
This was referenced Jun 16, 2021
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We've seen a number of workflows failing with
No space left on deviceerror (e.g.
https://github.com/keep-network/keep-ecdsa/actions/runs/924123923).
The problem was a combination of a big cache size and significant memory
allocation after building Docker image.
According to moby/buildkit#1850:
"At the moment caches are copied over the existing cache so it keeps
growing".
As a temporary fix (until issue gets fixed by GH), we introduce a step
that moves the cache (similarily to how it's described in
moby/buildkit#1896).
The solution may not keep us from failures forever, as we still may
hit the problems with the full memory if the size of the Docker image
or cache increases (see this FD discussion:
https://www.flowdock.com/app/cardforcoin/tech/threads/Clml1_lI4HYRKrr3N_vcSInDn6A).