-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ci: caches keep growing #1850
Labels
Comments
Merged
michalinacienciala
added a commit
to keep-network/keep-ecdsa
that referenced
this issue
Jun 10, 2021
…pying We've seen a number of workflows failing with `No space left on device` error (e.g. https://github.com/keep-network/keep-ecdsa/actions/runs/924123923). The problem was a combination of a big cache size and significant memory allocation after building Docker image. According to moby/buildkit#1850: "At the moment caches are copied over the existing cache so it keeps growing". As a temporary fix (until issue gets fixed by GH), we introduce a step that moves the cache (similarily to how it's described in moby/buildkit#1896). The solution may not keep us from failures forever, as we still may hit the problems with the full memory if the size of the Docker image or cache increases (see this FD discussion: https://www.flowdock.com/app/cardforcoin/tech/threads/Clml1_lI4HYRKrr3N_vcSInDn6A).
nkuba
pushed a commit
to keep-network/keep-ecdsa
that referenced
this issue
Jun 15, 2021
…pying We've seen a number of workflows failing with `No space left on device` error (e.g. https://github.com/keep-network/keep-ecdsa/actions/runs/924123923). The problem was a combination of a big cache size and significant memory allocation after building Docker image. According to moby/buildkit#1850: "At the moment caches are copied over the existing cache so it keeps growing". As a temporary fix (until issue gets fixed by GH), we introduce a step that moves the cache (similarily to how it's described in moby/buildkit#1896). The solution may not keep us from failures forever, as we still may hit the problems with the full memory if the size of the Docker image or cache increases (see this FD discussion: https://www.flowdock.com/app/cardforcoin/tech/threads/Clml1_lI4HYRKrr3N_vcSInDn6A).
michalinacienciala
added a commit
to keep-network/keep-core
that referenced
this issue
Jun 16, 2021
In `keep-ecdsa` we've seen a number of workflows failing with `No space lefton device` error. The problem was a combination of a big cache size and significant memory allocation after building Docker image. According to moby/buildkit#1850: "At the moment caches are copied over the existing cache so it keeps growing". As a temporary fix (until issue gets fixed by GH), we introduced a step that moves the cache (similarily to how it's described in moby/buildkit#1896) instead of copying. Although we havent seen the problems with cache size yet in other projects than `keep-ecdsa`, we're applying the solution across the repositories to decrease the likelyhood of encountering problem in the future.
michalinacienciala
added a commit
to keep-network/tbtc
that referenced
this issue
Jun 16, 2021
In `keep-ecdsa` we've seen a number of workflows failing with `No space lefton device` error. The problem was a combination of a big cache size and significant memory allocation after building Docker image. According to moby/buildkit#1850: "At the moment caches are copied over the existing cache so it keeps growing". As a temporary fix (until issue gets fixed by GH), we introduced a step that moves the cache (similarily to how it's described in moby/buildkit#1896) instead of copying. Although we havent seen the problems with cache size yet in other projects than `keep-ecdsa`, we're applying the solution across the repositories to decrease the likelyhood of encountering problem in the future.
michalinacienciala
added a commit
to keep-network/tbtc-dapp
that referenced
this issue
Jun 16, 2021
In `keep-ecdsa` we've seen a number of workflows failing with `No space lefton device` error. The problem was a combination of a big cache size and significant memory allocation after building Docker image. According to moby/buildkit#1850: "At the moment caches are copied over the existing cache so it keeps growing". As a temporary fix (until issue gets fixed by GH), we introduced a step that moves the cache (similarily to how it's described in moby/buildkit#1896) instead of copying. Although we havent seen the problems with cache size yet in other projects than `keep-ecdsa`, we're applying the solution across the repositories to decrease the likelyhood of encountering problem in the future.
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Because caches are copied over the existing cache so it keeps growing and getting slower and less effective.
The text was updated successfully, but these errors were encountered: