Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(ci): Increase CI disk size to 200GB #4945

Merged
merged 1 commit into from
Aug 25, 2022
Merged

fix(ci): Increase CI disk size to 200GB #4945

merged 1 commit into from
Aug 25, 2022

Conversation

teor2345
Copy link
Contributor

Motivation

Some tests are failing because they run out of disk space.

We can limit the size of the database by changing how it is configured in Zebra, but right now we just need to make CI pass.

Logs

Unread Stderr:
...
Message: expected that errors would not occur when writing to disk or updating note commitment and history trees: Error { message: "IO error: No space left on device: While appending to file: /var/cache/zebrad-cache/state/v25/mainnet/017386.log: No space left on device" }
Location: zebra-state/src/service.rs:301

https://github.com/ZcashFoundation/zebra/runs/8002360538?check_suite_focus=true#step:6:1216

Solution

  • Increase the size of cached state disks to 200GB
  • Increase the size of boot disks to 200GB (the send transaction test copies the state to /tmp)

Review

This is an urgent fix to CI, anyone can review it.

Reviewer Checklist

  • CI passes

@teor2345 teor2345 added C-bug Category: This is a bug A-devops Area: Pipelines, CI/CD and Dockerfiles P-Critical 🚑 I-heavy Problems with excessive memory, disk, or CPU usage A-state Area: State / database changes labels Aug 24, 2022
@teor2345 teor2345 self-assigned this Aug 24, 2022
@teor2345 teor2345 requested a review from a team as a code owner August 24, 2022 19:43
@teor2345 teor2345 requested review from gustavovalverde and removed request for a team August 24, 2022 19:43
Copy link
Member

@gustavovalverde gustavovalverde left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@teor2345 teor2345 mentioned this pull request Aug 24, 2022
1 task
@teor2345 teor2345 merged commit 7fc3cdd into main Aug 25, 2022
@teor2345 teor2345 deleted the fix-ci-disk-space branch August 25, 2022 06:41
@teor2345
Copy link
Contributor Author

The 200GB disk size only applies to newly created disks, so we need to:

  • admin-merge this PR
  • wait for a full sync to run on main

Then other PRs will start passing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-devops Area: Pipelines, CI/CD and Dockerfiles A-state Area: State / database changes C-bug Category: This is a bug I-heavy Problems with excessive memory, disk, or CPU usage
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants