Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

roachprod: add aws AZ override for c6id.24xlarge #105234

Conversation

srosenberg
Copy link
Member

Since the bump to new instance types in GCE and AWS [1], we are still experiencing occasional cluster creation issues owing to "insufficient capacity". GCE quota has already been bumped, with asia-northeast1 being the latest, and hopefully last.

The most recent cluster creation in AWS is owing to "insufficient capacity" of c6id.24xlarge in us-east-2a. As a workaround, we extend the existing zone override to place c6id.24xlarge into us-east-2b, which
allegedly has sufficient capacity.

Note, the long-term fix is to rework how cluster creation retry currently operates, by effectively trying other AZs.

[1] #104419

Epic: none
Fixes: #78601 (comment)

Release note: None

Since the bump to new instance types in GCE and AWS [1],
we are still experiencing occasional cluster creation
issues owing to "insufficient capacity". GCE quota has
already been bumped, with `asia-northeast1` being the
latest, and hopefully last.

The most recent cluster creation in AWS is owing to
"insufficient capacity" of `c6id.24xlarge` in us-east-2a.
As a workaround, we extend the existing zone override
to place `c6id.24xlarge` into us-east-2b, which
allegedly has sufficient capacity.

Note, the long-term fix is to rework how cluster creation
retry currently operates, by effectively trying other AZs.

[1] cockroachdb#104419

Epic: none
Fixes: cockroachdb#78601 (comment)

Release note: None
@srosenberg srosenberg requested a review from a team as a code owner June 20, 2023 22:24
@srosenberg srosenberg requested review from herkolategan and smg260 and removed request for a team June 20, 2023 22:24
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@srosenberg srosenberg added backport-22.2.x backport-23.1.x Flags PRs that need to be backported to 23.1 labels Jun 20, 2023
@srosenberg srosenberg requested a review from renatolabs June 20, 2023 22:25
@srosenberg
Copy link
Member Author

TFTR!

bors r=renatolabs

@craig
Copy link
Contributor

craig bot commented Jun 21, 2023

Build succeeded:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.1.x Flags PRs that need to be backported to 23.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

roachtest: cluster_creation failed
3 participants