-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
roachtest: use spot VM for all tests #127896
roachtest: use spot VM for all tests #127896
Conversation
2d4aeac
to
7ffe9ff
Compare
76f7f8c
to
08bb256
Compare
Ran 2 teamcity builds. The failure is ~13% |
Currently, we are using spot VMs only to run benchmark tests. The reason for the low adoption of spot VM is because of the test failures due to VM preemption. But, with the recent changes, we have better control on VM preemption changes where if there is a test failure due to preemption, the test is run on an on-demand VM the next time. Also, the current failed tests that are run on spot VM is 148 out of 1093 = 13.5%. If I consider failure out of total tests run is 148 out of 2508 = 5.9%. So, this PR removes the condition to use spot VMs only for benchmark tests and changes the probability to 75%. Fixes: cockroachdb#127236 Epic: None
08bb256
to
69da7de
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Thanks @shailendra-patel and @vidit-bhat for approving the change. |
bors r=@shailendra-patel,@vidit-bhat |
Based on the specified backports for this PR, I applied new labels to the following linked issue(s). Please adjust the labels as needed to match the branches actually affected by the issue(s), including adding any known older branches. Issue #127236: branch-release-24.1, branch-release-24.2. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
Encountered an error creating backports. Some common things that can go wrong:
You might need to create your backport manually using the backport tool. error creating merge commit from 69da7de to blathers/backport-release-24.1-127896: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 24.1.x failed. See errors above. error creating merge commit from 69da7de to blathers/backport-release-24.2-127896: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 24.2.x failed. See errors above. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
blathers backport 24.1 24.2 |
Encountered an error creating backports. Some common things that can go wrong:
You might need to create your backport manually using the backport tool. error creating merge commit from 69da7de to blathers/backport-release-24.1-127896: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 24.1 failed. See errors above. error creating merge commit from 69da7de to blathers/backport-release-24.2-127896: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict [] you may need to manually resolve merge conflicts with the backport tool. Backport to branch 24.2 failed. See errors above. 🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf. |
Currently, we are using spot VMs only to run benchmark tests. The reason for the low adoption of spot VM is because of the test failures due to VM preemption.
But, with the recent changes, we have better control on VM preemption changes where if there is a test failure due to preemption, the test is run on an on-demand VM the next time.
Also, the current failed tests that are run on spot VM is 148 out of 1093 = 13.5%. If I consider failure out of total tests run is 148 out of 2508 = 5.9%.
So, this PR removes the condition to use spot VMs only for benchmark tests and changes the probability to 75%.
Fixes: #127236
Epic: None