Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: clearrange roachtest should initialize monitor earlier #112049

Closed
jbowens opened this issue Oct 9, 2023 · 0 comments · Fixed by #112203
Closed

roachtest: clearrange roachtest should initialize monitor earlier #112049

jbowens opened this issue Oct 9, 2023 · 0 comments · Fixed by #112203
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. A-testing Testing tools and infrastructure C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-storage Storage Team

Comments

@jbowens
Copy link
Collaborator

jbowens commented Oct 9, 2023 •
edited by cockroach-jira-scripts
Loading

In #112016 we see a disk stall during the large bank import at beginning of the clearrange roachtest. At this point the node monitor hasn't been created. This resulted in the test timing out only after its 6 hour timeout has elapsed.

Jira issue: CRDB-32213
@jbowens jbowens added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. A-storage Relating to our storage engine (Pebble) on-disk storage. A-testing Testing tools and infrastructure T-storage Storage Team labels Oct 9, 2023
@jbowens jbowens mentioned this issue Oct 9, 2023
Closed
@jbowens jbowens mentioned this issue Oct 11, 2023
Merged
craig bot pushed a commit that referenced this issue Oct 12, 2023
@jbowens
Merge #112203
23faaa6 
112203: cmd/roachtest: create monitor earlier in clearrange roachtest r=sumeerbhola a=jbowens

Previously, the node monitor wasn't created until after the import. If a node died during the import, it went undetected. Now we set up a monitor for the import, and then a new monitor post-restart.

Close #112049.

Co-authored-by: Jackson Owens <[email protected]>
@craig craig bot closed this as completed in 874b3fc Oct 12, 2023
@jbowens jbowens moved this to Done in [Deprecated] Storage Jun 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
A-storage Relating to our storage engine (Pebble) on-disk storage. A-testing Testing tools and infrastructure C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-storage Storage Team
Projects
No open projects
[Deprecated] Storage
Archived in project
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

cmd/roachtest: create monitor earlier in clearrange roachtest jbowens/cockroach
1 participant
@jbowens