Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

rangefeed: remove catchup iterator creation from processor work loop #111040

Closed
aliher1911 opened this issue Sep 21, 2023 · 1 comment · Fixed by #111045
Closed

rangefeed: remove catchup iterator creation from processor work loop #111040

aliher1911 opened this issue Sep 21, 2023 · 1 comment · Fixed by #111045
Assignees
@aliher1911
Labels
A-kv-replication Relating to Raft, consensus, and coordination. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Comments

@aliher1911
Copy link
Contributor

aliher1911 commented Sep 21, 2023 •
edited by exalate-issue-sync bot
Loading

Currently iterator creation is deferred to registration step to simplify iterator lifecycle management.
It must be closed if we failed to create rangefeed registration for any reason.

raftMu is held throughout the whole rangefeed request handling op so there's no issue with getting inconsistent view between iterator and events.

By moving iterator creation out of processor work loop, we remove all io dependent operation from scheduler thus removing possibilies of starving neighbour rangefeeds for extended periods of time.
This issue could be particularly bad at startup when multiple rangefeeds start at the same time.

Jira issue: CRDB-31737

Epic CRDB-26372
@aliher1911 aliher1911 added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv-replication Relating to Raft, consensus, and coordination. T-kv-replication labels Sep 21, 2023
@blathers-crl
Copy link

blathers-crl bot commented Sep 21, 2023

cc @cockroachdb/replication

@aliher1911 aliher1911 mentioned this issue Sep 21, 2023
Merged
@craig craig bot closed this as completed in 235babd Sep 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@aliher1911 aliher1911

Labels
A-kv-replication Relating to Raft, consensus, and coordination. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

rangefeed: create catchup iterators eagerly aliher1911/cockroach
1 participant
@aliher1911