Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rangefeed: limit memory budget used by a single tenant per kv node #78942

Open
aliher1911 opened this issue Mar 29, 2022 · 4 comments
Open

rangefeed: limit memory budget used by a single tenant per kv node #78942

aliher1911 opened this issue Mar 29, 2022 · 4 comments
Labels
A-kv-replication Relating to Raft, consensus, and coordination. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@aliher1911
Copy link
Contributor

aliher1911 commented Mar 29, 2022

Is your feature request related to a problem? Please describe.
Current rangefeed memory budget model has one pool for system tenant system ranges and another for all user ranges. This model while protecting kv nodes from running out of memory doesn't isolate tenants from each other. One tenant could theoretically hoard all rangefeed memory in a slow feed thus forcing other tenants to restart feeds and make catch up scans.

Describe the solution you'd like
A better approach could be limiting amount of memory used by each individual tenant per node. That would add another level in budget hierarchy where each range has a budget which is pooled with a capped budget per tenant which is then further pooled with all rangefeeds on the node.

Jira issue: CRDB-14456

Epic CRDB-37617

@aliher1911 aliher1911 added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Mar 29, 2022
@aliher1911 aliher1911 added A-kv-replication Relating to Raft, consensus, and coordination. T-kv-replication labels Mar 29, 2022
@erikgrinaker
Copy link
Contributor

This should also have isolation between user and system ranges for the tenant, and be integrated with the new dynamic system span configs in #74555.

@aliher1911
Copy link
Contributor Author

Do we want it as a single feature with all the follow ups for rangefeed budgets? I think having per tenant is pretty straightforward to implement while dynamic spans need some thinking as we need to come up with a way to handle dynamic info.

@erikgrinaker
Copy link
Contributor

We can use a separate issue if you prefer.

@aliher1911
Copy link
Contributor Author

Added #78977 for separate budgets and dynamic configs.

@exalate-issue-sync exalate-issue-sync bot added T-kv KV Team and removed T-kv-replication labels Jun 28, 2024
@github-project-automation github-project-automation bot moved this to Incoming in KV Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv-replication Relating to Raft, consensus, and coordination. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
No open projects
Status: Incoming
Development

No branches or pull requests

2 participants