Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: timestamp and invalidate storepool estimates #93532

Open
kvoli opened this issue Dec 13, 2022 · 0 comments
Open

kvserver: timestamp and invalidate storepool estimates #93532

kvoli opened this issue Dec 13, 2022 · 0 comments
Labels
A-kv Anything in KV that doesn't belong in a more specific category. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@kvoli
Copy link
Collaborator

kvoli commented Dec 13, 2022

Is your feature request related to a problem? Please describe.
The storepool for each store locally tracks every store's capacity (qps, lease count etc). This capacity is gathered from gossip and used in allocation. When an allocation change occurs, which originates locally, the local store will update the storepool with an estimate of the impact. This is done to allow the local store to continue to make allocation decisions that take into account the recent change.

These local estimates are overridden by gossip, each time a new descriptor is received.

This leads to an issue where an action could take place at t2, while the capacity of a store involved in that action could be sent at t1 and received by another store that originated this action at t3. Here, the local update is overridden by the gossip update at t1.

The solution is to include time in both local estimate application and when gossip updates are received.

More specifically,

  1. when an estimated impact is applied locally to a store capacity, we include the current timestamp.
  2. when creating a store capacity to be gossiped, we timestamp it with the current time.
  3. when receiving a store capacity from gossip, we invalidate any estimated impacts which have a timestamp less than the timestamp associated with the gossiped capacity.

Jira issue: CRDB-22382

@kvoli kvoli added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-kv Anything in KV that doesn't belong in a more specific category. labels Dec 13, 2022
@blathers-crl blathers-crl bot added the T-kv KV Team label Dec 13, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-kv Anything in KV that doesn't belong in a more specific category. C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
None yet
Development

No branches or pull requests

1 participant