Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvserver, gossip: broadcast read amplification levels in StoreCapacity #74254

Closed
aayushshah15 opened this issue Dec 23, 2021 · 0 comments · Fixed by #77040
Closed

kvserver, gossip: broadcast read amplification levels in StoreCapacity #74254

aayushshah15 opened this issue Dec 23, 2021 · 0 comments · Fixed by #77040
Assignees
@kvoli
Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@aayushshah15
Copy link
Contributor

We currently don't gossip any metrics about the health of a store's LSM tree. There have been discussions about not rebalancing to stores with poor LSM health. To enable this, we'd like to start by broadcasting read amplification levels via the StoreDescriptors that are periodically gossiped around.
@aayushshah15 aayushshah15 added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Dec 23, 2021
@blathers-crl blathers-crl bot added the T-kv KV Team label Dec 23, 2021
@aayushshah15 aayushshah15 mentioned this issue Jan 6, 2022
Open
@erikgrinaker erikgrinaker mentioned this issue Jan 6, 2022
Closed
@shralex shralex assigned kvoli and unassigned amygao9 Feb 23, 2022
kvoli added a commit to kvoli/cockroach that referenced this issue Feb 25, 2022
@kvoli
kvserver: gossip store read amplification
1008d5c 
Previously, other store statistics were gossipped between stores to
inform allocation and rebalancing decisions. This patch introduces an
additional gosipped value, the store read amplification. This patch
allows the allocator to avoid rebalancing ranges onto stores with poor
lsm health, indicated by a large read amplification.

resolves cockroachdb#74254

Release note: None
@kvoli kvoli mentioned this issue Feb 25, 2022
Merged
craig bot pushed a commit that referenced this issue Feb 27, 2022
@kvoli
Merge #77040
eee8ef0 
77040: kvserver: gossip store read amplification r=kvoli a=kvoli

Previously, other store statistics were gossipped between stores to
inform allocation and rebalancing decisions. This patch introduces an
additional gosipped value, the store read amplification. This patch
allows the allocator to avoid rebalancing ranges onto stores with poor
lsm health, indicated by a large read amplification.

resolves #74254

Release note: None

Co-authored-by: Austen McClernon <[email protected]>
@craig craig bot closed this as completed in ffdc631 Feb 27, 2022
RajivTS pushed a commit to RajivTS/cockroach that referenced this issue Mar 6, 2022
@kvoli @RajivTS
kvserver: gossip store read amplification
054ddaa 
Previously, other store statistics were gossipped between stores to
inform allocation and rebalancing decisions. This patch introduces an
additional gosipped value, the store read amplification. This patch
allows the allocator to avoid rebalancing ranges onto stores with poor
lsm health, indicated by a large read amplification.

resolves cockroachdb#74254

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kvoli kvoli

Labels
C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvserver: gossip store read amplification kvoli/cockroach
3 participants
@aayushshah15 @amygao9 @kvoli