Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvflowcontrol: add store streams token counter interface #128010

Closed
Tracked by #123509
kvoli opened this issue Jul 31, 2024 · 0 comments · Fixed by #128195
Closed
Tracked by #123509

kvflowcontrol: add store streams token counter interface #128010

kvoli opened this issue Jul 31, 2024 · 0 comments · Fixed by #128195
Assignees
Labels
A-replication-admission-control-v2 Related to introduction of replication AC v2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@kvoli
Copy link
Collaborator

kvoli commented Jul 31, 2024

Define a new interface in kvflowcontrol which supports a similar interface to:

// StoreStreamsTokenCounter is one per node.
type StoreStreamsTokenCounter interface {
	// EvalTokenCounterForStrea returns the eval token counter for the given
	// stream.
	EvalTokenCounterForStream(kvflowcontrol.Stream) TokenCounter
	// SendTokenCounterForStream returns the send token counter for the given
	// stream.
	SendTokenCounterForStream(kvflowcontrol.Stream) TokenCounter
	// GetEvalTokenCounters returns every eval token counter. This is used for
	// metrics and inspection.
	GetEvalTokenCounters() map[kvflowcontrol.Stream]TokenCounter
	// GetSendTokenCounters returns every send token counter. This is used for
	// metrics and inspection.
	GetSendTokenCounters() map[kvflowcontrol.Stream]TokenCounter
}

Also see the relevant prototype code.

Jira issue: CRDB-40736

Epic CRDB-37515

@kvoli kvoli added the C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) label Jul 31, 2024
@blathers-crl blathers-crl bot added the T-kv KV Team label Jul 31, 2024
@kvoli kvoli added the A-replication-admission-control-v2 Related to introduction of replication AC v2 label Jul 31, 2024
@kvoli kvoli self-assigned this Aug 2, 2024
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 2, 2024
Introduce the `StreamTokenCounterProvider` interface, which will be used
by replication flow control v2 to access the `TokenCounter` for a given
tenant, store stream.

Note the original interface in the v2 prototype included two additional
methods for metrics gathering which are omitted here.

Resolves: cockroachdb#128010
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 2, 2024
Introduce the `StreamTokenCounterProvider` interface, which will be used
by replication flow control v2 to access the `TokenCounter` for a given
tenant, store stream.

Note the original interface in the v2 prototype included two additional
methods for metrics gathering which are omitted here.

Resolves: cockroachdb#128010
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 5, 2024
Introduce the `StreamTokenCounterProvider` interface, which will be used
by replication flow control v2 to access the `TokenCounter` for a given
tenant, store stream.

Note the original interface in the v2 prototype included two additional
methods for metrics gathering which are omitted here.

Resolves: cockroachdb#128010
Release note: None
craig bot pushed a commit that referenced this issue Aug 5, 2024
128195: kvflowcontrol: introduce v2 flow control core interfaces r=sumeerbhola a=kvoli

Introduce the `TokenCounter` interface, underneath a new package, `rac2`.
The `TokenCounter` will be used by replication flow control v2 in a
similar, but distinct manner from the token buckets in v1. Notably, the
`TokenCounter` supports returning an encapsulated handle that may be
used in waiting for available tokens, `TokenWaitingHandle`.

Resolves: #128014
Release note: None

---

Introduce the `StreamTokenCounterProvider` interface, which will be used
by replication flow control v2 to access the `TokenCounter` for a given
tenant, store stream.

Note the original interface in the v2 prototype included two additional
methods for metrics gathering which are omitted here.

Resolves: #128010
Release note: None

---

Introduce the `StreamTokenWatcher` interface, which will be used to
watch and be notified when there are tokens available.

Resolves: #128011
Release note: None

---

Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: #128021
Release note: None

Co-authored-by: Austen McClernon <[email protected]>
@craig craig bot closed this as completed in c2b14c8 Aug 5, 2024
@github-project-automation github-project-automation bot moved this to Incoming in KV Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-replication-admission-control-v2 Related to introduction of replication AC v2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
No open projects
Status: Incoming
Development

Successfully merging a pull request may close this issue.

1 participant