Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvflowcontrol: add range controller interface #128021

Closed
Tracked by #123509
kvoli opened this issue Jul 31, 2024 · 0 comments · Fixed by #128195
Closed
Tracked by #123509

kvflowcontrol: add range controller interface #128021

kvoli opened this issue Jul 31, 2024 · 0 comments · Fixed by #128195
Assignees
@kvoli
Labels
A-replication-admission-control-v2 Related to introduction of replication AC v2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@kvoli
Copy link
Collaborator

kvoli commented Jul 31, 2024 •
edited by exalate-issue-sync bot
Loading

Implement a new RangeController interface in kvflowcontrol, similar to:

type RangeController interface {
	// WaitForEval is called concurrently by all requests wanting to evaluate.
	WaitForEval(ctx context.Context, pri admissionpb.WorkPriority) error
	// HandleRaftEvent is called from handleRaftReadyRaftMuLocked, including the
	// case of snapshot application.
	HandleRaftEvent(ctx context.Context, e RaftEvent) error
	// HandleControllerSchedulerEvent ia called from the raftScheduler when an
	// event the controller scheduled can be processed.
	HandleControllerSchedulerEvent(ctx context.Context) error
	SetReplicas(ctx context.Context, replicas ReplicaSet) error
	SetLeaseholder(ctx context.Context, replica roachpb.ReplicaID)
	Close(ctx context.Context)
}

Also see https://github.com/sumeerbhola/cockroach/blob/2064b279e95bbd68b0a7898503168d384cd0158c/pkg/kv/kvserver/kvflowcontrol/kvflowconnectedstream/stream.go#L461

Jira issue: CRDB-40756

Epic CRDB-37515
@kvoli kvoli added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-replication-admission-control-v2 Related to introduction of replication AC v2 labels Jul 31, 2024
@blathers-crl blathers-crl bot added the T-kv KV Team label Jul 31, 2024
@kvoli kvoli self-assigned this Aug 2, 2024
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 2, 2024
@kvoli
rac2: introduce range controller interface
62a5d2e 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
@kvoli kvoli mentioned this issue Aug 2, 2024
Merged
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 2, 2024
@kvoli
rac2: introduce range controller interface
0149f92 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 2, 2024
@kvoli
rac2: introduce range controller interface
0de9b39 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 5, 2024
@kvoli
rac2: introduce range controller interface
44b3f1a 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 5, 2024
@kvoli
rac2: introduce range controller interface
81c7116 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 5, 2024
@kvoli
rac2: introduce range controller interface
a466f4d 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 5, 2024
@kvoli
rac2: introduce range controller interface
ec73a10 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
kvoli added a commit to kvoli/cockroach that referenced this issue Aug 5, 2024
@kvoli
rac2: introduce range controller interface
658b444 
Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: cockroachdb#128021
Release note: None
craig bot pushed a commit that referenced this issue Aug 5, 2024
@kvoli
Merge #128195
be9b35d 
128195: kvflowcontrol: introduce v2 flow control core interfaces r=sumeerbhola a=kvoli

Introduce the `TokenCounter` interface, underneath a new package, `rac2`.
The `TokenCounter` will be used by replication flow control v2 in a
similar, but distinct manner from the token buckets in v1. Notably, the
`TokenCounter` supports returning an encapsulated handle that may be
used in waiting for available tokens, `TokenWaitingHandle`.

Resolves: #128014
Release note: None

---

Introduce the `StreamTokenCounterProvider` interface, which will be used
by replication flow control v2 to access the `TokenCounter` for a given
tenant, store stream.

Note the original interface in the v2 prototype included two additional
methods for metrics gathering which are omitted here.

Resolves: #128010
Release note: None

---

Introduce the `StreamTokenWatcher` interface, which will be used to
watch and be notified when there are tokens available.

Resolves: #128011
Release note: None

---

Introduce the `RangeController` interface, which provides flow control
for replication traffic in KV, for a range.

Resolves: #128021
Release note: None

Co-authored-by: Austen McClernon <[email protected]>
@craig craig bot closed this as completed in b0f296c Aug 5, 2024
@kvoli kvoli mentioned this issue Aug 13, 2024
Open
26 tasks
@github-project-automation github-project-automation bot moved this to Incoming in KV Aug 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kvoli kvoli

Labels
A-replication-admission-control-v2 Related to introduction of replication AC v2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
No open projects
KV
Status: Incoming
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

kvflowcontrol: introduce v2 flow control core interfaces kvoli/cockroach
1 participant
@kvoli