Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvflowcontrol: add raft, scheduler, ready interface #128019

Closed
Tracked by #123509
kvoli opened this issue Jul 31, 2024 · 0 comments
Closed
Tracked by #123509

kvflowcontrol: add raft, scheduler, ready interface #128019

kvoli opened this issue Jul 31, 2024 · 0 comments
Assignees
Labels
A-replication-admission-control-v2 Related to introduction of replication AC v2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team

Comments

@kvoli
Copy link
Collaborator

kvoli commented Jul 31, 2024

Add three new interfaces, which define methods the replication AC v2 controller will call or use, eventually implemented ontop of raft:

type RaftEvent interface {
	// Ready can return nil if there is no Ready.
	Ready() Ready
}

type Ready interface {
	// GetEntries represents the new entries that are being added to the log.
	// This may be empty.
	GetEntries() []raftpb.Entry
}

// RaftInterface abstracts what the RangeController needs from the raft
// package, when running at the leader.
type RaftInterface interface {
	// FollowerState returns the current state of a follower. The value of
	// Match, Next, Admitted are populated iff in StateReplicate. All entries >=
	// Next have not had MsgApps constructed during the lifetime of this
	// StateReplicate (they may have been constructed previously).
	FollowerState(replicaID roachpb.ReplicaID) FollowerStateInfo
	// FollowerStateRLocked return the current state of a follower.
	FollowerStateRLocked(replicaID roachpb.ReplicaID) FollowerStateInfo
	// NextUnstableIndex returns the index of the next entry that will be sent to
	// local storage, if there are any. All entries < this index are either stored,
	// or have been sent to storage.
	NextUnstableIndex() uint64
	// MakeMsgApp is used to construct a MsgApp for entries in [start, end).
	MakeMsgApp(replicaID roachpb.ReplicaID, start, end uint64, maxSize int64) (raftpb.Message, error)
}

// Scheduler abstracts the raftScheduler to allow the RangeController to
// schedule its own internal processing. This internal processing is to pop
// some entries from the send queue and send them in a MsgApp.
type Scheduler interface {
	ScheduleControllerEvent(rangeID roachpb.RangeID)
}

See https://github.com/sumeerbhola/cockroach/blob/2064b279e95bbd68b0a7898503168d384cd0158c/pkg/kv/kvserver/kvflowcontrol/kvflowconnectedstream/stream.go#L653

Jira issue: CRDB-40754

Epic CRDB-37515

@kvoli kvoli added C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) A-replication-admission-control-v2 Related to introduction of replication AC v2 labels Jul 31, 2024
@blathers-crl blathers-crl bot added the T-kv KV Team label Jul 31, 2024
craig bot pushed a commit that referenced this issue Aug 27, 2024
128470: sql: scaffolding for db metadata update job r=xinhaoz a=xinhaoz

This commit creates a new forever running BG job called UpdateCachedTableMetadata. This is just some scaffolding - the job doesn't do anything right now but it will be used to populate the system table storing cached table metadata used by obs surfaces.

Epic: CRDB-37558
Fixes: #127891

Release note: None

129719: kvserver,rac2: add and plumb RaftEvent r=sumeerbhola a=pav-kv

This change updates the `RaftEvent` structure to reflect the semantics of a raft storage write, and plumbs it into RACv2 code in a few places.

Part of #128019

Co-authored-by: Xin Hao Zhang <[email protected]>
Co-authored-by: Pavel Kalinnikov <[email protected]>
@github-project-automation github-project-automation bot moved this to Incoming in KV Aug 28, 2024
@kvoli kvoli closed this as completed Sep 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-replication-admission-control-v2 Related to introduction of replication AC v2 C-enhancement Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception) T-kv KV Team
Projects
No open projects
Status: Incoming
Development

No branches or pull requests

2 participants