-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
kv: permit ExportRequest evaluation on followers #91405
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Nice!
@dt do you think there are any other backup-level tests that should be updated or added now that this is supported? Are there any tests that you would expect this change to have an impact on? For instance, should we be able to see the impact of this on intra-cluster network transfer in a long-running backup roachtest (where leases may move after the backup is planned)?
Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @arulajmani)
pkg/kv/kvserver/replica_follower_read.go
line 46 at r1 (raw file):
// read, then the batch might miss past writes served at higher timestamps // on the leaseholder. // 2. Each request in the batch needs to have clearly defined semantics when
small nit: If we're going to capitalize this sentence, let's do it for all of these conditions.
pkg/kv/kvserver/replica_follower_read.go
line 48 at r1 (raw file):
// 2. Each request in the batch needs to have clearly defined semantics when // served under the closed timestamp. This includes all "transactional" // requests and ExportRequests (which are non-transactional).
"which are non-transactional but which define a start and end timestamp"
pkg/roachpb/batch.go
line 195 at r1 (raw file):
} // IsAllTransactionalOrExportRequests returns true iff every request in the
Is this condition applicable more broadly than to just BatchCanBeEvaluatedOnFollower
? If not, consider inlining it into BatchCanBeEvaluatedOnFollower
. It might look like:
...
if tsFromServerClock {
return false
}
for _, ru := range ba.Requests {
r := ru.GetInner()
switch {
case roachpb.IsTransactional(r):
// TODO: explain
if !roachpb.IsReadOnly(r) || roachpb.IsLocking(r) {
return false
}
case r.Method() == roachpb.Export:
// TODO: explain
default:
return false
}
}
return true
0c460c5
to
f41c070
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for taking so long to get to the review comments; updated, PTAL.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten)
pkg/kv/kvserver/replica_follower_read.go
line 46 at r1 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
small nit: If we're going to capitalize this sentence, let's do it for all of these conditions.
No longer applies because I moved the explanation in-line. Let me know if you prefer otherwise.
pkg/roachpb/batch.go
line 195 at r1 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Is this condition applicable more broadly than to just
BatchCanBeEvaluatedOnFollower
? If not, consider inlining it intoBatchCanBeEvaluatedOnFollower
. It might look like:... if tsFromServerClock { return false } for _, ru := range ba.Requests { r := ru.GetInner() switch { case roachpb.IsTransactional(r): // TODO: explain if !roachpb.IsReadOnly(r) || roachpb.IsLocking(r) { return false } case r.Method() == roachpb.Export: // TODO: explain default: return false } } return true
Good call, this is much cleaner.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 3 of 3 files at r2, all commit messages.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @arulajmani)
pkg/kv/kvserver/replica_follower_read.go
line 68 at r2 (raw file):
// leaseholder. // Explanation of conditions required for a batch to be considered for // follower evaluation are defined inline.
Explanation of conditions required for a batch to be considered for follower evaluation are defined inline.
What is this sentence intending to convey? Didn't you just explain these conditions above?
pkg/kv/kvserver/replica_follower_read.go
line 74 at r2 (raw file):
case r.Method() == roachpb.Export: // Export requests also have clear semantics when served under the closed // timestamp as well, even though they aren't non-transactional, as they
"are non-transactional" or "aren't transactional"
Now that we've fixed cockroachdb#67016, and follower reads correctly synchronize with concurrent splits, it's safe for us to serve ExportRequests from followers. This patch permits that. Closes cockroachdb#88804 Release note: None
f41c070
to
df6f558
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten)
pkg/kv/kvserver/replica_follower_read.go
line 68 at r2 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
Explanation of conditions required for a batch to be considered for follower evaluation are defined inline.
What is this sentence intending to convey? Didn't you just explain these conditions above?
Lol I think I accidentally pressed p
here and I had that on my clipboard from before. Fixed.
pkg/kv/kvserver/replica_follower_read.go
line 74 at r2 (raw file):
Previously, nvanbenschoten (Nathan VanBenschoten) wrote…
"are non-transactional" or "aren't transactional"
Done.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewed 1 of 1 files at r3, all commit messages.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @arulajmani)
TFTR! bors r=nvanbenschoten |
Build failed (retrying...): |
Build succeeded: |
…in backup Following cockroachdb#91405 which enables followers to serve ExportRequests, this patch introduces an oracle that prefers non-potential-leaseholders when selecting a replica. This oracle is then used in backups so that ExportRequests during backups have a preference to be served by non-leaseholders. Release note: None
Now that we've fixed #67016, and follower reads correctly synchronize with concurrent splits, it's safe for us to serve ExportRequests from followers. This patch permits that.
Closes #88804
Release note: None