-
Notifications
You must be signed in to change notification settings - Fork 3.9k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
kv: mark AddSSTable requests "unsplittable"
Usually we split span requests automatically in DistSender however AddSSTable requests are special: their large SST file is an opaque payload that contains keys that could span the post-split requests. We currently assert during ingestion that the span of an SST's contents matches the request's span, as the file is ingested whole, without modification, meaning that currently a split-and-truncated request fails. However we if a request will fail, we can reject it as soon as we'd try to split it, before sending these large, expensive files around. More importantly, if communicate back to the caller the actual range bounds, it can try again, generating re-chunked SSTs with bounds that should be OK. Note: we could try to make the recipient able to correctly ingest only the subset of the unsplit SST by iterating over it and generating a new file containing just those keys in the request span. However this is significant complexity to add to a raft command evaluation, and changes the performance characteristics of `AddSSTable` that make it appealing in the first place. Release note: none.
- Loading branch information
Showing
3 changed files
with
25 additions
and
9 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters