Skip to content

Commit

Permalink
Merge #87303
Browse files Browse the repository at this point in the history
87303: kvserver,storage: calculate GCBytesAge in AddSSTable for points r=erikginaker a=itsbilal

Currently, the stats calculations in `AddSSTable` and `CheckSSTConflicts`
does not adjust for any GCBytesAge differences that arise from sst keys
deleting or shadowing engine keys. This results in mismatching GCBytesAge
values if we were to run the existing TestEvalAddSSTable test with
sufficiently-large "now" timestamps to accrue GCBytesAge.

This change updates TestEvalAddSSTable to multiply each timestamp
with 1e9 so that it operates in full-second increments to see more
interesting GCBytesAge behaviour. It also updates the code path
where we mass-update SST timestamps to make the corresponding
GCBytesAge adjustment. Finally, it ensures that GCBytesAge
is accrued from the right timestamps in cases of sst keys
/ tombstones shadowing engine keys / tombstones.

Note that this change only fixes GCBytesAge for point keys;
a fix for range keys will come in a follow-up.

Fixes #82920.

Release note: None.

Co-authored-by: Bilal Akhtar <[email protected]>
  • Loading branch information
craig[bot] and itsbilal committed Sep 8, 2022
2 parents 29c8e6e + 4f3d318 commit 9e331bb
Show file tree
Hide file tree
Showing 4 changed files with 207 additions and 100 deletions.
4 changes: 3 additions & 1 deletion pkg/kv/kvserver/batcheval/cmd_add_sstable.go
Original file line number Diff line number Diff line change
Expand Up @@ -173,11 +173,12 @@ func EvalAddSSTable(
// request timestamp. This ensures the writes comply with the timestamp cache
// and closed timestamp, i.e. by not writing to timestamps that have already
// been observed or closed.
var sstReqStatsDelta enginepb.MVCCStats
if sstToReqTS.IsSet() && (h.Timestamp != sstToReqTS || forceRewrite) {
st := cArgs.EvalCtx.ClusterSettings()
// TODO(dt): use a quotapool.
conc := int(AddSSTableRewriteConcurrency.Get(&cArgs.EvalCtx.ClusterSettings().SV))
sst, err = storage.UpdateSSTTimestamps(ctx, st, sst, sstToReqTS, h.Timestamp, conc)
sst, sstReqStatsDelta, err = storage.UpdateSSTTimestamps(ctx, st, sst, sstToReqTS, h.Timestamp, conc, args.MVCCStats)
if err != nil {
return result.Result{}, errors.Wrap(err, "updating SST timestamps")
}
Expand Down Expand Up @@ -219,6 +220,7 @@ func EvalAddSSTable(
args.Key, args.EndKey, desc.StartKey.AsRawKey(), desc.EndKey.AsRawKey())
statsDelta, err = storage.CheckSSTConflicts(ctx, sst, readWriter, start, end, leftPeekBound, rightPeekBound,
args.DisallowShadowing, args.DisallowShadowingBelow, maxIntents, usePrefixSeek)
statsDelta.Add(sstReqStatsDelta)
if err != nil {
return result.Result{}, errors.Wrap(err, "checking for key collisions")
}
Expand Down
Loading

0 comments on commit 9e331bb

Please sign in to comment.