Skip to content

Commit

Permalink
kvserver,storage: calculate GCBytesAge in AddSSTable for points
Browse files Browse the repository at this point in the history
Currently, the stats calculations in `AddSSTable` and `CheckSSTConflicts`
does not adjust for any GCBytesAge differences that arise from sst keys
deleting or shadowing engine keys. This results in mismatching GCBytesAge
values if we were to run the existing TestEvalAddSSTable test with
sufficiently-large "now" timestamps to accrue GCBytesAge.

This change updates TestEvalAddSSTable to multiply each timestamp
with 1e9 so that it operates in full-second increments to see more
interesting GCBytesAge behaviour. It also updates the code path
where we mass-update SST timestamps to make the corresponding
GCBytesAge adjustment. Finally, it ensures that GCBytesAge
is accrued from the right timestamps in cases of sst keys
/ tombstones shadowing engine keys / tombstones.

Note that this change only fixes GCBytesAge for point keys;
a fix for range keys will come in a follow-up.

Fixes #82920.

Release note: None.

Release justification:
  • Loading branch information
itsbilal committed Sep 8, 2022
1 parent d33e93f commit 4f3d318
Show file tree
Hide file tree
Showing 4 changed files with 207 additions and 100 deletions.
4 changes: 3 additions & 1 deletion pkg/kv/kvserver/batcheval/cmd_add_sstable.go
Original file line number Diff line number Diff line change
Expand Up @@ -173,11 +173,12 @@ func EvalAddSSTable(
// request timestamp. This ensures the writes comply with the timestamp cache
// and closed timestamp, i.e. by not writing to timestamps that have already
// been observed or closed.
var sstReqStatsDelta enginepb.MVCCStats
if sstToReqTS.IsSet() && (h.Timestamp != sstToReqTS || forceRewrite) {
st := cArgs.EvalCtx.ClusterSettings()
// TODO(dt): use a quotapool.
conc := int(AddSSTableRewriteConcurrency.Get(&cArgs.EvalCtx.ClusterSettings().SV))
sst, err = storage.UpdateSSTTimestamps(ctx, st, sst, sstToReqTS, h.Timestamp, conc)
sst, sstReqStatsDelta, err = storage.UpdateSSTTimestamps(ctx, st, sst, sstToReqTS, h.Timestamp, conc, args.MVCCStats)
if err != nil {
return result.Result{}, errors.Wrap(err, "updating SST timestamps")
}
Expand Down Expand Up @@ -219,6 +220,7 @@ func EvalAddSSTable(
args.Key, args.EndKey, desc.StartKey.AsRawKey(), desc.EndKey.AsRawKey())
statsDelta, err = storage.CheckSSTConflicts(ctx, sst, readWriter, start, end, leftPeekBound, rightPeekBound,
args.DisallowShadowing, args.DisallowShadowingBelow, maxIntents, usePrefixSeek)
statsDelta.Add(sstReqStatsDelta)
if err != nil {
return result.Result{}, errors.Wrap(err, "checking for key collisions")
}
Expand Down
Loading

0 comments on commit 4f3d318

Please sign in to comment.