Skip to content

Commit

Permalink
GC: Consider size of value while rewriting (#1357)
Browse files Browse the repository at this point in the history
The existing code doesn't consider the size of the value while
calculating the size of the entry batch in rewrite method.
This PR fixes it.

Fixes #1292
  • Loading branch information
Ibrahim Jarif committed Oct 2, 2020
1 parent 5d84f93 commit 9ae6f6c
Showing 1 changed file with 5 additions and 0 deletions.
5 changes: 5 additions & 0 deletions value.go
Original file line number Diff line number Diff line change
Expand Up @@ -564,6 +564,11 @@ func (vlog *valueLog) rewrite(f *logFile, tr trace.Trace) error {

ne.Value = append([]byte{}, e.Value...)
es := int64(ne.estimateSize(vlog.opt.ValueThreshold))
// Consider size of value as well while considering the total size
// of the batch. There have been reports of high memory usage in
// rewrite because we don't consider the value size. See #1292.
es += int64(len(e.Value))

// Ensure length and size of wb is within transaction limits.
if int64(len(wb)+1) >= vlog.opt.maxBatchCount ||
size+es >= vlog.opt.maxBatchSize {
Expand Down

0 comments on commit 9ae6f6c

Please sign in to comment.