Skip to content

Commit

Permalink
rangefeed: improve memory accounting for event queue
Browse files Browse the repository at this point in the history
To prevent OOMs, we previously introduced memory budget to limit event buffering
to processor.eventC channel. These events could point to large underlying data
structure and keeping them in the channel prevent them from being garbage
collected. However, the current memory accounting did not have a good way to
predict the memory footprint of events. It uses calculateDateSize which 1. does
not account for memory of the new RangeFeedEvents that could be spawned (such as
checkpoint events) 2. inaccurately uses protobuf-encoded wire size for data 3.
does not account for memory of the base structs.

This patch improves the memory estimation by resolving the three points above.
It tries its best effort to account for base struct's memory, predict generation
of new events, and use actual data size rather than compressed protobuf size.
Since it is challenging to predict whether there would be a new checkpoint
event, our strategy in this PR is over-accounting >> under-accounting. GitHub
issue (todo) tracks the remaining work to improve accuracy.

Resolves: #114190
Release note: None
  • Loading branch information
wenyihu6 committed Mar 13, 2024
1 parent 2d9ca1e commit a11824c
Show file tree
Hide file tree
Showing 6 changed files with 1,056 additions and 17 deletions.
7 changes: 7 additions & 0 deletions pkg/kv/kvserver/rangefeed/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go_library(
srcs = [
"budget.go",
"catchup_scan.go",
"event_size.go",
"filter.go",
"metrics.go",
"processor.go",
Expand Down Expand Up @@ -56,6 +57,7 @@ go_test(
"budget_test.go",
"catchup_scan_bench_test.go",
"catchup_scan_test.go",
"event_size_test.go",
"processor_test.go",
"registry_test.go",
"resolved_timestamp_test.go",
Expand All @@ -71,11 +73,16 @@ go_test(
"//pkg/kv/kvserver/concurrency/lock",
"//pkg/roachpb",
"//pkg/settings/cluster",
"//pkg/sql/randgen",
"//pkg/sql/rowenc/keyside",
"//pkg/sql/sem/tree",
"//pkg/sql/types",
"//pkg/storage",
"//pkg/storage/enginepb",
"//pkg/storage/fs",
"//pkg/testutils",
"//pkg/testutils/skip",
"//pkg/testutils/storageutils",
"//pkg/util",
"//pkg/util/encoding",
"//pkg/util/future",
Expand Down
Loading

0 comments on commit a11824c

Please sign in to comment.