rangefeed: improve memory accounting for event queue

To prevent OOMs, we previously introduced memory budget to limit event buffering
to processor.eventC channel. These events could point to large underlying data
structure and keeping them in the channel prevent them from being garbage
collected. However, the current memory accounting did not have a good way to
predict the memory footprint of events. It uses calculateDateSize which 1. does
not account for memory of the new RangeFeedEvents that could be spawned (such as
checkpoint events) 2. inaccurately uses protobuf-encoded wire size for data 3.
does not account for memory of the base structs.

This patch improves the memory estimation by resolving the three points above.
It tries its best effort to account for base struct's memory, predict generation
of new events, and use actual data size rather than compressed protobuf size.
Since it is challenging to predict whether there would be a new checkpoint
event, our strategy in this PR is over-accounting >> under-accounting. GitHub
issue (todo) tracks the remaining work to improve accuracy.

Resolves: cockroachdb#114190
Release note: None

pkg/kv/kvserver/rangefeed/budget.go

@@ -266,6 +266,31 @@ func (a *SharedBudgetAllocation) Release(ctx context.Context) {
 	}
 }
 
+// ReleaseAmount releases amount to the budget feed if amount <= what the
+// allocation token holds. Otherwise, it releases what it has and logs about
+// memory accounting inaccuracy.
+func (a *SharedBudgetAllocation) ReleaseAmount(ctx context.Context, amount int64) {
+	if a != nil {
+		newSize := atomic.AddInt64(&a.size, -amount) // newSize := oldSize - amount
+		if newSize >= 0 {
+			// if newSize >= 0, release amount
+			a.feed.returnAllocation(ctx, amount)
+			return
+		}
+
+		if oldSize := newSize + amount; oldSize > 0 {
+			// if newSize < 0, release oldSize(newSize+amount) if oldSize > 0
+			a.feed.returnAllocation(ctx, oldSize)
+			log.VErrorf(ctx, 4, "excess memory usage: %d bytes more than accounted", -newSize)
+		} else {
+			// releasing more memory than allocation token budget: warn about memory
+			// accounting inaccuracy
+			log.VErrorf(ctx, 4, "excess memory usage: %d bytes more than accounted", amount)
+		}
+
+	}
+}
+
 // BudgetFactory creates memory budget for rangefeed according to system
 // settings.
 type BudgetFactory struct {