Skip to content

Commit

Permalink
storage: reorder EventListeners
Browse files Browse the repository at this point in the history
To be defensive, sequence the EventListener responsible for crashing the
process during a disk stall first, before the Pebble logging event listener.

Informs #94373.
Epic: None
Release note: None
  • Loading branch information
jbowens committed Jan 25, 2023
1 parent 6fc1022 commit e40bd52
Showing 1 changed file with 7 additions and 1 deletion.
8 changes: 7 additions & 1 deletion pkg/storage/pebble.go
Original file line number Diff line number Diff line change
Expand Up @@ -981,12 +981,18 @@ func NewPebble(ctx context.Context, cfg PebbleConfig) (p *Pebble, err error) {
return int(atomic.LoadUint64(&p.atomic.compactionConcurrency))
}

// NB: The ordering of the event listeners passed to TeeEventListener is
// deliberate. The listener returned by makeMetricEtcEventListener is
// responsible for crashing the process if a DiskSlow event indicates the
// disk is stalled. While the logging subsystem should also be robust to
// stalls and crash the process if unable to write logs, there's less risk
// to sequencing the crashing listener first.
el := pebble.TeeEventListener(
p.makeMetricEtcEventListener(ctx),
pebble.MakeLoggingEventListener(pebbleLogger{
ctx: logCtx,
depth: 2, // skip over the EventListener stack frame
}),
p.makeMetricEtcEventListener(ctx),
)

p.eventListener = &el
Expand Down

0 comments on commit e40bd52

Please sign in to comment.