storage: export info in pebble.InternalIntervalMetrics as CockroachDB metrics

[sumeerbhola]

These would help us understand bottlenecks and slowness inside Pebble. Specifically, I think it would be useful to have

• fsync latency: our only current metric is higher-level (raft.process.logcommit.latency) and does not give us a direct view into the fsync latency Pebble is observing.
• Flush utilization: This can be computed as ThroughputMetric's WorkDuration/(WorkDuration+IdleDuration) and will give us a better understanding of write stalls due to high memtable count.

There is additional info in InternalIntervalMetrics that may also be helpful, like the various LogWriter utilization values, though I don't know of any tickets where they could have been helpful.

type InternalIntervalMetrics struct {
	// LogWriter metrics.
	LogWriter struct {
		// WriteThroughput is the WAL throughput.
		WriteThroughput ThroughputMetric
		// PendingBufferUtilization is the utilization of the WAL writer's
		// finite-sized pending blocks buffer. It provides an additional signal
		// regarding how close to "full" the WAL writer is. The value is in the
		// interval [0,1].
		PendingBufferUtilization float64
		// SyncQueueUtilization is the utilization of the WAL writer's
		// finite-sized queue of work that is waiting to sync. The value is in the
		// interval [0,1].
		SyncQueueUtilization float64
		// SyncLatencyMicros is a distribution of the fsync latency observed by
		// the WAL writer. It can be nil if there were no fsyncs.
		SyncLatencyMicros *hdrhistogram.Histogram
	}
	// Flush loop metrics.
	Flush struct {
		// WriteThroughput is the flushing throughput.
		WriteThroughput ThroughputMetric
	}
	// NB: the LogWriter throughput and the Flush throughput are not directly
	// comparable because the former does not compress, unlike the latter.
}

Jira issue: CRDB-18424

Epic CRDB-20293

[jbowens]

Eyeing this as a starter project for Leon.

[coolcom200]

Closing as #90082 and #89459 are both merged.