raft: separate log and state storage logically

[pav-kv]

In service of the separate raft log [#16624] and witness projects, it appears that the best results can be achieved with the support from the raft package. While the log and state storage in CRDB physically sit in the same storage engine, it is possible to start separating them logically in raft, and introducing assumptions that the two can work asynchronously.

When two storages are separated, the model becomes:

• The log storage supports all the functions of an "acceptor" (voting+log). It requires regular fsyncs so that entries accepted into it can be committed at the "proposer"/"learner" level (when a sufficient quorum is collected).
• The state storage backs the "learner" concept, and is updated with committed entries and snapshots. It does not require eager fsyncs, except in some limited circumstances (TBD; might include things like config changes). It still has to do fsyncs somewhat regularly so that the RawNode can report the durable applied state, which gets input into the log compaction decisions, and prevents long apply catchups upon restart.
• Due to the physical separation and fsync asynchrony between the two storages, it may happen that they are out of sync (one "happens before" the other, or vice versa) when the RawNode restarts. It must be possible to reconciliate a correct in-memory raft state from the initial state read from both storages.

To support the above, both storages have to provide logical clocks (HardState-style) that make it possible to compare the two states. Initially, both can be sourced from the unified HardState that we have today (which means they will always be in sync), and eventually they can become asynchronous at the physical level.

Doing the logical separation first gives the benefit of being able to test it extensively in raft datadriven tests long before the actual physical separation happens. The CRDB-specific aspects of the physical separation would be added on top.

Jira issue: CRDB-42792