kvserver: improved lease protocol

[erikgrinaker]

This is mostly a placeholder issue, we haven't hashed out a concrete plan yet.

We want the properties of expiration-based leases, but without the cost of a replicated write per range every 3 seconds, and without the cost of ticking/heartbeating Raft ranges. In particular, we want to ensure the lease is functional (can process reads and replicated writes) and that it eagerly acquires leases.

One way to do this is to pull both lease/leader elections and failure detection out of Raft entirely. We would do lease elections at a higher level, with lease votes and expiration possibly stored in memory (mandating waiting out the expiration time on reboot), and instruct Raft who the leader should be (subject to the leader completeness requirement). This would then also take the place of Raft heartbeating and ticking.

See rough internal proposal.

Jira issue: CRDB-28909

Epic CRDB-34218

[erikgrinaker]

If leaseholder election is no longer dependant on a Raft leader, and we mandate lease/leader colocation, then we could also significantly improve (or get rid of, rather) reproposals: #103908 (comment)

[nvanbenschoten]

Folding into #123847, now that we're getting more clarity on where we want to go.