*: introduce pkg/migrations

[irfansharif]

Implements the long-running migrations RFC (#48843). This PR builds on top a
few core components we've introduced to crdb over the past few months. They
are (in order):

• clusterversion: significantly rework cluster version handling #45455 (re-works cluster version settings to be a standalone component)
• kvserver,cli,roachtest,sql: introduce a fully decommissioned bit #50329 (introduces a fully decommissioned bit)
• server: introduce join rpc for node id allocation #52526 (introduces the join RPC for node ID allocation, and to hand out
  active cluster versions to joining nodes)
• server: always create a liveness record before starting up #53842 (always creating a liveness record for nodes, before startup)
• server,rpc: block decommissioned node from interacting with the cluster #55286 (blocks decommissioned nodes from interacting with the cluster)

The components above (save for #55286) were included as part of the 20.2
release. That paved the way for us to introduce this machinery for 21.1 (it
also lets us define migrations starting this cycle). For the detailed story
around how it all fits together, read the RFC.

---

Big picture: going forward SET CLUSTER SETTING version = <whatever> will step
through each intermediate (unstable) version between what's currently active,
and what the intended target is, and execute associated migrations for each
one (if any). For each "step", the migration manager first checks to see that
every node in the cluster is running an appropriate binary version. It then
activates the relevant version gate on every node (through a new RPC, instead
of gossip like we used to previously). Finally it executes the migration
itself, providing it with the guarantee that all .IsActive(VersionTag) will
return true going forward. After successfully migrating through each
intermediate version, the statement returns.

---

Package migration captures the facilities needed to define and execute
migrations for a crdb cluster. These migrations can be long running, are free
to send out arbitrary requests cluster wide, change internal DB state, and much
more. They're typically reserved for crdb internal operations and state. Each
migration is idempotent in nature, is associated with a specific cluster
version, and executed when the cluster version is made activate on every node
in the cluster.

Examples of migrations that apply would be migrations to move all raft
state from one storage engine to another, or purging all usage of the
replicated truncated state in KV.

---

To get a sense of the primitives made available to migrations, a rough sketch
of the APIs is presented below.

  type Migration func(context.Context, *Helper) error
  func ExampleMigration (ctx context.Context, h *Helper) error {
    // NB: The harness has already made sure that the version associated with
    // this particular migration this is already active on all nodes in the
    // cluster.

    // Migrations have access to a *kv.DB.
    txn := h.DB().NewTxn(ctx, ...)

    // Migrations can run for arbitrarily long.
    time.Sleep(time.Second)

    // Migrations can run SQL.
    _, err := h.Conn().Queryf(`SELECT 1`)

    // Migrations can paginate through each range descriptor.
    _ = h.IterRangeDescriptors(pageSize, func(descriptors ...roachpb.RangeDescriptor) error {
      for _, desc := range descriptors {
        // Migrations can selective execute below-raft migrations.
        h.DB().Migrate(ctx, desc.StartKey, desc.EndKey, h.Version())
      }

      // Migrations can persist progress to a system table, for introspection.
      h.Logf("paginated through %d descriptors", len(descriptors))
    })

    // Migrations can define/execute server level operations across every node
    // (like forcing all replicas through GC, etc.)
    h.EveryNode(ctx, op)
  }

---

See individual commits. This PR introduces individual components without it
being wired up to anything first, and then slowly incorporates them into
actual running code. The commits of note (in order) are pulled out below.

  sql: add scaffolding for version upgrade hook
  
  This callback will be called after validating a `SET CLUSTER SETTING
  version` but before executing it. It will be used in future commits to
  execute long-running migrations.

  server,kvserver: introduce the `EveryNode` rpc
  
  ...and populate it with the `AckClusterVersion` op.
  
  EveryNode is the RPC that will powers the long running migrations
  infrastructure. It'll let callers define and execute arbitrary commands
  across every node in the cluster. To motivate what this would look like,
  we introduce alongside it one such command, the `AckClusterVersion`
  operation. It isn't currently hooked up to anything, but this will
  eventually be the mechanism through which we'll propagate cluster
  version bumps across the crdb cluster, replacing our gossip based
  distribution mechanism in-place today. This will let the orchestrator
  bump version gates in a controlled fashion across the cluster.
  
  To achieve this, the stubbed out implementation of AckClusterVersion
  makes use of the same `StoreClusterVersionKey` otherwise used in
  callbacks attached to the gossip handler for cluster version bumps.

  kvserver: introduce GetLivenessesFromKV
  
  Now that we always create a liveness record on start up (#53805), we can
  simply fetch all liveness records from KV when wanting an up-to-date
  view of all nodes in the cluster. We add a helper to do as much,
  which we'll rely on in future commits. 

  It's a bit unfortunate that we're further adding on to the NodeLiveness API
  without changing the underlying look-aside caching structure, but the
  methods fetching records from KV is the world we're hoping to start moving
  towards over time.

  migration: introduce pkg/migrations
  
  Package migration captures the facilities needed to define and execute
  migrations for a crdb cluster. These migrations can be arbitrarily long
  running, are free to send out arbitrary requests cluster wide, change
  internal DB state, and much more. They're typically reserved for crdb
  internal operations and state. Each migration is idempotent in nature,
  is associated with a specific cluster version, and executed when the
  cluster version is made activate on every node in the cluster.
  
  Examples of migrations that apply would be migrations to move all raft
  state from one storage engine to another, or purging all usage of the
  replicated truncated state in KV. A "sister" package of interest is
  pkg/sqlmigrations.
  
  This commit only introduces the basic scaffolding and wiring from
  existing functionality. We'll flesh in the missing bits in future
  commits.

  server,migration: generate EveryNode req/resp helpers
  
  We expect to add multiple req/resp types as individual operations for
  the EveryNode RPC. Each of these operations will correspond to a
  "primitive" of sorts for the (long running) migrations infrastructure.
  It's a bit cumbersome to wield this nested union type, so we
  autogenerate helper code to do it for us. We take precedence from
  api.proto and all the very many batch request/responses.

  server: expose GetLivenessesFromKV in node liveness interfaces
  
  To flesh out the migrations infrastructure (in future commits), we'll
  need a handle on all the nodes present in the system. Now that we always
  create a liveness record on start up (#53805), we can simply fetch all
  liveness records from KV. We add a helper to do so.
  
  This does mean that we're introducing a direct dependency on
  NodeLiveness in the sql layer, and there's improvements to be made here
  around interfaces delineating between "system tenant only" sql code and
  everything else. Only system tenants have the privilege to set cluster
  settings (or at least the version setting specifically), which is what
  this API will look to power.

  server: introduce ValidateTargetClusterVersionRequest
  
  We'll use this primitive in a future commit to introduce additional
  safeguards (not present today) around cluster version upgrades.
  Specifically, we'll use this EveryNode operation to validate that every
  node in the cluster is running a binary that's able to support the
  specified cluster version.

  settings,clusterversion: disconnect cluster version from gossip
  
  ...in favor of direct RPCs to all nodes in the cluster. This commit in
  particular deserves a bit of scrutiny. It uses the building blocks we've
  added thus far to replace the use of gossip to disseminate the cluster
  version. It does so by sending out individual RPCs to each node in the
  cluster, informing them of a version bump, all the while retaining the
  same guarantees provided by our (now previously) gossip-backed mechanism.
  
  This diff has the following "pieces":
  - It disconnecting version setting updates through gossip (by
    disconnecting the setting type within the updater process)
  - It using the EveryNode RPC to send out RPCs to each node in the
    cluster, containing the payload that each node would otherwise receive
    through gossip.
  - It expands the clusterversion.Handle interface to allow setting the
    active version directly through it.
  - It persisting any cluster versions received from other nodes first,
    within keys.StoreClusterVersionKey, before bumping the version gate.
    This was previously achieved by attaching callbacks on the version
    handle (look towards all the diffs around SetBeforeChange). This is an
    invariant we also maintained earlier.
  - It using the active version provided by the join RPC to set the
    version setting directly (after having persisted it first). This too
    was previously achieved through gossip + the callback.
  
  While here, we add a few comments and chip away at the callback hooks
  that are no longer needed.

  sql: introduce system.migrations
  
  We'll use it in a future commit to store migration state, for
  introspection.
  
  > SHOW CREATE system.migrations;
           table_name        |                     create_statement
  ---------------------------+------------------------------------------------------------
    system.public.migrations | CREATE TABLE public.migrations (
                             |     id INT8 NOT NULL DEFAULT unique_rowid(),
                             |     metadata STRING NOT NULL,
                             |     started TIMESTAMP NOT NULL DEFAULT now():::TIMESTAMP,
                             |     progress BYTES NULL,
                             |     CONSTRAINT "primary" PRIMARY KEY (id ASC),
                             |     FAMILY "primary" (id, metadata, started),
                             |     FAMILY progress (progress)
                             | )

---

Still need to add tests. But it "works".

[cockroach-teamcity]

This change is 

[irfansharif]

This prototype was shredded up into all the various PRs that have since landed on master (see the referring commits above). The few comments here were addressed in those other PRs. The last in-flight one is #57445, which moves past what was prototyped here, so closing this one out. Thanks for the reviews!