release-23.1: server: add node_id label to _status/vars output

[blathers-crl]

Backport 1/1 commits from #98640 on behalf of @dhartunian.

/cc @cockroachdb/release

---

Previously, the output of the prometheus metrics via _status/ vars did not include any node labels. This caused challenges for customers who want to monitor large clusters as it requires additional configuration on the scrape- side to ensure a node ID is added to the metrics. This can be challenging to deal with when nodes come and go in a cluster and the scrape configuration must change as well.

This change adds a node_id prometheus label to the metrics we output that matches the current node's ID. Since _status/vars is output from a single node there is only ever one single value that's appropriate here.

Secondary tenants will mark their metrics with either the nodeID of the shared- process system tenant, or the instanceID of the tenant process.

Resolves: #94763
Epic: None

Release note (ops change): Prometheus metrics available at the _status/vars path now contain a node_id label that identifies the node they were scraped from.

---

Release justification: low-risk high impact addition of a feature that should be in the dot zero.

[blathers-crl]

Thanks for opening a backport.

Please check the backport criteria before merging:

• Patches should only be created for serious issues or test-only changes.
• Patches should not break backwards-compatibility.
• Patches should change as little code as possible.
• Patches should not change on-disk formats or node communication protocols.
• Patches should not add new functionality.
• Patches must not add, edit, or otherwise modify cluster versions; or add version gates.

If some of the basic criteria cannot be satisfied, ensure that the exceptional criteria are satisfied within.

• There is a high priority need for the functionality that cannot wait until the next release and is difficult to address in another way.
• The new functionality is additive-only and only runs for clusters which have specifically “opted in” to it (e.g. by a cluster setting).
• New code is protected by a conditional check that is trivial to verify and ensures that it only runs for opt-in clusters.
• The PM and TL on the team that owns the changed code have signed off that the change obeys the above rules.

Add a brief release justification to the body of your PR to justify this backport.

Some other things to consider:

• What did we do to ensure that a user that doesn’t know & care about this backport, has no idea that it happened?
• Will this work in a cluster of mixed patch versions? Did we test that?
• If a user upgrades a patch version, uses this feature, and then downgrades, what happens?

[cockroach-teamcity]

This change is 

[dhartunian]

This change has been requested for a long time and will improve debugging quite a bit for anyone who scrapes metrics off of nodes and wants to remember what the node ID was without having to do additional work. Additionally, our 3rd party metrics integrations have been asking for this to improve their ingest process as well.

[abarganier]

CI Failure appears to be a flake/unrelated.

Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @knz)