kvclient: fix rangefeed retry reason counters

[stevendanna]

Previously, retriable errors would often be recorded in the wrong
retry counter because the code that initialised the metrics assumed a
stable ordering when iterating a map.

Given the amount of work that is about to happen when restarting a
rangefeed, I doubt the optimization of a slice lookup vs a map lookup
is that important here.

Further, we almost surely do not want to panic just because we
couldn't find a metric for the reason a node might have sent to us, so
now rather than panic'ing we have a fallback counter for unknown error
types.

Epic: none

Release note: Fix bug that could result in incorrect metrics related
to retriable rangefeed errors.

[cockroach-teamcity]

This change is 

[wenyihu6]

Rangefeed fix LGTM. I'm not too familiar with registry metrics package - I gave it a few reads and it seems good overall. I ran a roachtest and confirmed that all retry metrics are populated just in case.

[stevendanna]

bors r+

[craig]

Build succeeded:

• acceptance
• macos_amd64_build
• linux_amd64_fips_build
• windows_build
• examples_orms
• linux_arm64_build
• linux_amd64_build
• lint
• local_roachtest_fips
• docker_image_amd64
• macos_arm64_build
• unit_tests
• check_generated_code
• local_roachtest

[blathers-crl]

Encountered an error creating backports. Some common things that can go wrong:

1. The backport branch might have already existed.
2. There was a merge conflict.
3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.

---

error creating merge commit from d772ac5 to blathers/backport-release-23.2-133991: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 23.2.x failed. See errors above.

---

error creating merge commit from d772ac5 to blathers/backport-release-24.1-133991: POST https://api.github.com/repos/cockroachdb/cockroach/merges: 409 Merge conflict []

you may need to manually resolve merge conflicts with the backport tool.

Backport to branch 24.1.x failed. See errors above.

---

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.}