roachtest: enable kv/gracefuldraining/nodes=3

[kvoli]

This commit re-enables the kv/gracefuldraining/nodes=3 roachtest. The
test is still likely to fail occasionally however has produced
interesting findings just in testing to re-enable.

Informs: #59094

Release note: None

[cockroach-teamcity]

This change is 

[andrewbaptist]

I'm confused why you switched from the admin ts stats to Prometheus stats. I was hoping with https://grafana.testeng.crdb.io/ set up, we could stop launching separate Prometheus instances with each test, but this change makes that harder to do.

[kvoli]

I'm confused why you switched from the admin ts stats to Prometheus stats. I was hoping with grafana.testeng.crdb.io set up, we could stop launching separate Prometheus instances with each test, but this change makes that harder to do.

Discussed offline. I updated this commit just to re-enable the test. I was originally very suspicious of the internal timeseries metrics possibly running into an erroneous windowing issue, which could be dealt with separately.

However, upon double checking the debug stuff I had, I realized the drop in QPS was legitimate and showed up in the workload runner (I looked at a different run initially).

I think we should re-enable the test as is. It has already produced useful findings.

[andrewbaptist]

LGTM - We might have to disable again, but we will at least get a chance to see if things are better.

[kvoli]

@andrewbaptist do you think now's a good time to re-enable? It will likely flake in a couple/few weeks but could also find interesting regressions.

[kvoli]

Going to re-enable this now, I'm anticipating some possible failures in the future but it will be good to start getting coverage again.

bors r=andrewbaptist

[craig]

Build succeeded:

• Bazel Essential CI (Cockroach)