Error registering conflicting prom metrics during startup

[mmedvede]

I see a few "msg":"register metric error","error":"a previously registered descriptor with the same fully-qualified name as Desc show up during M3Coordinator, M3DB, and M3Aggregator startup.

Running v0.15.0-rc.9.

The metrics it complains about:

aggregator
 m3aggregator_aggregator_client_invalid_write counter
 m3aggregator_aggregator_election_manager_campaign_errors counter
 m3aggregator_aggregator_election_manager_resign_errors counter
 m3aggregator_aggregator_flush_handler_invalid_write counter
 m3aggregator_aggregator_flush_times_manager_flush_times_persist_errors counter
 m3aggregator_aggregator_flush_times_manager_flush_times_persist_errors_latency histogram
 m3aggregator_aggregator_flush_times_manager_flush_times_persist_success_latency histogram
 m3aggregator_aggregator_list_writer_errors counter

coordinator
 coordinator_downsampler_remote_aggregator_client_invalid_write counter
 coordinator_fetch_errors counter
 coordinator_fetch_success counter
 coordinator_fetch_success_latency summary
 coordinator_max_datapoints gauge

dbnode
 coordinator_fetch_errors counter
 coordinator_fetch_success counter
 coordinator_fetch_success_latency summary
 coordinator_max_datapoints gauge

Both coordinators and aggregators are using m3msg protocol.

Startup logs:

m3aggr-start.log
m3coord-start.log
m3dbnode-start.log

[robskillington]

@linasm is going to take a look at this, thanks @mmedvede .

[linasm]

@mmedvede you should see an improvement with v0.15.2 (in m3dbnode and m3coordinator, but not in m3aggregator yet).

[linasm]

@mmedvede #2443 should fix all the metrics conflicts in the aggregator.
There is still coordinator_downsampler_remote_aggregator_client_invalid_write to be fixed in the coordinator.

[linasm]

Turns out I was wrong, but in a good way: I see that #2443 also fixed coordinator_downsampler_remote_aggregator_client_invalid_write.

[gibbscullen]

As an update, we are continuing to work on this.

[linasm]

Fixed via #2406 and #2443.