perf: remove extra metrics

[brianluong]

Description

• Removing the URL label as I think it's unnecessary. upstream_id is descriptive enough
• Remove the total metrics - we can use duration.count instead
• Rename the label endpoint_id to upstream_id for consistency

Type of change

• 🐛 Bug fix (non-breaking change which fixes an issue)
• 😎 New feature (non-breaking change which adds functionality)
• ⁉️ Breaking change (fix or feature that would cause existing functionality to not work as expected)
• ⚒️ Refactor (no functional changes)
• 📖 Documentation (updating or adding docs)

How Has This Been Tested?

Ran it and seeing metrics in data dog

[dianwen]

Is there any downside to keeping the URL? It might be useful if we're replacing a node but want to keep the same ID. No strong opinions.

Are there any concerns with using count on the duration metric? A theoretical concern: datadog or the client could sample data points for distribution-type metrics under the hood. Although it's a bit redundant, it might be more readable / clear to have separate metrics for count and duration. I'm not concerned about the minor redundancy.

[brianluong]

Is there any downside to keeping the URL? It might be useful if we're replacing a node but want to keep the same ID. No strong opinions.

Are there any concerns with using count on the duration metric? A theoretical concern: datadog or the client could sample data points for distribution-type metrics under the hood. Although it's a bit redundant, it might be more readable / clear to have separate metrics for count and duration. I'm not concerned about the minor redundancy.

The motivation behind this was costs. About the replacements - that shouldn't be a very common event. I was warned by @kevin-satsuma that DD custom metrics are expensive.

On the sampling - I haven't seen any docs that indicate sampling happens. I graphed a few duration.count and requests.count and they were the same. I can move it add it back now and remove it if we need to cut costs later.

[dianwen]

Gotcha. I think they charge by existence of combination of tags, so if we have a 1:1 mapping of URL to ID, then it's the same cardinality / cost as having only ID.

[dianwen]

No strong opinion on the URL. Up to you!

[brianluong]

Oh good point on the unique combinations https://docs.datadoghq.com/account_management/billing/custom_metrics/?tab=countrate

A custom metric is uniquely identified by a combination of a metric name and tag values (including the host tag).