Initialize Atlantis metrics with zeroes

[tchelovilar]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request. Searching for pre-existing feature requests helps us consolidate datapoints for identical requirements into a single place, thank you!
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.
• If you are interested in working on this issue or have submitted a pull request, please leave a comment.

---

• I'd be willing to implement this feature (contributing guide)

Describe the user story

Atlantis doesn't initialize the metrics after starting only after receiving some event, which causes wrong query results from Prometheus when using functions like rate or increase.

Describe the solution you'd like

My request is about pre-populate the Atlantis metrics like atlantis_cmd_comment_apply_builder_execution_success with zeroes after the process starts then the Prometheus should have the proper metric history. This is important special for error metrics to set up alerts.

[albertorm95]

I was debugging the metrics scopes when adding tags to the metrics, and looking at the way the scopes are being created I don't think init them with 0 will be easy task, you will need to execute code at least once in order to get to that part of the code where the metrics is being registered, at the beginning of Atlantis server Tally (the package use for metrics) does not know all the possible metrics.

[nitrocode]

A slack message we received also seems related to this. I'll add it here so it comes up on searches.

altantis metrics returns NAN as the value of a all the quantiles
atlantis_github_hide_prev_plan_comments_execution_time{quantile="0.999"} NaN

[Fabianoshz]

Did some testing here: #2767

And got theses results upon starting the server:

I'm unsure that adding the function NewInstrumentedProjectCommandRunner is the way and even if these functions should be used to initialize data like a constructor. By no means consider this a final implementation, I've just scrambled some code to help my friend @tchelovilar.