Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

services/horizon: Add Go and process related metrics #3103

Merged
merged 2 commits into from
Oct 8, 2020

Conversation

bartekn
Copy link
Contributor

@bartekn bartekn commented Oct 7, 2020

PR Checklist

PR Structure

  • This PR has reasonably narrow scope (if not, break it down into smaller PRs).
  • This PR avoids mixing refactoring changes with feature changes (split into two PRs
    otherwise).
  • This PR's title starts with name of package that is most changed in the PR, ex.
    services/friendbot, or all or doc if the changes are broad or impact many
    packages.

Thoroughness

  • This PR adds tests for the most critical parts of the new functionality or fixes.
  • I've updated any docs (developer docs, .md
    files, etc... affected by this change). Take a look in the docs folder for a given service,
    like this one.

Release planning

  • I've updated the relevant CHANGELOG (here for Horizon) if
    needed with deprecations, added features, breaking changes, and DB schema changes.
  • I've decided if this PR requires a new major/minor version according to
    semver, or if it's mainly a patch change. The PR is targeted at the next
    release branch if it's not a patch change.

What

Adds Go and process related metrics provided by Prometheus package.

Why

Extra metrics can be useful to debug performance issues.

Known limitations

  • Process metrics are available on operating systems with a Linux-style proc filesystem and on Microsoft Windows.

@bartekn bartekn requested a review from a team October 7, 2020 17:12
@cla-bot cla-bot bot added the cla: yes label Oct 7, 2020
// initGoMetrics registers the Go collector provided by prometheus package which
// includes Go-related metrics.
func initGoMetrics(app *App) {
app.prometheusRegistry.MustRegister(prometheus.NewGoCollector())
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jacekn this will render metrics without horizon_ prefix, like: go_memstats_buck_hash_sys_bytes. Is it OK?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think no namespace for the general metrics including the processor ones below is what you should go with. See my comment below.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@bartekn yeah go_memstats_buck_hash_sys_bytes is absolutely fine, that's how most projects I know, including prometheus itself, export go metrics.

app.prometheusRegistry.MustRegister(
prometheus.NewProcessCollector(
prometheus.ProcessCollectorOpts{
Namespace: "horizon",
Copy link
Member

@leighmcculloch leighmcculloch Oct 7, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure you want to use the namespace on the processor collector. I think with the webauth and recoverysigner services we landed on only using namespaces for metrics that are truly uniquely defined by that service. General metrics should otherwise have no namespace so that the same metric that means the same thing has the same name across all services in a deployment.

Maybe this logic doesn't apply to Horizon's use case though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this makes sense. It's just not clear to me why setting a prefix is allowed here and not in GoCollector.

Copy link
Contributor

@jacekn jacekn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@bartekn bartekn merged commit acd2b81 into stellar:release-horizon-v1.10.0 Oct 8, 2020
@bartekn bartekn deleted the go-process-metrics branch October 8, 2020 10:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants