WIP: feat(prometheus): add prometheus endpoint providing basic request met… #404
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ecktom
force-pushed
the
feat/add_prometheus
branch
from
b92a8ed to
b3c5b21
Compare
aeneasr
reviewed
Apr 10, 2020
cmd/server/server.go
Outdated
| } | ||
|
|
||
| // TODO: Make prom settings (port/path) configurable | ||
| promAddr := ":9124" |
There was a problem hiding this comment.
I think we can simply export this at the admin api endpoint :)
There was a problem hiding this comment.
Ehm, what admin api endpoint? :D
|
Sorry, I forgot to press on "submit review" :) |
|
So I'm not sure what you mean regarding the config but I've build the version from the PR locally and tested it by curling the scraping endpoint which looks good: |
Now that I see your TODO I don't understand you comment :D |
ecktom
force-pushed
the
feat/add_prometheus
branch
from
b3c5b21 to
58a499e
Compare
aeneasr
reviewed
May 1, 2020
There was a problem hiding this comment.
This looks pretty good to me! I've found a couple of things that could be improved. Also, would you be able to add a small section at the bottom of https://github.com/ory/oathkeeper/blob/master/docs/docs/configure-deploy.md#authorizing-requests on setting up ORY Oathkeeper with prometheus (optionally grafana if it's easy to do).
metrics/prometheus.go
Outdated
| } | ||
|
|
||
| // NewPrometheusRepository creates a new prometheus repository with the given settings | ||
| func NewPrometheusRepository(logger log.FieldLogger) (*PrometheusRepository, error) { |
There was a problem hiding this comment.
No error is returned so is the error return value required?
cmd/server/server.go
Outdated
| promAddr := d.Configuration().PrometheusServeAddress() | ||
| go func() { | ||
| // Expose the registered metrics via HTTP. | ||
| httpServer := &http.Server{Handler: promhttp.HandlerFor(prometheusRepo.Registry, promhttp.HandlerOpts{}), Addr: promAddr} |
There was a problem hiding this comment.
prometheusRepo.Registry will panic when prometheusRepo, err := metrics.NewPrometheusRepository(logger) returns an error!
There was a problem hiding this comment.
yeah right, I removed the returning error is it did not make sense anyways...
cmd/server/server.go
Outdated
|
|
||
| promPath := d.Configuration().PrometheusMetricsPath() | ||
| promAddr := d.Configuration().PrometheusServeAddress() | ||
| go func() { |
There was a problem hiding this comment.
You could add this to the tasks listed below: https://github.com/ory/oathkeeper/pull/404/files#diff-4c51d95b26718af4a2ac3bec33e54e49R220
Because it makes sense to use a waitgroup for shutdown here right?
cmd/server/server.go
Outdated
| httpServer := &http.Server{Handler: promhttp.HandlerFor(prometheusRepo.Registry, promhttp.HandlerOpts{}), Addr: promAddr} | ||
| http.Handle(promPath, promhttp.Handler()) | ||
| logger.Infof("Proemtheus listening on %s...", promAddr) | ||
| if err := httpServer.ListenAndServe(); err != nil { |
There was a problem hiding this comment.
Let's run this gracefully - see: https://github.com/ory/oathkeeper/pull/404/files#diff-4c51d95b26718af4a2ac3bec33e54e49R100-R111
|
If you merge with master, the weird failing test cases (flaky) will stop to happen :) |
ecktom
force-pushed
the
feat/add_prometheus
branch
from
8ede182 to
9b77989
Compare
aeneasr
reviewed
May 1, 2020
There was a problem hiding this comment.
Not sure if you've missed this comment:
This looks pretty good to me! I've found a couple of things that could be improved. Also, would you be able to add a small section at the bottom of https://github.com/ory/oathkeeper/blob/master/docs/docs/configure-deploy.md#authorizing-requests on setting up ORY Oathkeeper with prometheus (optionally grafana if it's easy to do).
Let me know :)
|
Oh yeah, I've missed that comment :D Will add some words about this once it's running as I would also setup a Grafana board for us internally which I surely can share afterwards. |
|
If you have ideas how to e2e test it that would be awesome! |
|
A dirty one... ;) Put it on our staging ENV, didn't crash or log any error and provided metrics on :9000/metrics as expected. So I guess it's good to go ;) |
|
Ok nice, I thought you meant automated e2e test :D Do you need help with the docs? |
|
Will come up with something next week, should be OK. |
|
Awesome, thanks! |
|
Thank you for your hard work! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed changes
This will add an basic prometheus endpoint providing request metrics to get some application insights
Checklist
vulnerability, I confirm that I got green light (please contact [email protected]) from the maintainers to push the changes.
Further comments