Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add metrics for otelhttp client #427

Closed
wants to merge 21 commits into from
Closed

Conversation

vchitai
Copy link

@vchitai vchitai commented Oct 30, 2020

Add some metrics for the otelhttp transport for http client net/http

Includes those metrics:

  • http_client_duration

Tested sample

$ curl localhost:10080/metrics
# HELP http_client_duration
# TYPE http_client_duration histogram
http_client_duration_bucket{http_client_host="localhost:10080",http_client_method="GET",http_client_path="/hello1",http_client_status="200",http_host="localhost:10080",http_method="GET",http_path="/hello1",http_status="200",le="+Inf"} 2
http_client_duration_sum{http_client_host="localhost:10080",http_client_method="GET",http_client_path="/hello1",http_client_status="200",http_host="localhost:10080",http_method="GET",http_path="/hello1",http_status="200"} 0.645514
http_client_duration_count{http_client_host="localhost:10080",http_client_method="GET",http_client_path="/hello1",http_client_status="200",http_host="localhost:10080",http_method="GET",http_path="/hello1",http_status="200"} 2

Referencing: https://github.com/census-instrumentation/opencensus-go/blob/master/plugin/ochttp/stats.go

@linux-foundation-easycla
Copy link

linux-foundation-easycla bot commented Oct 30, 2020

CLA Check
The committers are authorized under a signed CLA.

Copy link
Contributor

@MrAlias MrAlias left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The http metrics we implement need to conform to the OpenTelemetry metric semantic conventions: https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/metrics/semantic_conventions/http-metrics.md

@vchitai
Copy link
Author

vchitai commented Nov 1, 2020

The http metrics we implement need to conform to the OpenTelemetry metric semantic conventions: https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/metrics/semantic_conventions/http-metrics.md

Thanks for the feedback, I will change it later.

@vchitai
Copy link
Author

vchitai commented Nov 3, 2020

@MrAlias I have updated this PR according to your document, can you review it again?

@vchitai vchitai requested a review from MrAlias November 3, 2020 13:26
go.sum Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/common.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/common.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/stats.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/stats.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/stats.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/stats.go Outdated Show resolved Hide resolved
track.end()
} else {
track.body = resp.Body
resp.Body = wrappedBodyIO(track, resp.Body)
Copy link
Contributor

@MrAlias MrAlias Nov 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reading of the response body should not be included in the measurement of a requests duration. Right?

Copy link
Author

@vchitai vchitai Nov 13, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The request will be counted as ended after we finish reading all the data from the response body, won't it?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is probably utility in measurements of both time to first byte and time to the end of the body being read, though the latter is potentially less a measure of the client and more of whatever is processing the response data.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This only wrap and returns reader interface rather than actual read and parsing actual data itself. The actual parsing time could be useful, but should not be included in this package, dont you think ?

@vchitai vchitai force-pushed the master branch 2 times, most recently from bb3a22d to 8dd6cef Compare November 13, 2020 03:48
@vchitai vchitai force-pushed the master branch 2 times, most recently from 39e4b87 to b166165 Compare November 13, 2020 03:54
@vchitai vchitai requested a review from MrAlias November 13, 2020 04:05
func NewTransport(base http.RoundTripper, opts ...Option) *Transport {
t := Transport{
rt: base,
func NewTransport(base http.RoundTripper, opts ...Option) http.RoundTripper {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is the primary mechanism by which I expect otelhttp.Transport instances to be created and this leaves no configuration option for enabling/disabling metric collection, does it make more sense to integrate the metrics collection directly in the Transport.RoundTrip method? This would also be more in line with the approach taken by the otelhttp.Handler.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer to split up the implementation of metrics and tracing parts for ease of maintenance.

instrumentation/net/http/otelhttp/common.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/metrics.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/metrics.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/metrics.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/metrics.go Outdated Show resolved Hide resolved
// Perform request.;
resp, err := trans.base.RoundTrip(req)
if err != nil {
tracker.labels = append(labels, semconv.HTTPAttributesFromHTTPStatusCode(http.StatusInternalServerError)...)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should include the response status not http.StatusInternalServerError

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case of HTTP connection error occurred, should we recorded nothing or 500 as the HTTP status code. What's your opinion?

instrumentation/net/http/otelhttp/metrics.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/metrics.go Outdated Show resolved Hide resolved
instrumentation/net/http/otelhttp/metrics.go Outdated Show resolved Hide resolved
@vchitai vchitai requested review from MrAlias and Aneurysm9 November 20, 2020 03:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants