Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(tracing): dont trace spans with full request paths as operation name in ExtractFromHTTPRequest #15971

Merged
merged 2 commits into from
Nov 25, 2019

Conversation

GeorgeMac
Copy link
Contributor

@GeorgeMac GeorgeMac commented Nov 19, 2019

This updates our automatic span creation from http request method in the tracing kit.

Prior to this change all request paths were being logged in spans as operation names.
This has led to many operation names (and therefore increased cardinality required).

Now we have all operations being reported as request along with the following tags:

  • base_path which is an attempt to capture import resource identifiers e.g. (/api/v2/buckets) but not the resource identifier
  • handler_name which is the handler which is called to invoke this operation

It also logs the full url path with the key path. This is for a little more context.

My proposed change has increased the number of allocations required. So I thought I would share here for your consideration:

Before:
BenchmarkOpentracing_ExtractFromHTTPRequest-16    	 2000000	      1003 ns/op	     672 B/op	       5 allocs/op



After:
BenchmarkOpentracing_ExtractFromHTTPRequest-16    	 1000000	      1213 ns/op	     816 B/op	      10 allocs/op

Update:

So I have a PR open on httprouter julienschmidt/httprouter#286

This now vendors that change and relies on it to get the router match from the request context (this what my PR allows you to do with httprouter).

This means we don't need to do any complicated deriving of some base path. Instead we tag the spans with the router path as was desired.

e.g. a handler mounted on route /api/v2/buckets/:bucket_id would produce the tag route=/api/v2/buckets/:bucket_id .

goos: darwin
goarch: amd64
pkg: github.com/influxdata/influxdb/kit/tracing
BenchmarkOpentracing_ExtractFromHTTPRequest-16    	 2000000	      1008 ns/op	     688 B/op	       7 allocs/op
PASS
ok  	github.com/influxdata/influxdb/kit/tracing	3.032s
  • CHANGELOG.md updated with a link to the PR (not the Issue)
  • Well-formatted commit messages
  • Rebased/mergeable
  • Tests pass
  • http/swagger.yml updated (if modified Go structs or API)
  • Documentation updated or issue created (provide link to issue/pr)
  • Signed CLA (if not already signed)

@GeorgeMac GeorgeMac changed the title feat(tracing): dont trace spans with full URL path names in ExtractFromHTTPRequest feat(tracing): dont trace spans with full paths as operation name in ExtractFromHTTPRequest Nov 19, 2019
@GeorgeMac GeorgeMac changed the title feat(tracing): dont trace spans with full paths as operation name in ExtractFromHTTPRequest feat(tracing): dont trace spans with full request paths as operation name in ExtractFromHTTPRequest Nov 19, 2019
Copy link
Member

@jacobmarble jacobmarble left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is unfortunate that we don't have access to the httprouter path associated with the handler. I just walked through some of the implementation using my debugger, and it appears that there is no way to access that path (I'm talking about /api/v2/foo/:id not the called path).

I also don't like handlerName because it can't be kept consistent between usage. Reflection would be a better way to fill that blank.

What do you think about adjusting ExtractFromHTTPRequest to take routerPath instead of handlerName?

kit/tracing/tracing.go Outdated Show resolved Hide resolved
kit/tracing/tracing.go Outdated Show resolved Hide resolved
kit/tracing/tracing.go Outdated Show resolved Hide resolved
@jacobmarble jacobmarble self-requested a review November 21, 2019 16:32
@GeorgeMac
Copy link
Contributor Author

I tagged you in this @mark-rushakoff to review the approach of doing a hard cut-over to an influxdata fork of httprouter.

@GeorgeMac
Copy link
Contributor Author

I will come back to this. Do a full replace everywhere means updating flux to use our fork as well.

"github.com/julienschmidt/httprouter"
"github.com/influxdata/httprouter"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought the go.mod replace was supposed to make import changes unnecessary?

Copy link
Contributor Author

@GeorgeMac GeorgeMac Nov 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah as @mark-rushakoff says. This is great until the compat tests run and then it breaks :( but seems like bringing the fork under our banner works.

My latest problem is flux depends on the original version 😂 that is my next challenge. figured it out

Copy link
Contributor

@mark-rushakoff mark-rushakoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me.

@jacobmarble using a replacement directive only affects the current module. Projects that import influxdb would still use the julienschmidt/httprouter module, which means they would not be able to compile influxdb due to our httprouter fork becoming intentionally API-incompatible.

go.sum Outdated Show resolved Hide resolved
@GeorgeMac GeorgeMac merged commit 3dbb9c0 into master Nov 25, 2019
@GeorgeMac GeorgeMac deleted the gm/dont-trace-paths branch November 25, 2019 14:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants