Rewrite tracing support using the stats handler interface

[dfawley]

One notable problem here is that we can currently only install one stats handler at a time. This means we can't enable our default tracing and also let the user install a stats handler.

See also: #695

[roadrunner]

net/trace has such a bad implementation as well as grpc-go tracing decision.

what is the point of registering debug handlers at package-init level? for customisation, there is nothing you can do.

and grpc-go enables tracing default. there should be a better way to configure it, other than that mutating init level global variables.
it's not possible to define something like only enable tracing for specific requests or api methods.

I think this is not suitable for a production use case, should be disabled by default.

[dfawley]

I think this is not suitable for a production use case, should be disabled by default.

We agree; tracing was disabled by default in #1509.

We may delete the global entirely and just provide a stats handler package the user can install if they want net/trace integration.

[dzbarsky]

#1434 would let you have more than one stats handler :) I've been having some trouble finding time to finish it off, maybe next week (or you can take it over if you want)

[egonelbre]

Also, noticed that golang.org/x/net/trace dependency makes binaries about ~1MB larger.

[stale]

This issue is labeled as requiring an update from the reporter, and no update has been received after 7 days. If no update is provided in the next 7 days, this issue will be automatically closed.

[menghanl]

We have multiple monitoring features (channelz, stats handler, binary logging), and we are making a new API to unify all them (https://xkcd.com/927/)
We will look more into x/net/trace support in this project.

[easwars]

Support for multiple stats handler on the client and server was added as part of #5347.