Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update OpenTelemetry Tracing version status #13808

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG/CHANGELOG-3.6.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,6 +57,7 @@ See [code changes](https://github.com/etcd-io/etcd/compare/v3.5.0...v3.6.0).
- Fix [etcd gateway doesn't format the endpoint of IPv6 address correctly](https://github.com/etcd-io/etcd/pull/13551)
- Fix [A client can cause a nil dereference in etcd by passing an invalid SortTarget](https://github.com/etcd-io/etcd/pull/13555)
- Fix [Grant lease with negative ID can possibly cause db out of sync](https://github.com/etcd-io/etcd/pull/13676)
- Fix [Update OpenTelemetry Tracing version status](https://github.com/etcd-io/etcd/pull/13808)
hanyuancheung marked this conversation as resolved.
Show resolved Hide resolved

### tools/benchmark

Expand Down
29 changes: 16 additions & 13 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -33,23 +33,25 @@ require (
go.etcd.io/etcd/tests/v3 v3.6.0-alpha.0
go.uber.org/zap v1.17.0
golang.org/x/time v0.0.0-20210220033141-f8bda1e9f3ba
google.golang.org/grpc v1.41.0
google.golang.org/grpc v1.45.0
google.golang.org/protobuf v1.27.1
gopkg.in/cheggaaa/pb.v1 v1.0.28
)

require (
github.com/beorn7/perks v1.0.1 // indirect
github.com/cenkalti/backoff/v4 v4.1.1 // indirect
github.com/cenkalti/backoff/v4 v4.1.2 // indirect
github.com/cespare/xxhash/v2 v2.1.1 // indirect
github.com/coreos/go-systemd/v22 v22.3.2 // indirect
github.com/davecgh/go-spew v1.1.1 // indirect
github.com/go-logr/logr v1.2.3 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
github.com/gogo/protobuf v1.3.2 // indirect
github.com/golang-jwt/jwt v3.2.2+incompatible // indirect
github.com/golang/groupcache v0.0.0-20210331224755-41bb18bfe9da // indirect
github.com/golang/protobuf v1.5.2 // indirect
github.com/google/btree v1.0.1 // indirect
github.com/google/go-cmp v0.5.6 // indirect
github.com/google/go-cmp v0.5.7 // indirect
github.com/gorilla/websocket v1.4.2 // indirect
github.com/grpc-ecosystem/go-grpc-middleware v1.3.0 // indirect
github.com/grpc-ecosystem/go-grpc-prometheus v1.2.0 // indirect
Expand All @@ -70,21 +72,22 @@ require (
github.com/stretchr/testify v1.7.0 // indirect
github.com/tmc/grpc-websocket-proxy v0.0.0-20201229170055-e5319fda7802 // indirect
github.com/xiang90/probing v0.0.0-20190116061207-43a291ad63a2 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.26.1 // indirect
go.opentelemetry.io/otel v1.2.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.1.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.1.0 // indirect
go.opentelemetry.io/otel/sdk v1.2.0 // indirect
go.opentelemetry.io/otel/trace v1.2.0 // indirect
go.opentelemetry.io/proto/otlp v0.10.0 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.30.0 // indirect
go.opentelemetry.io/otel v1.5.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/internal/retry v1.5.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.5.0 // indirect
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.5.0 // indirect
go.opentelemetry.io/otel/sdk v1.5.0 // indirect
go.opentelemetry.io/otel/trace v1.5.0 // indirect
go.opentelemetry.io/proto/otlp v0.12.0 // indirect
go.uber.org/atomic v1.7.0 // indirect
go.uber.org/multierr v1.7.0 // indirect
golang.org/x/crypto v0.0.0-20210513164829-c07d793c2f9a // indirect
golang.org/x/net v0.0.0-20220105145211-5b0dc2dfae98 // indirect
golang.org/x/sys v0.0.0-20211123173158-ef496fb156ab // indirect
golang.org/x/net v0.0.0-20220225172249-27dd8689420f // indirect
golang.org/x/sys v0.0.0-20220318055525-2edf467146b5 // indirect
golang.org/x/text v0.3.7 // indirect
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1 // indirect
google.golang.org/genproto v0.0.0-20211118181313-81c1377c94b1 // indirect
google.golang.org/genproto v0.0.0-20220317150908-0efb43f6373e // indirect
gopkg.in/natefinch/lumberjack.v2 v2.0.0 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.0-20210107192922-496545a6307b // indirect
Expand Down
58 changes: 45 additions & 13 deletions go.sum

Large diffs are not rendered by default.

30 changes: 15 additions & 15 deletions server/embed/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -347,22 +347,22 @@ type Config struct {
ListenMetricsUrls []url.URL
ListenMetricsUrlsJSON string `json:"listen-metrics-urls"`

// ExperimentalEnableDistributedTracing indicates if experimental tracing using OpenTelemetry is enabled.
ExperimentalEnableDistributedTracing bool `json:"experimental-enable-distributed-tracing"`
// ExperimentalDistributedTracingAddress is the address of the OpenTelemetry Collector.
// Can only be set if ExperimentalEnableDistributedTracing is true.
ExperimentalDistributedTracingAddress string `json:"experimental-distributed-tracing-address"`
// ExperimentalDistributedTracingServiceName is the name of the service.
// Can only be used if ExperimentalEnableDistributedTracing is true.
ExperimentalDistributedTracingServiceName string `json:"experimental-distributed-tracing-service-name"`
// ExperimentalDistributedTracingServiceInstanceID is the ID key of the service.
// EnableDistributedTracing indicates if tracing using OpenTelemetry is enabled.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's have a separate PR to stabilize distributing tracing feature. We still have some work before stabilization.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I get it. So this PR should just work on updating the version of OpenTelemetry right?
If so, I'll cancel the changes of all stabilization work ~

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we can update OpenTelemetry now and decide separably to stabilize the tracing.

EnableDistributedTracing bool `json:"enable-distributed-tracing"`
// DistributedTracingAddress is the address of the OpenTelemetry Collector.
// Can only be set if EnableDistributedTracing is true.
DistributedTracingAddress string `json:"distributed-tracing-address"`
// DistributedTracingServiceName is the name of the service.
// Can only be used if DistributedTracing is true.
DistributedTracingServiceName string `json:"distributed-tracing-service-name"`
// DistributedTracingServiceInstanceID is the ID key of the service.
// This ID must be unique, as helps to distinguish instances of the same service
// that exist at the same time.
// Can only be used if ExperimentalEnableDistributedTracing is true.
ExperimentalDistributedTracingServiceInstanceID string `json:"experimental-distributed-tracing-instance-id"`
// ExperimentalDistributedTracingSamplingRatePerMillion is the number of samples to collect per million spans.
// Can only be used if EnableDistributedTracing is true.
DistributedTracingServiceInstanceID string `json:"distributed-tracing-instance-id"`
// DistributedTracingSamplingRatePerMillion is the number of samples to collect per million spans.
// Defaults to 0.
ExperimentalDistributedTracingSamplingRatePerMillion int `json:"experimental-distributed-tracing-sampling-rate"`
DistributedTracingSamplingRatePerMillion int `json:"distributed-tracing-sampling-rate"`

// Logger is logger options: currently only supports "zap".
// "capnslog" is removed in v3.5.
Expand Down Expand Up @@ -738,8 +738,8 @@ func (cfg *Config) Validate() error {
}

// Validate distributed tracing configuration but only if enabled.
if cfg.ExperimentalEnableDistributedTracing {
if err := validateTracingConfig(cfg.ExperimentalDistributedTracingSamplingRatePerMillion); err != nil {
if cfg.EnableDistributedTracing {
if err := validateTracingConfig(cfg.DistributedTracingSamplingRatePerMillion); err != nil {
return fmt.Errorf("distributed tracing configurition is not valid: (%v)", err)
}
}
Expand Down
19 changes: 10 additions & 9 deletions server/embed/config_tracing.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
"go.opentelemetry.io/otel/sdk/resource"
tracesdk "go.opentelemetry.io/otel/sdk/trace"
semconv "go.opentelemetry.io/otel/semconv/v1.4.0"

"go.uber.org/zap"
)

Expand All @@ -43,22 +44,22 @@ func validateTracingConfig(samplingRate int) error {
func setupTracingExporter(ctx context.Context, cfg *Config) (exporter tracesdk.SpanExporter, options []otelgrpc.Option, err error) {
exporter, err = otlptracegrpc.New(ctx,
otlptracegrpc.WithInsecure(),
otlptracegrpc.WithEndpoint(cfg.ExperimentalDistributedTracingAddress),
otlptracegrpc.WithEndpoint(cfg.DistributedTracingAddress),
)
if err != nil {
return nil, nil, err
}

res, err := resource.New(ctx,
resource.WithAttributes(
semconv.ServiceNameKey.String(cfg.ExperimentalDistributedTracingServiceName),
semconv.ServiceNameKey.String(cfg.DistributedTracingServiceName),
),
)
if err != nil {
return nil, nil, err
}

if resWithIDKey := determineResourceWithIDKey(cfg.ExperimentalDistributedTracingServiceInstanceID); resWithIDKey != nil {
if resWithIDKey := determineResourceWithIDKey(cfg.DistributedTracingServiceInstanceID); resWithIDKey != nil {
// Merge resources into a new
// resource in case of duplicates.
res, err = resource.Merge(res, resWithIDKey)
Expand All @@ -79,18 +80,18 @@ func setupTracingExporter(ctx context.Context, cfg *Config) (exporter tracesdk.S
tracesdk.WithBatcher(exporter),
tracesdk.WithResource(res),
tracesdk.WithSampler(
tracesdk.ParentBased(determineSampler(cfg.ExperimentalDistributedTracingSamplingRatePerMillion)),
tracesdk.ParentBased(determineSampler(cfg.DistributedTracingSamplingRatePerMillion)),
),
),
),
)

cfg.logger.Debug(
"distributed tracing enabled",
zap.String("address", cfg.ExperimentalDistributedTracingAddress),
zap.String("service-name", cfg.ExperimentalDistributedTracingServiceName),
zap.String("service-instance-id", cfg.ExperimentalDistributedTracingServiceInstanceID),
zap.Int("sampling-rate", cfg.ExperimentalDistributedTracingSamplingRatePerMillion),
zap.String("address", cfg.DistributedTracingAddress),
zap.String("service-name", cfg.DistributedTracingServiceName),
zap.String("service-instance-id", cfg.DistributedTracingServiceInstanceID),
zap.Int("sampling-rate", cfg.DistributedTracingSamplingRatePerMillion),
)

return exporter, options, err
Expand All @@ -110,7 +111,7 @@ func determineSampler(samplingRate int) tracesdk.Sampler {
func determineResourceWithIDKey(serviceInstanceID string) *resource.Resource {
if serviceInstanceID != "" {
return resource.NewSchemaless(
(semconv.ServiceInstanceIDKey.String(serviceInstanceID)),
semconv.ServiceInstanceIDKey.String(serviceInstanceID),
)
}
return nil
Expand Down
2 changes: 1 addition & 1 deletion server/embed/etcd.go
Original file line number Diff line number Diff line change
Expand Up @@ -205,7 +205,7 @@ func StartEtcd(inCfg *Config) (e *Etcd, err error) {
Logger: cfg.logger,
ForceNewCluster: cfg.ForceNewCluster,
EnableGRPCGateway: cfg.EnableGRPCGateway,
ExperimentalEnableDistributedTracing: cfg.ExperimentalEnableDistributedTracing,
ExperimentalEnableDistributedTracing: cfg.EnableDistributedTracing,
UnsafeNoFsync: cfg.UnsafeNoFsync,
EnableLeaseCheckpoint: cfg.ExperimentalEnableLeaseCheckpoint,
LeaseCheckpointPersist: cfg.ExperimentalEnableLeaseCheckpointPersist,
Expand Down
10 changes: 5 additions & 5 deletions server/etcdmain/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -280,11 +280,11 @@ func newConfig() *config {
fs.StringVar(&cfg.ec.Metrics, "metrics", cfg.ec.Metrics, "Set level of detail for exported metrics, specify 'extensive' to include server side grpc histogram metrics")

// experimental distributed tracing
fs.BoolVar(&cfg.ec.ExperimentalEnableDistributedTracing, "experimental-enable-distributed-tracing", false, "Enable experimental distributed tracing using OpenTelemetry Tracing.")
fs.StringVar(&cfg.ec.ExperimentalDistributedTracingAddress, "experimental-distributed-tracing-address", embed.ExperimentalDistributedTracingAddress, "Address for distributed tracing used for OpenTelemetry Tracing (if enabled with experimental-enable-distributed-tracing flag).")
fs.StringVar(&cfg.ec.ExperimentalDistributedTracingServiceName, "experimental-distributed-tracing-service-name", embed.ExperimentalDistributedTracingServiceName, "Configures service name for distributed tracing to be used to define service name for OpenTelemetry Tracing (if enabled with experimental-enable-distributed-tracing flag). 'etcd' is the default service name. Use the same service name for all instances of etcd.")
fs.StringVar(&cfg.ec.ExperimentalDistributedTracingServiceInstanceID, "experimental-distributed-tracing-instance-id", "", "Configures service instance ID for distributed tracing to be used to define service instance ID key for OpenTelemetry Tracing (if enabled with experimental-enable-distributed-tracing flag). There is no default value set. This ID must be unique per etcd instance.")
fs.IntVar(&cfg.ec.ExperimentalDistributedTracingSamplingRatePerMillion, "experimental-distributed-tracing-sampling-rate", 0, "Number of samples to collect per million spans for OpenTelemetry Tracing (if enabled with experimental-enable-distributed-tracing flag).")
fs.BoolVar(&cfg.ec.EnableDistributedTracing, "enable-distributed-tracing", false, "Enable experimental distributed tracing using OpenTelemetry Tracing.")
fs.StringVar(&cfg.ec.DistributedTracingAddress, "distributed-tracing-address", embed.ExperimentalDistributedTracingAddress, "Address for distributed tracing used for OpenTelemetry Tracing (if enabled with enable-distributed-tracing flag).")
fs.StringVar(&cfg.ec.DistributedTracingServiceName, "distributed-tracing-service-name", embed.ExperimentalDistributedTracingServiceName, "Configures service name for distributed tracing to be used to define service name for OpenTelemetry Tracing (if enabled with enable-distributed-tracing flag). 'etcd' is the default service name. Use the same service name for all instances of etcd.")
fs.StringVar(&cfg.ec.DistributedTracingServiceInstanceID, "distributed-tracing-instance-id", "", "Configures service instance ID for distributed tracing to be used to define service instance ID key for OpenTelemetry Tracing (if enabled with enable-distributed-tracing flag). There is no default value set. This ID must be unique per etcd instance.")
fs.IntVar(&cfg.ec.DistributedTracingSamplingRatePerMillion, "distributed-tracing-sampling-rate", 0, "Number of samples to collect per million spans for OpenTelemetry Tracing (if enabled with enable-distributed-tracing flag).")

// auth
fs.StringVar(&cfg.ec.AuthToken, "auth-token", cfg.ec.AuthToken, "Specify auth token specific options.")
Expand Down
10 changes: 5 additions & 5 deletions server/etcdmain/help.go
Original file line number Diff line number Diff line change
Expand Up @@ -228,15 +228,15 @@ Logging:
Configures log rotation if enabled with a JSON logger config. MaxSize(MB), MaxAge(days,0=no limit), MaxBackups(0=no limit), LocalTime(use computers local time), Compress(gzip)".

Experimental distributed tracing:
--experimental-enable-distributed-tracing 'false'
--enable-distributed-tracing 'false'
Enable experimental distributed tracing.
--experimental-distributed-tracing-address 'localhost:4317'
--distributed-tracing-address 'localhost:4317'
Distributed tracing collector address.
--experimental-distributed-tracing-service-name 'etcd'
--distributed-tracing-service-name 'etcd'
Distributed tracing service name, must be same across all etcd instances.
--experimental-distributed-tracing-instance-id ''
--distributed-tracing-instance-id ''
Distributed tracing instance ID, must be unique per each etcd instance.
--experimental-distributed-tracing-sampling-rate '0'
--distributed-tracing-sampling-rate '0'
Number of samples to collect per million spans for distributed tracing. Disabled by default.

v2 Proxy (to be deprecated in v3.6):
Expand Down