Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Align configuration with OpenTelemetry SDK, HTTP exporter #5

Merged
merged 1 commit into from
Jun 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 28 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,36 @@ A work in progress k6 extension to output real-time test metrics in [OpenTelemet
> [!WARNING]
> It's work in progress implementation and not ready for production use.

Configuration options (currently environment variables only):
## Configuration options

Currently, environment variables only. It's worth to mention that the extension is using the [OpenTelemetry Go SDK](https://opentelemetry.io/docs/languages/go/getting-started/) that's why it's possible to use the configuration environment variables from the SDK. However, if the `K6_OTEL_*` environment variables are set, they will take precedence over the SDK configuration.

### k6-specific configuration

* `K6_OTEL_RECEIVER_TYPE` - OpenTelemetry receiver type, currently only `grpc` is supported. Default is `grpc`.
* `K6_OTEL_RECEIVER_ENDPOINT` - OpenTelemetry receiver endpoint. Default is `localhost:4317`.
* `K6_OTEL_METRIC_PREFIX` - Metric prefix. Default is empty.
* `K6_OTEL_FLUSH_INTERVAL` - How frequently to flush metrics to the receiver from k6. Default is `1s`.
* `K6_OTEL_PUSH_INTERVAL` - How frequently to push metrics to the receiver from k6. Default is `1s`.
* `K6_OTEL_FLUSH_INTERVAL` - How frequently to flush metrics from k6 metrics engine. Default is `1s`.

### OpenTelemetry-specific configuration

* `K6_OTEL_EXPORT_INTERVAL` - configures the intervening time between metrics exports. Default is `1s`.
* `K6_OTEL_EXPORTER_TYPE` - metric exporter type. Default is `grpc`.

#### GRPC exporter

* `K6_OTEL_GRPC_EXPORTER_INSECURE` - disables client transport security for the gRPC exporter.
* `K6_OTEL_GRPC_EXPORTER_ENDPOINT` - configures the gRPC exporter endpoint. Default is `localhost:4317`.

> [!TIP]
> Also, you can use [OpenTelemetry SDK configuration environment variables](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlpmetric/[email protected]).

#### HTTP exporter

* `K6_OTEL_HTTP_EXPORTER_INSECURE` - disables client transport security for the HTTP exporter.
* `K6_OTEL_HTTP_EXPORTER_ENDPOINT` - configures the HTTP exporter endpoint. Default is `localhost:4318`.
* `K6_OTEL_HTTP_EXPORTER_URL_PATH` - configures the HTTP exporter path. Default is `/v1/metrics`.

> [!TIP]
> Also, you can use [OpenTelemetry SDK configuration environment variables](https://pkg.go.dev/go.opentelemetry.io/otel/exporters/otlp/otlpmetric/[email protected]).

## Build

Expand Down
3 changes: 2 additions & 1 deletion go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,11 @@ require (
go.k6.io/k6 v0.51.0
go.opentelemetry.io/otel v1.26.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.26.0
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.26.0
go.opentelemetry.io/otel/metric v1.26.0
go.opentelemetry.io/otel/sdk v1.26.0
go.opentelemetry.io/otel/sdk/metric v1.26.0
gopkg.in/guregu/null.v3 v3.5.0
)

require (
Expand Down Expand Up @@ -40,6 +42,5 @@ require (
google.golang.org/genproto/googleapis/rpc v0.0.0-20240401170217-c3f982113cda // indirect
google.golang.org/grpc v1.63.2 // indirect
google.golang.org/protobuf v1.33.0 // indirect
gopkg.in/guregu/null.v3 v3.5.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
2 changes: 2 additions & 0 deletions go.sum
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,8 @@ go.opentelemetry.io/otel v1.26.0 h1:LQwgL5s/1W7YiiRwxf03QGnWLb2HW4pLiAhaA5cZXBs=
go.opentelemetry.io/otel v1.26.0/go.mod h1:UmLkJHUAidDval2EICqBMbnAd0/m2vmpf/dAM+fvFs4=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.26.0 h1:+hm+I+KigBy3M24/h1p/NHkUx/evbLH0PNcjpMyCHc4=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetricgrpc v1.26.0/go.mod h1:NjC8142mLvvNT6biDpaMjyz78kyEHIwAJlSX0N9P5KI=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.26.0 h1:HGZWGmCVRCVyAs2GQaiHQPbDHo+ObFWeUEOd+zDnp64=
go.opentelemetry.io/otel/exporters/otlp/otlpmetric/otlpmetrichttp v1.26.0/go.mod h1:SaH+v38LSCHddyk7RGlU9uZyQoRrKao6IBnJw6Kbn+c=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.24.0 h1:t6wl9SPayj+c7lEIFgm4ooDBZVb01IhLB4InpomhRw8=
go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.24.0/go.mod h1:iSDOcsnSA5INXzZtwaBPrKp/lWu/V14Dd+llD0oI2EA=
go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc v1.24.0 h1:Mw5xcxMwlqoJd97vwPxA8isEaIoxsta9/Q51+TTJLGE=
Expand Down
157 changes: 130 additions & 27 deletions pkg/opentelemetry/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,82 +5,185 @@ import (
"fmt"
"time"

k6Const "go.k6.io/k6/lib/consts"
"go.k6.io/k6/output"
"gopkg.in/guregu/null.v3"
)

const (
grpcReceiverType = "grpc"
// grpcExporterType GRPC exporter type
grpcExporterType = "grpc"
// httpExporterType HTTP exporter type
httpExporterType = "http"
)

// Config is the config for the template collector
type Config struct {
// ServiceName is the name of the service to use for the metrics
// export, if not set it will use "k6"
ServiceName string
// ServiceVersion is the version of the service to use for the metrics
// export, if not set it will use k6's library version
ServiceVersion string
// MetricPrefix is the prefix to use for the metrics
MetricPrefix string
// ReceiverType is the type of the receiver to use
ReceiverType string
// GRPCReceiverEndpoint is the endpoint of the gRPC receiver
GRPCReceiverEndpoint string
// PushInterval is the interval at which to push metrics to the receiver
PushInterval time.Duration
// FlushInterval is the interval at which to flush metrics from the k6
FlushInterval time.Duration

// ExporterType sets the type of OpenTelemetry Exporter to use
ExporterType string
// ExportInterval configures the intervening time between metrics exports
ExportInterval time.Duration

// HTTPExporterInsecure disables client transport security for the Exporter's HTTP
// connection.
HTTPExporterInsecure null.Bool
// HTTPExporterEndpoint sets the target endpoint the OpenTelemetry Exporter
// will connect to.
HTTPExporterEndpoint string
// HTTPExporterURLPath sets the target URL path the OpenTelemetry Exporter
HTTPExporterURLPath string

// GRPCExporterEndpoint sets the target endpoint the OpenTelemetry Exporter
// will connect to.
GRPCExporterEndpoint string
// GRPCExporterInsecure disables client transport security for the Exporter's gRPC
// connection.
GRPCExporterInsecure null.Bool
}

// NewConfig creates and validates a new config
func NewConfig(p output.Params) (Config, error) {
cfg := Config{
MetricPrefix: "",
ReceiverType: grpcReceiverType,
GRPCReceiverEndpoint: "localhost:4317",
PushInterval: 1 * time.Second,
FlushInterval: 1 * time.Second,
ServiceName: "k6",
ServiceVersion: k6Const.Version,
MetricPrefix: "",
ExporterType: grpcExporterType,

HTTPExporterInsecure: null.BoolFrom(false),
HTTPExporterEndpoint: "localhost:4318",
HTTPExporterURLPath: "/v1/metrics",

GRPCExporterInsecure: null.BoolFrom(false),
GRPCExporterEndpoint: "localhost:4317",

ExportInterval: 1 * time.Second,
FlushInterval: 1 * time.Second,
Comment on lines 56 to +71
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: In general it is better idea to always use null types for everything as that lets you know if you should use a default value or if somebody just managed to match it.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implemented as part of: #11

}

var err error
for k, v := range p.Environment {
switch k {
case "K6_OTEL_PUSH_INTERVAL":
cfg.PushInterval, err = time.ParseDuration(v)
if err != nil {
return cfg, fmt.Errorf("error parsing environment variable 'K6_OTEL_PUSH_INTERVAL': %w", err)
}
case "K6_OTEL_SERVICE_NAME":
cfg.ServiceName = v
case "K6_OTEL_SERVICE_VERSION":
cfg.ServiceVersion = v
Comment on lines 75 to +80
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not certain if jut using envconfig.Process as all the other outputs will not be better especially before we move the whole configuration over to not using envconfig.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, implemented as part of: #11

case "K6_OTEL_METRIC_PREFIX":
cfg.MetricPrefix = v
case "K6_OTEL_EXPORT_INTERVAL":
cfg.ExportInterval, err = time.ParseDuration(v)
if err != nil {
return cfg, fmt.Errorf("error parsing environment variable 'K6_OTEL_EXPORT_INTERVAL': %w", err)
}
case "K6_OTEL_FLUSH_INTERVAL":
cfg.FlushInterval, err = time.ParseDuration(v)
if err != nil {
return cfg, fmt.Errorf("error parsing environment variable 'K6_OTEL_FLUSH_INTERVAL': %w", err)
}
case "K6_OTEL_RECEIVER_TYPE":
cfg.ReceiverType = v
case "K6_OTEL_GRPC_RECEIVER_ENDPOINT":
cfg.GRPCReceiverEndpoint = v
case "K6_OTEL_EXPORTER_TYPE":
cfg.ExporterType = v
case "K6_OTEL_GRPC_EXPORTER_ENDPOINT":
cfg.GRPCExporterEndpoint = v
case "K6_OTEL_HTTP_EXPORTER_ENDPOINT":
cfg.HTTPExporterEndpoint = v
case "K6_OTEL_HTTP_EXPORTER_URL_PATH":
cfg.HTTPExporterURLPath = v
case "K6_OTEL_HTTP_EXPORTER_INSECURE":
cfg.HTTPExporterInsecure, err = parseBool(k, v)
if err != nil {
return cfg, err
}
case "K6_OTEL_GRPC_EXPORTER_INSECURE":
cfg.GRPCExporterInsecure, err = parseBool(k, v)
if err != nil {
return cfg, err
}
}
}

// TDOO: consolidated config

if err = cfg.Validate(); err != nil {
return cfg, fmt.Errorf("error validating config: %w", err)
return cfg, fmt.Errorf("error validating OpenTelemetry output config: %w", err)
}

return cfg, nil
}

func parseBool(k, v string) (null.Bool, error) {
bv := null.NewBool(false, false)

err := bv.UnmarshalText([]byte(v))
if err != nil {
return bv, fmt.Errorf("error parsing %q environment variable: %w", k, err)
}

return bv, nil
}

// Validate validates the config
func (c Config) Validate() error {
if c.ReceiverType != grpcReceiverType {
return fmt.Errorf("unsupported receiver type %q, currently only %q supported", c.ReceiverType, grpcReceiverType)
if c.ServiceName == "" {
return errors.New("providing service name is required")
}

if c.GRPCReceiverEndpoint == "" {
return errors.New("gRPC receiver endpoint is required")
if c.ServiceVersion == "" {
return errors.New("providing service version is required")
}

if c.ExporterType != grpcExporterType && c.ExporterType != httpExporterType {
return fmt.Errorf(
"unsupported exporter type %q, currently only %q and %q are supported",
c.ExporterType,
grpcExporterType,
httpExporterType,
)
}

if c.ExporterType == grpcExporterType {
if c.GRPCExporterEndpoint == "" {
return errors.New("gRPC exporter endpoint is required")
}
}

if c.ExporterType == httpExporterType {
if c.HTTPExporterEndpoint == "" {
return errors.New("HTTP exporter endpoint is required")
}
}

return nil
}

// String returns a string representation of the config
func (c Config) String() string {
return fmt.Sprintf("%s, %s", c.ReceiverType, c.GRPCReceiverEndpoint)
var endpoint string
exporter := c.ExporterType

if c.ExporterType == httpExporterType {
endpoint = "http"
if !c.HTTPExporterInsecure.Bool {
endpoint += "s"
}

endpoint += "://" + c.HTTPExporterEndpoint + c.HTTPExporterURLPath
} else {
endpoint = c.GRPCExporterEndpoint

if c.GRPCExporterInsecure.Bool {
exporter += " (insecure)"
}
}

return fmt.Sprintf("%s, %s", exporter, endpoint)
}
36 changes: 26 additions & 10 deletions pkg/opentelemetry/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,11 @@ import (
"time"

"github.com/stretchr/testify/require"
"gopkg.in/guregu/null.v3"

"go.k6.io/k6/output"

k6Const "go.k6.io/k6/lib/consts"
)

func TestConfig(t *testing.T) {
Expand All @@ -21,31 +25,43 @@ func TestConfig(t *testing.T) {
}{
"default": {
expectedConfig: Config{
ReceiverType: grpcReceiverType,
GRPCReceiverEndpoint: "localhost:4317",
PushInterval: 1 * time.Second,
ServiceName: "k6",
ServiceVersion: k6Const.Version,
ExporterType: grpcExporterType,
HTTPExporterInsecure: null.NewBool(false, true),
HTTPExporterEndpoint: "localhost:4318",
HTTPExporterURLPath: "/v1/metrics",
GRPCExporterInsecure: null.NewBool(false, true),
GRPCExporterEndpoint: "localhost:4317",
ExportInterval: 1 * time.Second,
FlushInterval: 1 * time.Second,
},
},

"overwrite": {
env: map[string]string{"K6_OTEL_GRPC_RECEIVER_ENDPOINT": "else", "K6_OTEL_PUSH_INTERVAL": "4ms"},
env: map[string]string{"K6_OTEL_GRPC_EXPORTER_ENDPOINT": "else", "K6_OTEL_EXPORT_INTERVAL": "4ms"},
expectedConfig: Config{
ReceiverType: grpcReceiverType,
GRPCReceiverEndpoint: "else",
PushInterval: 4 * time.Millisecond,
ServiceName: "k6",
ServiceVersion: k6Const.Version,
ExporterType: grpcExporterType,
HTTPExporterInsecure: null.NewBool(false, true),
HTTPExporterEndpoint: "localhost:4318",
HTTPExporterURLPath: "/v1/metrics",
GRPCExporterInsecure: null.NewBool(false, true),
GRPCExporterEndpoint: "else",
ExportInterval: 4 * time.Millisecond,
FlushInterval: 1 * time.Second,
},
},

"early error": {
env: map[string]string{"K6_OTEL_GRPC_RECEIVER_ENDPOINT": "else", "K6_OTEL_PUSH_INTERVAL": "4something"},
env: map[string]string{"K6_OTEL_GRPC_EXPORTER_ENDPOINT": "else", "K6_OTEL_EXPORT_INTERVAL": "4something"},
err: `time: unknown unit "something" in duration "4something"`,
},

"unsupported receiver type": {
env: map[string]string{"K6_OTEL_GRPC_RECEIVER_ENDPOINT": "else", "K6_OTEL_PUSH_INTERVAL": "4m", "K6_OTEL_RECEIVER_TYPE": "http"},
err: `error validating config: unsupported receiver type "http", currently only "grpc" supported`,
env: map[string]string{"K6_OTEL_GRPC_EXPORTER_ENDPOINT": "else", "K6_OTEL_EXPORT_INTERVAL": "4m", "K6_OTEL_EXPORTER_TYPE": "socket"},
err: `error validating OpenTelemetry output config: unsupported exporter type "socket", currently only "grpc" and "http" are supported`,
},
}

Expand Down
Loading
Loading