Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Init Elasticsearch exporter #2324

Merged
merged 14 commits into from
Feb 18, 2021
Merged
4 changes: 2 additions & 2 deletions exporter/elasticsearchexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ This exporter supports sending OpenTelemetry logs to [Elasticsearch](https://www
[ID](https://www.elastic.co/guide/en/cloud/current/ec-cloud-id.html) of the
Elastic Cloud Cluster to publish events to. The `cloudid` can be used instead
of `endpoints`.
- `workers` (optional): Number of workers publishing bulk requests concurrently.
- `nuym_workers` (optional): Number of workers publishing bulk requests concurrently.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- `nuym_workers` (optional): Number of workers publishing bulk requests concurrently.
- `num_workers` (optional): Number of workers publishing bulk requests concurrently.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed, thank you.

- `index`: The
[index](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices.html)
or [datastream](https://www.elastic.co/guide/en/elasticsearch/reference/current/data-streams.html)
Expand All @@ -23,7 +23,7 @@ This exporter supports sending OpenTelemetry logs to [Elasticsearch](https://www
- `retry`: Event retry settings
- `enabled` (default=true): Enable/Disable event retry on error. Retry
support is enabled by default.
- `max` (default=3): Number of HTTP retry attempts.
- `max_requests` (default=3): Number of HTTP request retries.
- `initial_interval` (default=100ms): Initial waiting time if a HTTP request failed.
- `max_interval` (default=1m): Max waiting time if a HTTP request failed.
- `mapping`: Events are encoded to JSON. The `mapping` allows users to
Expand Down
8 changes: 4 additions & 4 deletions exporter/elasticsearchexporter/config.go
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,8 @@ type Config struct {
// This setting is required if no URL is configured.
CloudID string `mapstructure:"cloudid"`

// Workers configures the number of workers publishing bulk requests.
Workers int `mapstructure:"workers"`
// NumWorkers configures the number of workers publishing bulk requests.
NumWorkers int `mapstructure:"num_workers"`

// Index configures the index, index alias, or data stream name events should be indexed in.
//
Expand Down Expand Up @@ -134,8 +134,8 @@ type RetrySettings struct {
// Enabled allows users to disable retry without having to comment out all settings.
Enabled bool `mapstructure:"enabled"`

// Max configures how often an HTTP request is retried before it is assumed to be failed.
Max int `mapstructure:"max"`
// MaxRequests configures how often an HTTP request is retried before it is assumed to be failed.
MaxRequests int `mapstructure:"max_requests"`

// InitialInterval configures the initial waiting time if a request failed.
InitialInterval time.Duration `mapstructure:"initial_interval"`
Expand Down
2 changes: 1 addition & 1 deletion exporter/elasticsearchexporter/config_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ func TestLoadConfig(t *testing.T) {
},
Retry: RetrySettings{
Enabled: true,
Max: 5,
MaxRequests: 5,
InitialInterval: 100 * time.Millisecond,
MaxInterval: 1 * time.Minute,
},
Expand Down
4 changes: 2 additions & 2 deletions exporter/elasticsearchexporter/exporter.go
Original file line number Diff line number Diff line change
Expand Up @@ -134,7 +134,7 @@ func newElasticsearchClient(logger *zap.Logger, config *Config) (*esClientCurren
RetryOnStatus: retryOnStatus,
DisableRetry: false,
EnableRetryOnTimeout: true,
MaxRetries: config.Retry.Max,
MaxRetries: config.Retry.MaxRequests,
RetryBackoff: createElasticsearchBackoffFunc(&config.Retry),

// configure sniffing
Expand Down Expand Up @@ -166,7 +166,7 @@ func newTransport(config *Config, tlsCfg *tls.Config) *http.Transport {
func newBulkIndexer(client *elasticsearch7.Client, config *Config) (esBulkIndexerCurrent, error) {
// TODO: add debug logger
return esutil7.NewBulkIndexer(esutil7.BulkIndexerConfig{
NumWorkers: config.Workers,
NumWorkers: config.NumWorkers,
FlushBytes: config.Flush.Bytes,
FlushInterval: config.Flush.Interval,
Client: client,
Expand Down
2 changes: 1 addition & 1 deletion exporter/elasticsearchexporter/factory.go
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ func createDefaultConfig() configmodels.Exporter {
Index: "logs-generic-default",
Retry: RetrySettings{
Enabled: true,
Max: 3,
MaxRequests: 3,
InitialInterval: 100 * time.Millisecond,
MaxInterval: 1 * time.Minute,
},
Expand Down
2 changes: 1 addition & 1 deletion exporter/elasticsearchexporter/testdata/config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ exporters:
flush:
bytes: 10485760
retry:
max: 5
max_requests: 5

service:
pipelines:
Expand Down