diff --git a/data-prepper-plugins/otel-trace-source/README.md b/data-prepper-plugins/otel-trace-source/README.md index 609c012565..0e16584736 100644 --- a/data-prepper-plugins/otel-trace-source/README.md +++ b/data-prepper-plugins/otel-trace-source/README.md @@ -4,12 +4,27 @@ This is a source which follows the [OTLP Protocol](https://github.com/open-telem ## Usages -Example `.yaml` configuration: +Example `.yaml` configuration for Data Prepper 2.0: + ```yaml source: - otel_trace_source: + otel_trace_source: ``` + +Example `.yaml` configuration for Data Prepper 1.4 and above: + +```yaml +source: + otel_trace_source: + record_type: event +``` + +### Migrating to Data Prepper 2.0 + +For more information on migrating from Data Prepper 1.x to Data Prepper 2.x, see _Migrating to Data Prepper 2.0_ in the +[Trace Analytics documentation](https://github.com/opensearch-project/data-prepper/blob/main/docs/trace_analytics.md). + ## Configurations * port(Optional) => An `int` represents the port Otel trace source is running on. Default is ```21890```. @@ -21,6 +36,7 @@ source: * thread_count(Optional) => the number of threads to keep in the ScheduledThreadPool. Default is `200`. * max_connection_count(Optional) => the maximum allowed number of open connections. Default is `500`. * authentication(Optional) => An authentication configuration. By default, this runs an unauthenticated server. See below for more information. +* record_type(Optional) => A string represents the supported record data type that is written into the buffer plugin. Value options are `otlp` or `event`. Default is `otlp`. ### Authentication Configurations @@ -31,6 +47,7 @@ The following is an example of how to run the server with HTTP Basic authenticat ```yaml source: otel_trace_source: + record_type: event authentication: http_basic: username: my-user @@ -42,6 +59,7 @@ You can also explicitly disable authentication with: ```yaml source: otel_trace_source: + record_type: event authentication: unauthenticated: ``` @@ -66,10 +84,11 @@ Create the following otel-trace-source configuration in your `pipeline.yaml`. ```yaml source: otel_trace_source: - ssl: true - sslKeyCertChainFile: "/full/path/to/certfile.crt" - sslKeyFile: "/full/path/to/keyfile.key" - unframed_requests: true + record_type: event + ssl: true + sslKeyCertChainFile: "/full/path/to/certfile.crt" + sslKeyFile: "/full/path/to/keyfile.key" + unframed_requests: true ``` Generate a private key named `keyfile.key`, along with a self-signed certificate named `certfile.crt`. diff --git a/docs/trace_analytics.md b/docs/trace_analytics.md index e0f50b4d44..0f57f1a61e 100644 --- a/docs/trace_analytics.md +++ b/docs/trace_analytics.md @@ -53,6 +53,7 @@ Example `otel-trace-source` with SSL and Basic Authentication enabled. Note that ```yaml source: otel_trace_source: + #record_type: event # Add this when using Data Prepper 1.x. This option is removed in 2.0 ssl: true sslKeyCertChainFile: "/full/path/to/certfile.crt" sslKeyFile: "/full/path/to/keyfile.key" @@ -77,6 +78,7 @@ otel-trace-pipeline: delay: "100" source: otel_trace_source: + #record_type: event # Add this when using Data Prepper 1.x. This option is removed in 2.0 ssl: false # Change this to enable encryption in transit authentication: unauthenticated: @@ -237,3 +239,17 @@ documents how to use OpenSearch Dashboards. The [Trace Tuning page](trace_tuning.md) has information to help you tune and scale Data Prepper for trace analytics use cases. + +## Migrating to Data Prepper 2.0 + +Starting in Data Prepper 1.4, the trace processing uses Data Prepper's Event model. This allows +pipeline authors the ability to configure other processors to modify spans or traces. + +To provide a migration path, Data Prepper 1.4 introduced the following changes. +* The `otel_trace_source` has an optional parameter `record_type` which can be set to `event`. When configured, it will output event objects. +* The `otel_trace_raw` replaces `otel_trace_raw_prepper` for event-based spans. +* The `otel_trace_group` replaces `otel_trace_group_prepper` for event-based spans. + +In Data Prepper 2.0, the `otel_trace_source` will only output Events. Data Prepper 2.0 also removes +`otel_trace_raw_prepper` and `otel_trace_group_prepper` entirely. To help migrate to 2.0, +you can configure your trace pipeline using the Event model.