Updates Trace Analytics documentation for Data Prepper 2.0

data-prepper-plugins/otel-trace-source/README.md

@@ -4,12 +4,27 @@ This is a source which follows the [OTLP Protocol](https://github.com/open-telem
 
 
 ## Usages
-Example `.yaml` configuration:
+Example `.yaml` configuration for Data Prepper 2.0:
+
 ```yaml
 source:
-    otel_trace_source:
+  otel_trace_source:
 ```
 
+
+Example `.yaml` configuration for Data Prepper 1.4 and above:
+
+```yaml
+source:
+  otel_trace_source:
+    record_type: event
+```
+
+### Migrating to Data Prepper 2.0
+
+For more information on migrating from Data Prepper 1.x to Data Prepper 2.x, see _Migrating to Data Prepper 2.0_ in the 
+[Trace Analytics documentation](https://github.com/opensearch-project/data-prepper/blob/main/docs/trace_analytics.md).
+
 ## Configurations
 
 * port(Optional) => An `int` represents the port Otel trace source is running on. Default is ```21890```.
@@ -21,6 +36,7 @@ source:
 * thread_count(Optional) => the number of threads to keep in the ScheduledThreadPool. Default is `200`.
 * max_connection_count(Optional) => the maximum allowed number of open connections. Default is `500`. 
 * authentication(Optional) => An authentication configuration. By default, this runs an unauthenticated server. See below for more information.
+* record_type(Optional) => A string represents the supported record data type that is written into the buffer plugin. Value options are `otlp` or `event`. Default is `otlp`.
 
 ### Authentication Configurations
 
@@ -31,6 +47,7 @@ The following is an example of how to run the server with HTTP Basic authenticat
 ```yaml
 source:
   otel_trace_source:
+    record_type: event
     authentication:
       http_basic:
         username: my-user
@@ -42,6 +59,7 @@ You can also explicitly disable authentication with:
 ```yaml
 source:
   otel_trace_source:
+    record_type: event
     authentication:
       unauthenticated:
 ```
@@ -66,10 +84,11 @@ Create the following otel-trace-source configuration in your `pipeline.yaml`.
 ```yaml
 source:
   otel_trace_source:
-      ssl: true
-      sslKeyCertChainFile: "/full/path/to/certfile.crt"
-      sslKeyFile: "/full/path/to/keyfile.key"
-      unframed_requests: true
+    record_type: event
+    ssl: true
+    sslKeyCertChainFile: "/full/path/to/certfile.crt"
+    sslKeyFile: "/full/path/to/keyfile.key"
+    unframed_requests: true
 ```
 
 Generate a private key named `keyfile.key`, along with a self-signed certificate named `certfile.crt`.

docs/trace_analytics.md

@@ -53,6 +53,7 @@ Example `otel-trace-source` with SSL and Basic Authentication enabled. Note that
 ```yaml
 source:
   otel_trace_source:
+    #record_type: event  # Add this when using Data Prepper 1.x. This option is removed in 2.0
     ssl: true
     sslKeyCertChainFile: "/full/path/to/certfile.crt"
     sslKeyFile: "/full/path/to/keyfile.key"
@@ -77,6 +78,7 @@ otel-trace-pipeline:
   delay: "100" 
   source:
     otel_trace_source:
+      #record_type: event  # Add this when using Data Prepper 1.x. This option is removed in 2.0
       ssl: false # Change this to enable encryption in transit
       authentication:
         unauthenticated:
@@ -237,3 +239,17 @@ documents how to use OpenSearch Dashboards.
 
 The [Trace Tuning page](trace_tuning.md) has information to help you tune and scale Data Prepper for
 trace analytics use cases.
+
+## Migrating to Data Prepper 2.0
+
+Starting in Data Prepper 1.4, the trace processing uses Data Prepper's Event model. This allows
+pipeline authors the ability to configure other processors to modify spans or traces.
+
+To provide a migration path, Data Prepper 1.4 introduced the following changes.
+* The `otel_trace_source` has an optional parameter `record_type` which can be set to `event`. When configured, it will output event objects.
+* The `otel_trace_raw` replaces `otel_trace_raw_prepper` for event-based spans.
+* The `otel_trace_group` replaces `otel_trace_group_prepper` for event-based spans.
+
+In Data Prepper 2.0, the `otel_trace_source` will only output Events. Data Prepper 2.0 also removes
+`otel_trace_raw_prepper` and `otel_trace_group_prepper` entirely. To help migrate to 2.0,
+you can configure your trace pipeline using the Event model.