Skip to content

Commit

Permalink
Add Trace peer forwarder doc. (#3238)
Browse files Browse the repository at this point in the history
* Add trace peer forwarder doc.

Signed-off-by: carolxob <[email protected]>

* Updated file title.

Signed-off-by: carolxob <[email protected]>

* Minor updates based on tech review feedback.

Signed-off-by: carolxob <[email protected]>

* Fixed header info and links.

Signed-off-by: carolxob <[email protected]>

* Minor updates.

Signed-off-by: carolxob <[email protected]>

* Minor updates.

Signed-off-by: carolxob <[email protected]>

* Added questions as comments to ask SME.

Signed-off-by: carolxob <[email protected]>

* Added Trace analytics link.

Signed-off-by: carolxob <[email protected]>

* Minor update.

Signed-off-by: carolxob <[email protected]>

* Removed markdown comments and added them in GitHub.

Signed-off-by: carolxob <[email protected]>

* Made updates based on tech review feedback.

Signed-off-by: carolxob <[email protected]>

* Removed markdown comment.

Signed-off-by: carolxob <[email protected]>

* Minor edits.

Signed-off-by: carolxob <[email protected]>

* Minor change to unnecessary punctuation.

Signed-off-by: carolxob <[email protected]>

* Some feedback from doc review.

Signed-off-by: carolxob <[email protected]>

* Fixed broken links.

Signed-off-by: carolxob <[email protected]>

* Fixed links.

Signed-off-by: carolxob <[email protected]>

* Updates based on editorial feedback.

Signed-off-by: carolxob <[email protected]>

* Update _data-prepper/pipelines/configuration/processors/trace-peer-forwarder.md

Co-authored-by: Nathan Bower <[email protected]>

* Update _data-prepper/pipelines/configuration/processors/trace-peer-forwarder.md

Co-authored-by: Nathan Bower <[email protected]>

* Fixed hyperlinks to include data-prepper in path for peer forwarder.

Signed-off-by: carolxob <[email protected]>

---------

Signed-off-by: carolxob <[email protected]>
Co-authored-by: Nathan Bower <[email protected]>
  • Loading branch information
carolxob and natebower authored Apr 11, 2023
1 parent 25edf66 commit 8eac00e
Showing 1 changed file with 52 additions and 0 deletions.
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
---
layout: default
title: trace peer forwarder
parent: Processors
grand_parent: Pipelines
nav_order: 45
---

# trace peer forwarder

The `trace peer forwarder` processor is used with [peer forwarder]({{site.url}}{{site.baseurl}}/data-prepper/managing-data-prepper/peer-forwarder/) to reduce by half the number of events forwarded in a [Trace Analytics]({{site.url}}{{site.baseurl}}/data-prepper/common-use-cases/trace-analytics/) pipeline. In Trace Analytics, each event is typically duplicated when it is sent from `otel-trace-pipeline` to `raw-pipeline` and `service-map-pipeline`. When pipelines forward events, this causes the core peer forwarder to send multiple HTTP requests for the same event. You can use `trace peer forwarder` to forward an event once through the `otel-trace-pipeline` instead of `raw-pipeline` and `service-map-pipeline`, which prevents unnecessary HTTP requests.

You should use `trace peer forwarder` for Trace Analytics pipelines when you have multiple nodes.

## Usage

To get started with `trace peer forwarder`, first configure [peer forwarder]({{site.url}}{{site.baseurl}}/data-prepper/managing-data-prepper/peer-forwarder/). Then create a `pipeline.yaml` file and specify `trace peer forwarder` as the processor. You can configure `peer forwarder` in your `data-prepper-config.yaml` file. For more detailed information, see [Configuring Data Prepper]({{site.url}}{{site.baseurl}}/data-prepper/getting-started/#2-configuring-data-prepper).

See the following example `pipeline.yaml` file:

```yaml
otel-trace-pipeline:
delay: "100"
source:
otel_trace_source:
processor:
- trace_peer_forwarder:
sink:
- pipeline:
name: "raw-pipeline"
- pipeline:
name: "service-map-pipeline"
raw-pipeline:
source:
pipeline:
name: "entry-pipeline"
processor:
- otel_trace_raw:
sink:
- opensearch:
service-map-pipeline:
delay: "100"
source:
pipeline:
name: "entry-pipeline"
processor:
- service_map_stateful:
sink:
- opensearch:
```
In the preceding `pipeline.yaml` file, events are forwarded in the `otel-trace-pipeline` to the target peer, and no forwarding is performed in `raw-pipeline` or `service-map-pipeline`. This process helps improve network performance by forwarding events (as HTTP requests) once instead of twice.

0 comments on commit 8eac00e

Please sign in to comment.