From 30559040ed621ea0798fb670617b028d78a2d915 Mon Sep 17 00:00:00 2001 From: carolxob Date: Mon, 6 Feb 2023 14:40:33 -0700 Subject: [PATCH 1/3] Corrected capitalization of peer forwarder. Signed-off-by: carolxob --- _data-prepper/peer-forwarder.md | 34 ++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/_data-prepper/peer-forwarder.md b/_data-prepper/peer-forwarder.md index 7f4ba14a3b..2a6b874c54 100644 --- a/_data-prepper/peer-forwarder.md +++ b/_data-prepper/peer-forwarder.md @@ -1,14 +1,14 @@ --- layout: default -title: Peer Forwarder +title: Peer forwarder nav_order: 12 --- # Peer Forwarder -Peer Forwarder is an HTTP service that performs peer forwarding of an `event` between Data Prepper nodes for aggregation. This HTTP service uses a hash-ring approach to aggregate events and determine which Data Prepper node it should handle on a given trace before rerouting it to that node. Currently, Peer Forwarder is supported by the `aggregate`, `service_map_stateful`, and `otel_trace_raw` [processors]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/processors/). +Peer forwarder is an HTTP service that performs peer forwarding of an `event` between Data Prepper nodes for aggregation. This HTTP service uses a hash-ring approach to aggregate events and determine which Data Prepper node it should handle on a given trace before rerouting it to that node. Currently, Peer forwarder is supported by the `aggregate`, `service_map_stateful`, and `otel_trace_raw` [processors]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/processors/). -Peer Forwarder groups events based on the identification keys provided by the supported processors. For `service_map_stateful` and `otel_trace_raw`, the identification key is `traceId` by default and cannot be configured. The `aggregate` processor is configured using the `identification_keys` configuration option. From here, you can specify which keys to use for Peer Forwarder. See [Aggregate Processor page](https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/aggregate-processor#identification_keys) for more information about identification keys. +Peer forwarder groups events based on the identification keys provided by the supported processors. For `service_map_stateful` and `otel_trace_raw`, the identification key is `traceId` by default and cannot be configured. The `aggregate` processor is configured using the `identification_keys` configuration option. From here, you can specify which keys to use for Peer forwarder. See [Aggregate Processor page](https://github.com/opensearch-project/data-prepper/tree/main/data-prepper-plugins/aggregate-processor#identification_keys) for more information about identification keys. Peer discovery allows Data Prepper to find other nodes that it will communicate with. Currently, peer discovery is provided by a static list, a DNS record lookup, or AWS Cloud Map. @@ -40,7 +40,7 @@ peer_forwarder: [AWS Cloud Map](https://docs.aws.amazon.com/cloud-map/latest/dg/what-is-cloud-map.html) provides API-based service discovery as well as DNS-based service discovery. -Peer Forwarder can use the API-based service discovery in AWS Cloud Map. To support this, you must have an existing namespace configured for API instance discovery. You can create a new one by following the instructions provided by the [AWS Cloud Map documentation](https://docs.aws.amazon.com/cloud-map/latest/dg/working-with-namespaces.html). +Peer forwarder can use the API-based service discovery in AWS Cloud Map. To support this, you must have an existing namespace configured for API instance discovery. You can create a new one by following the instructions provided by the [AWS Cloud Map documentation](https://docs.aws.amazon.com/cloud-map/latest/dg/working-with-namespaces.html). Your Data Prepper configuration needs to include the following: * `aws_cloud_map_namespace_name` – Set to your AWS Cloud Map namespace name. @@ -91,11 +91,11 @@ The following table provides optional configuration values. | Value | Type | Description | | ---- | --- | ----------- | -| `port` | Integer | A value between 0 and 65535 that represents the port that the Peer Forwarder server is running on. Default value is `4994`. | -| `request_timeout` | Integer | Represents the request timeout duration in milliseconds for the Peer Forwarder HTTP server. Default value is `10000`. | -| `server_thread_count` | Integer | Represents the number of threads used by the Peer Forwarder server. Default value is `200`.| -| `client_thread_count` | Integer | Represents the number of threads used by the Peer Forwarder client. Default value is `200`.| -| `maxConnectionCount` | Integer | Represents the maximum number of open connections for the Peer Forwarder server. Default value is `500`. | +| `port` | Integer | A value between 0 and 65535 that represents the port that the peer forwarder server is running on. Default value is `4994`. | +| `request_timeout` | Integer | Represents the request timeout duration in milliseconds for the peer forwarder HTTP server. Default value is `10000`. | +| `server_thread_count` | Integer | Represents the number of threads used by the peer forwarder server. Default value is `200`.| +| `client_thread_count` | Integer | Represents the number of threads used by the peer forwarder client. Default value is `200`.| +| `maxConnectionCount` | Integer | Represents the maximum number of open connections for the peer forwarder server. Default value is `500`. | | `discovery_mode` | String | Represents the peer discovery mode to be used. Allowable values are `local_node`, `static`, `dns`, and `aws_cloud_map`. Defaults to `local_node`, which processes events locally. | | `static_endpoints` | List | Contains the endpoints of all Data Prepper instances. Required if `discovery_mode` is set to `static`. | | `domain_name` | String | Represents the single domain name to query DNS against. Typically used by creating multiple [DNS A records](https://www.cloudflare.com/learning/dns/dns-records/dns-a-record/) for the same domain. Required if `discovery_mode` is set to `dns`. | @@ -105,11 +105,11 @@ The following table provides optional configuration values. | `buffer_size` | Integer | Represents the maximum number of unchecked records the buffer accepts (the number of unchecked records equals the number of records written into the buffer plus the number of records that are still processing and not yet checked by the Checkpointing API). Default is `512`. | | `batch_size` | Integer | Represents the maximum number of records that the buffer returns on read. Default is `48`. | | `aws_region` | String | Represents the AWS Region that uses `ACM`, `Amazon S3`, or `AWS Cloud Map` and is required when any of the following conditions are met:
- The `use_acm_certificate_for_ssl` setting is set to `true`.
- Either `ssl_certificate_file` or `ssl_key_file` specifies an Amazon Simple Storage Service (Amazon S3) URI (for example, s3://mybucket/path/to/public.cert).
- The `discovery_mode` is set to `aws_cloud_map`. | -| `drain_timeout` | Duration | Represents the amount of time that Peer Forwarder will wait to complete data processing before shutdown. | +| `drain_timeout` | Duration | Represents the amount of time that peer forwarder will wait to complete data processing before shutdown. | ## SSL configuration -The following table provides optional SSL configuration values that allow you to set up a trust manager for the Peer Forwarder client in order to connect to other Data Prepper instances. +The following table provides optional SSL configuration values that allow you to set up a trust manager for the peer forwarder client in order to connect to other Data Prepper instances. | Value | Type | Description | | ----- | ---- | ----------- | @@ -147,14 +147,14 @@ peer_forwarder: ## Metrics -Core Peer Forwarder introduces the following custom metrics. All the metrics are prefixed by `core.peerForwarder`. +Core peer forwarder introduces the following custom metrics. All the metrics are prefixed by `core.peerForwarder`. ### Timer -Peer Forwarder's timer capability provides the following information: +Peer forwarder's timer capability provides the following information: -- `requestForwardingLatency`: Measures latency of requests forwarded by the Peer Forwarder client. -- `requestProcessingLatency`: Measures latency of requests processed by the Peer Forwarder server. +- `requestForwardingLatency`: Measures latency of requests forwarded by the Peer forwarder client. +- `requestProcessingLatency`: Measures latency of requests processed by the Peer forwarder server. ### Counter @@ -165,8 +165,8 @@ The following table provides counter metric options. | `requests`| Measures the total number of forwarded requests. | | `requestsFailed`| Measures the total number of failed requests. Applies to requests with an HTTP response code other than `200`. | | `requestsSuccessful`| Measures the total number of successful requests. Applies to requests with HTTP response code `200`. | -| `requestsTooLarge`| Measures the total number of requests that are too large to be written to the Peer Forwarder buffer. Applies to requests with HTTP response code `413`. | -| `requestTimeouts`| Measures the total number of requests that time out while writing content to the Peer Forwarder buffer. Applies to requests with HTTP response code `408`. | +| `requestsTooLarge`| Measures the total number of requests that are too large to be written to the Peer forwarder buffer. Applies to requests with HTTP response code `413`. | +| `requestTimeouts`| Measures the total number of requests that time out while writing content to the Peer forwarder buffer. Applies to requests with HTTP response code `408`. | | `requestsUnprocessable`| Measures the total number of requests that fail due to an unprocessable entity. Applies to requests with HTTP response code `422`. | | `badRequests`| Measures the total number of requests with a bad request format. Applies to requests with HTTP response code `400`. | | `recordsSuccessfullyForwarded`| Measures the total number of successfully forwarded records. | From 121c89c697dd16ded54b1f00d369c88aefc75695 Mon Sep 17 00:00:00 2001 From: carolxob Date: Tue, 7 Feb 2023 09:35:18 -0700 Subject: [PATCH 2/3] Updated first heading. Signed-off-by: carolxob --- _data-prepper/peer-forwarder.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/peer-forwarder.md b/_data-prepper/peer-forwarder.md index 2a6b874c54..5255119fbf 100644 --- a/_data-prepper/peer-forwarder.md +++ b/_data-prepper/peer-forwarder.md @@ -4,7 +4,7 @@ title: Peer forwarder nav_order: 12 --- -# Peer Forwarder +# Peer forwarder Peer forwarder is an HTTP service that performs peer forwarding of an `event` between Data Prepper nodes for aggregation. This HTTP service uses a hash-ring approach to aggregate events and determine which Data Prepper node it should handle on a given trace before rerouting it to that node. Currently, Peer forwarder is supported by the `aggregate`, `service_map_stateful`, and `otel_trace_raw` [processors]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/processors/). From 97905b81b6388c6bbda5c40f6bd7fa9d6efbf5be Mon Sep 17 00:00:00 2001 From: Naarcha-AWS Date: Wed, 8 Feb 2023 16:41:27 -0600 Subject: [PATCH 3/3] Fix capitilization on sentence Signed-off-by: Naarcha-AWS --- _data-prepper/managing-data-prepper/peer-forwarder.md | 7 ------- 1 file changed, 7 deletions(-) diff --git a/_data-prepper/managing-data-prepper/peer-forwarder.md b/_data-prepper/managing-data-prepper/peer-forwarder.md index 8e43921f42..c5888b6d90 100644 --- a/_data-prepper/managing-data-prepper/peer-forwarder.md +++ b/_data-prepper/managing-data-prepper/peer-forwarder.md @@ -152,17 +152,10 @@ Core peer forwarder introduces the following custom metrics. All the metrics are ### Timer -<<<<<<< HEAD:_data-prepper/peer-forwarder.md Peer forwarder's timer capability provides the following information: -- `requestForwardingLatency`: Measures latency of requests forwarded by the Peer forwarder client. -- `requestProcessingLatency`: Measures latency of requests processed by the Peer forwarder server. -======= -peer forwarder's timer capability provides the following information: - - `requestForwardingLatency`: Measures latency of requests forwarded by the peer forwarder client. - `requestProcessingLatency`: Measures latency of requests processed by the peer forwarder server. ->>>>>>> main:_data-prepper/managing-data-prepper/peer-forwarder.md ### Counter