Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds Jaeger trace data for analytics documentation #2374

Merged
merged 35 commits into from
Jan 17, 2023
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
d5c71a5
for new page how to analyze Jaeger trace data
alicejw1 Jan 11, 2023
9c5be21
remove old image
alicejw1 Jan 11, 2023
660c5e3
for new information and doc writer checklist
alicejw1 Jan 11, 2023
86cf05e
for new information and doc writer checklist
alicejw1 Jan 11, 2023
4bfa459
small rewrite
alicejw1 Jan 11, 2023
01e30ff
new clean images from Dashboards URL directly
alicejw1 Jan 11, 2023
cdd04d6
for additional information
alicejw1 Jan 11, 2023
c320d66
remove blank lines
alicejw1 Jan 11, 2023
b1c1551
for tech review feedback updates
alicejw1 Jan 11, 2023
87241b7
add requirements section
alicejw1 Jan 11, 2023
35f3784
for new procedure
alicejw1 Jan 11, 2023
3c4b4d7
for tech review feedback updates
alicejw1 Jan 11, 2023
1fd987b
continued updates
alicejw1 Jan 12, 2023
0bac730
for docker compose file instructions
alicejw1 Jan 12, 2023
d1b5c5a
for docker usage instruction
alicejw1 Jan 12, 2023
2af0f4e
for step 2 view dashboards
alicejw1 Jan 12, 2023
3cbbc61
for additional link provided in tech review
alicejw1 Jan 12, 2023
1d34e2d
for link to index page to introduce the feature
alicejw1 Jan 12, 2023
7ed601b
final checklist
alicejw1 Jan 12, 2023
ae75639
add warning not to use sample file in prod env
alicejw1 Jan 12, 2023
5220a61
updated docker file that is safe for prod env, remove warning note fo…
alicejw1 Jan 12, 2023
8042c96
for small update to parent page
alicejw1 Jan 12, 2023
0bd55fb
for tech review
alicejw1 Jan 12, 2023
f6a9757
typo fix for font
alicejw1 Jan 12, 2023
6fb5d02
for doc review #1 feedback updates
alicejw1 Jan 12, 2023
7e14e4d
for doc review feedback #2 updates
alicejw1 Jan 12, 2023
2b18d99
for a couple minor changes
alicejw1 Jan 12, 2023
fa8efe1
spell out dashboard URI directly to trace analytics for accessibility…
alicejw1 Jan 12, 2023
cfd0152
need to add additional step from eng to generate sample data
alicejw1 Jan 12, 2023
9bd9903
for additional step image of sample app
alicejw1 Jan 12, 2023
85cae77
rename step numbers
alicejw1 Jan 12, 2023
36e2b98
minor fix heading levels
alicejw1 Jan 12, 2023
360eba6
updates recommended by the editorial reviewer
alicejw1 Jan 17, 2023
7763331
clarify Spans window function
alicejw1 Jan 17, 2023
cc6f1fa
clarified individual trace details section
alicejw1 Jan 17, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions _observability-plugin/trace/trace-analytics-jaeger.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,85 @@
---
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved
layout: default
title: Analyze Jaeger trace data
parent: Trace analytics
nav_order: 55
---

# Analyze Jaeger trace data
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

Introduced 2.5
{: .label .label-purple }
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

Trace analytics now supports Jaeger trace data. Jaeger can store its trace data in OpenSearch, but users that use OpenSearch as their backend for Jaeger data can now benefit from trace analytics capability.

When you perform trace analytics, you can select from two data sources:

- **Data Prepper** – Data ingested into OpenSearch through Data Prepper. This data source requires OpenTelemetry (OTEL) index type.
- **Jaeger** – Trace data stored within OpenSearch as its backend. This data source requires the Jaeger index type.

If you currently store your Jaeger trace data in OpenSearch, you can now use the trace analytics capability. When you ingest Jaeger data into OpenSearch, it gets stored in a different index than the OTA-generated index that gets created when you run data through the Data Prepper. You can indicate which data source on which you want to perform trace analytics with the data source selector in the Dashboards.

Jaeger trace data that you can analyze includes span data, as well as service and operation endpoint data. Jaeger span data analysis requires some configuration.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

Each time you ingest data for Jaeger, it creates a separate index for that day. The Dashboards will show the current index that has a mapping.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

To learn more about Jaeger data tracing, see the [Jaeger](https://www.jaegertracing.io/) open source documentation.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

## Requirements

Jaeger data that is ingested for OpenSearch needs to have the `--es.tags-as-fields.all=true` flag set for errors. If data is not ingested in this format it will not work for trace analytics with OpenSearch.

### About Data ingestion with Jaeger indexes
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

Trace analytics for non-Jaeger data use OTEL indexes with the naming conventions `otel-v1-apm-span-*` or `otel-v1-apm-service-map*`.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

Jaeger indexes follow the naming conventions `jaeger-span-*` or `jaeger-service-*`.

Jaeger and OTEL indexes have different field names. Therefore when you run trace analytics, you'll need to create different queries and components depending on which index type you are using.
{:.note}
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

## Use trace analytics in OpenSearch Dashboards
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

To analyze your Jaeger trace data in the Dashboards, you need to set up Trace Analytics first. To get started, see [Get started with Trace Analytics]({{site.url}}{{site.baseurl}}/observability-plugin/trace/get-started/).
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

### Data sources

You can specify either Data Prepper or Jaeger as your data source when you perform trace analytics.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved
From the OpenSearch Dashboards, go to **Observability > Trace Analytics** and select Jaeger.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

![Select data source]({{site.url}}{{site.baseurl}}/images/trace-analytics/select-data.png)

## Dashboards views

After you select Jaeger for the data source, you can view all of your indexed data in the Dashboards, including Error rate, Throughput, and Services.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved
### Error rate

You can view the Trace error rate over time in the Dashboard, and also view the top five combinations of services and operations that have a non-zero error rate.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

![Error rate]({{site.url}}{{site.baseurl}}/images/trace-analytics/error-rate.png)

### Throughput

With **Throughput** selected, you can see the throughput of traces on Jaeger indexes that are coming in over time.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

You can select an individual Trace from **Top 5 Service and Operation Latency** list and view the detailed trace data.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

![Throughput]({{site.url}}{{site.baseurl}}/images/trace-analytics/throughput.png)

You can also see the combinations of services and operations that have the highest latency.

If you select one of the entries for Service and Operation Name and go to the **Traces** column to select a trace, it will add the service and operation as filters for you.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

In **Traces**, you can see the latency and errors for the filtered service and operation for each individual Trace ID in the list.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

![Select data source]({{site.url}}{{site.baseurl}}/images/trace-analytics/service-trace-data.png)

If you select an individual Trace ID, you can see more detailed information about the trace, such as time spent by the service and each span for the service and operation. You can also view the payload that you get from the index in JSON format.
alicejw1 marked this conversation as resolved.
Show resolved Hide resolved

![Select data source]({{site.url}}{{site.baseurl}}/images/trace-analytics/trace-details.png)

### Services

You can also look at individual error rates and latency for each individual service. Go to **Observability > Trace Analytics > Services**. In **Services**, you can see the average latency, error rate, throughput and trace for each service in the list.

![Services list]({{site.url}}{{site.baseurl}}/images/trace-analytics/services-jaeger.png)
Binary file added images/trace-analytics/error-rate.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/trace-analytics/select-data.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/trace-analytics/service-trace-data.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/trace-analytics/services-jaeger.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/trace-analytics/throughput.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added images/trace-analytics/trace-details.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.