Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add semantic conventions for Elasticsearch client instrumentation #3358

Closed
wants to merge 13 commits into from
Closed
Show file tree
Hide file tree
Changes from 9 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
42 changes: 42 additions & 0 deletions semantic_conventions/trace/database.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -385,6 +385,47 @@ groups:
The collection being accessed within the database stated in `db.name`.
examples: [ 'customers', 'products' ]

- id: db.elasticsearch
prefix: db.elasticsearch
type: span
extends: db
brief: >
Call-level attributes for Elasticsearch
attributes:
- id: doc_id
type: string
requirement_level:
conditionally_required: when the request targets a specific document by id
tag: call-level-tech-specific
estolfo marked this conversation as resolved.
Show resolved Hide resolved
brief: >
The document that the request targets.
examples: [ '123', '456' ]
- id: url.query
type: string
requirement_level:
conditionally_required: when query params are provided as part of the request
tag: call-level-tech-specific
brief: >
The query params of the request, as a json string.
examples: [ '"{\"q\":\"test\"}", "{\"refresh\":true}"' ]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
examples: [ '"{\"q\":\"test\"}", "{\"refresh\":true}"' ]
examples: [ 'q=test&refresh=true' ]

- id: target
type: string
requirement_level:
conditionally_required: when a specific index or data stream is targeted by the request
tag: call-level-tech-specific
brief: >
The name of the data stream or index that is targeted.
examples: [ 'users' ]
- id: url.path
type: string
requirement_level: required
tag: call-level-tech-specific
brief: >
The path of the request, including the target and exact document id.
examples: [ '/test-index/_search', '/test-index/_doc/123' ]
- ref: http.method
requirement_level: required

- id: db.sql
prefix: 'db.sql'
type: span
Expand Down Expand Up @@ -517,3 +558,4 @@ groups:
- include: 'db.mongodb'
- include: 'db.sql'
- include: 'db.cosmosdb'
- include: 'db.elasticsearch'
1 change: 1 addition & 0 deletions specification/trace/semantic_conventions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ The following library-specific semantic conventions are defined:

* [AWS Lambda](instrumentation/aws-lambda.md): For AWS Lambda spans.
* [AWS SDK](instrumentation/aws-sdk.md): For AWS SDK spans.
* [Elasticsearch](instrumentation/elasticsearch.md): For Elasticsearch spans.
estolfo marked this conversation as resolved.
Show resolved Hide resolved
* [GraphQL](instrumentation/graphql.md): For GraphQL spans.

Apart from semantic conventions for traces and [metrics](../../metrics/semantic_conventions/README.md),
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Semantic conventions for Elasticsearch

**Status**: [Experimental](../../../document-status.md)

This document defines semantic conventions to apply when instrumenting requests to Elasticsearch. They map Elasticsearch
requests to attributes on a Span.

### Span Name
estolfo marked this conversation as resolved.
Show resolved Hide resolved

The **span name** SHOULD be of the format `<db.elasticsearch.method> <db.elasticsearch.url *with placeholders*>`.
estolfo marked this conversation as resolved.
Show resolved Hide resolved

The elasticsearch url is modified with placeholders in order to reduce the cardinality of the span name. When the url
contains a document id, it SHOULD be replaced by the identifier `{id}`. When the url contains a target data stream or
index, it SHOULD be replaced by `{target}`.
For example, a request to `/test-index/_doc/123` should have the span name `GET /{target}/_doc/{id}`.
estolfo marked this conversation as resolved.
Show resolved Hide resolved
When there is no target or document id, the span name will contain the exact url, as in `POST /_search`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when looking at a given elasticsearch url, what's the best way for instrumentation to detect which part of it should be replaced by {id} and which part should be replaced with {target}? is there a limited set elasticsearch url patterns?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, there is a file in the Elasticsearch specification repo that can be used.
As an example, the .NET client uses this generator to create this path lookup file that is used in the instrumentation.


### Span attributes
estolfo marked this conversation as resolved.
Show resolved Hide resolved

<!-- semconv elasticsearch -->
| Attribute | Type | Description | Examples | Requirement Level |
|----------------------------|---|----------------------------------------------------------------------|---------------------------------------------------------|------------------------|
| `db.elasticsearch.doc_id` | string | The document that the request targets, specified in the path. | `'123'` | Conditionally Required |
| `db.elasticsearch.target` | string | The name of the data stream or index that is targeted. | `'users'` | Conditionally Required |
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this also extracted from the path?

Suggested change
| `db.elasticsearch.target` | string | The name of the data stream or index that is targeted. | `'users'` | Conditionally Required |
| `db.elasticsearch.target` | string | The name of the data stream or index that is targeted, specified in the path. | `'users'` | Conditionally Required |

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That depends on the implementation of the client. It's possible that the client is written in way such that the target is specified as an argument or part of a hash passed to the function/method. The Ruby client, for example, has the target specified in the hash passed as an arg to the action method.
Example source / usage
So your suggested change makes sense; the target is determined not necessarily from the path, though it can appear as part of the path. I had this wording in there because I wanted to emphasize that the db.elasticsearch.target span attribute should match exactly what is in the path and what is replaced by {target} in the span name.

| `db.elasticsearch.url.path` | string | The exact path of the request, including the target and document id. | `'/test-index/_doc/123'` | Required |
| `db.elasticsearch.url.query` | string | The query params of the request, as a json string. | `'"{\"q\":\"test\"}", "{\"refresh\":true}"'` | Conditionally Required |
| `db.statement` | string | The request body, as a json string. [1] | `"{\"name\":\"TestUser\",\"password\":\"top_secret\"}"` | Conditionally Required |
| `http.method` | string | HTTP request method. | `GET`; `POST`; `HEAD` | Required |

**[1]:** The value may be sanitized to exclude sensitive information.
<!-- endsemconv -->