[Profiling] Switch to OTEL cloud.provider, cloud.region, host.type #106656
Conversation
rockdaboot
added
>enhancement
:UniversalProfiling/Application
elasticsearchmachine
added
the
external-contributor
|
Pinging @elastic/obs-knowledge-team (Team:obs-knowledge) |
|
|
||
| // history for individual indices / index templates. Only bump these for breaking changes that require to create a new index | ||
| public static final int PROFILING_EVENTS_VERSION = 2; | ||
| public static final int PROFILING_EXECUTABLES_VERSION = 1; | ||
| public static final int PROFILING_METRICS_VERSION = 1; | ||
| public static final int PROFILING_HOSTS_VERSION = 1; | ||
| public static final int PROFILING_HOSTS_VERSION = 2; |
There was a problem hiding this comment.
I decided for a roll-over since it's easier to argue about changes to be done in the collector.
x-pack/plugin/profiling/src/test/java/org/elasticsearch/xpack/profiling/HostMetadataTests.java
Outdated
Show resolved
Hide resolved
| @@ -47,13 +47,14 @@ public class ProfilingIndexTemplateRegistry extends IndexTemplateRegistry { | |||
| // version 4: Added 'service.name' keyword mapping to profiling-events | |||
| // version 5: Add optional component template '<idx-name>@custom' to all index templates that reference component templates | |||
| // version 6: Added 'host.arch' keyword mapping to profiling-hosts | |||
| public static final int INDEX_TEMPLATE_VERSION = 6; | |||
| // version 7: Added 'host.type', 'cloud.provider', 'cloud.region' keyword mappings to profiling-hosts | |||
| public static final int INDEX_TEMPLATE_VERSION = 7; | |||
There was a problem hiding this comment.
I missed this in the last review but it is actually not required that we bump the index template version on every change. Instead, it is sufficient to bump it once per release because no cluster can observe the intermediate index template versions.
Long story short: Can you please change this value to 5 (which applies to 8.14.0) and merge the comments accordingly to include all changes that went into version 5?
There was a problem hiding this comment.
That is a good point I already thought of. My conclusion was that I do not know whether version 5 or 6 has already been deployed (locally) and used by developers. And if you don't know that, why risking to break things even if it is just one or two people?
What is the downside of increasing the version several times per release cycle?
There was a problem hiding this comment.
As discussed offline, it's less risky to bump the version all the time, even if this means version jumps between releases (which are harmless) so let's stick to this convention in the future.
|
cc @christos68k FYA |
Description
The profiling agent is going to be donated to OTEL.
We currently don't have an own field for the provider name and derive the provider name from other existing fields. The OTEL semantic conventions suggest to use
cloud.provideras field name.We currently don't have an own field for the cloud region and check different provider-specific fields for the region. For some providers, this even requires string parsing.
Ideally, the host agent sends the normalized values as
cloud.region. This would be in sync with the OTEL semantic conventions.We currently don't have an own field for the cloud instance type and check different provider-specific fields. For some providers, this even requires string parsing.
Ideally, the host agent sends the normalized values as
host.type. This would be in sync with the OTEL semantic conventions.Fixes #106625