[1.x] Tracing fields should be at the root (#1165)

* Add notice to the tracing field set, about not nesting field names. (#1162)
* Tracing fields should be at top level in Beats artifact (#1164)

CHANGELOG.next.md

@@ -30,6 +30,8 @@ Thanks, you're awesome :-) -->
 
 #### Bugfixes
 
+* `tracing` fields should be at root of Beats `fields.ecs.yml` artifacts. #1164
+
 #### Added
 
 * Added the `path` key when type is `alias`, to support the [alias field type](https://www.elastic.co/guide/en/elasticsearch/reference/current/alias.html). #877
@@ -41,6 +43,9 @@ Thanks, you're awesome :-) -->
 
 #### Improvements
 
+* Added a notice highlighting that the `tracing` fields are not nested under the
+  namespace `tracing.` #1162
+
 #### Deprecated
 
 

docs/field-details.asciidoc

@@ -7187,6 +7187,8 @@ example: `tls`
 
 Distributed tracing makes it possible to analyze performance throughout a microservice architecture all in one view. This is accomplished by tracing all of the requests - from the initial web request in the front-end service - to queries made through multiple back-end services.
 
+Unlike most field sets in ECS, the tracing fields are *not* nested under the field set name. In other words, the correct field name is `trace.id`, not `tracing.trace.id`, and so on.
+
 [discrete]
 ==== Tracing Field Details
 

experimental/generated/beats/fields.ecs.yml

@@ -5212,43 +5212,34 @@
       description: Normalized lowercase protocol name parsed from original string.
       example: tls
       default_field: false
-  - name: tracing
-    title: Tracing
-    group: 2
-    description: Distributed tracing makes it possible to analyze performance throughout
-      a microservice architecture all in one view. This is accomplished by tracing
-      all of the requests - from the initial web request in the front-end service
-      - to queries made through multiple back-end services.
-    type: group
-    fields:
-    - name: span.id
-      level: extended
-      type: keyword
-      ignore_above: 1024
-      description: 'Unique identifier of the span within the scope of its trace.
-
-        A span represents an operation within a transaction, such as a request to
-        another service, or a database query.'
-      example: 3ff9a8981b7ccd5a
-      default_field: false
-    - name: trace.id
-      level: extended
-      type: keyword
-      ignore_above: 1024
-      description: 'Unique identifier of the trace.
+  - name: span.id
+    level: extended
+    type: keyword
+    ignore_above: 1024
+    description: 'Unique identifier of the span within the scope of its trace.
+
+      A span represents an operation within a transaction, such as a request to another
+      service, or a database query.'
+    example: 3ff9a8981b7ccd5a
+    default_field: false
+  - name: trace.id
+    level: extended
+    type: keyword
+    ignore_above: 1024
+    description: 'Unique identifier of the trace.
 
-        A trace groups multiple events like transactions that belong together. For
-        example, a user request handled by multiple inter-connected services.'
-      example: 4bf92f3577b34da6a3ce929d0e0e4736
-    - name: transaction.id
-      level: extended
-      type: keyword
-      ignore_above: 1024
-      description: 'Unique identifier of the transaction within the scope of its trace.
+      A trace groups multiple events like transactions that belong together. For example,
+      a user request handled by multiple inter-connected services.'
+    example: 4bf92f3577b34da6a3ce929d0e0e4736
+  - name: transaction.id
+    level: extended
+    type: keyword
+    ignore_above: 1024
+    description: 'Unique identifier of the transaction within the scope of its trace.
 
-        A transaction is the highest level of work measured within a service, such
-        as a request to a server.'
-      example: 00f067aa0ba902b7
+      A transaction is the highest level of work measured within a service, such as
+      a request to a server.'
+    example: 00f067aa0ba902b7
   - name: url
     title: URL
     group: 2

experimental/generated/ecs/ecs_nested.yml

@@ -9228,10 +9228,14 @@ tls:
   title: TLS
   type: group
 tracing:
-  description: Distributed tracing makes it possible to analyze performance throughout
+  description: 'Distributed tracing makes it possible to analyze performance throughout
     a microservice architecture all in one view. This is accomplished by tracing all
     of the requests - from the initial web request in the front-end service - to queries
     made through multiple back-end services.
+
+    Unlike most field sets in ECS, the tracing fields are *not* nested under the field
+    set name. In other words, the correct field name is `trace.id`, not `tracing.trace.id`,
+    and so on.'
   fields:
     span.id:
       dashed_name: span-id

generated/beats/fields.ecs.yml

@@ -5299,43 +5299,34 @@
       description: Normalized lowercase protocol name parsed from original string.
       example: tls
       default_field: false
-  - name: tracing
-    title: Tracing
-    group: 2
-    description: Distributed tracing makes it possible to analyze performance throughout
-      a microservice architecture all in one view. This is accomplished by tracing
-      all of the requests - from the initial web request in the front-end service
-      - to queries made through multiple back-end services.
-    type: group
-    fields:
-    - name: span.id
-      level: extended
-      type: keyword
-      ignore_above: 1024
-      description: 'Unique identifier of the span within the scope of its trace.
-
-        A span represents an operation within a transaction, such as a request to
-        another service, or a database query.'
-      example: 3ff9a8981b7ccd5a
-      default_field: false
-    - name: trace.id
-      level: extended
-      type: keyword
-      ignore_above: 1024
-      description: 'Unique identifier of the trace.
+  - name: span.id
+    level: extended
+    type: keyword
+    ignore_above: 1024
+    description: 'Unique identifier of the span within the scope of its trace.
+
+      A span represents an operation within a transaction, such as a request to another
+      service, or a database query.'
+    example: 3ff9a8981b7ccd5a
+    default_field: false
+  - name: trace.id
+    level: extended
+    type: keyword
+    ignore_above: 1024
+    description: 'Unique identifier of the trace.
 
-        A trace groups multiple events like transactions that belong together. For
-        example, a user request handled by multiple inter-connected services.'
-      example: 4bf92f3577b34da6a3ce929d0e0e4736
-    - name: transaction.id
-      level: extended
-      type: keyword
-      ignore_above: 1024
-      description: 'Unique identifier of the transaction within the scope of its trace.
+      A trace groups multiple events like transactions that belong together. For example,
+      a user request handled by multiple inter-connected services.'
+    example: 4bf92f3577b34da6a3ce929d0e0e4736
+  - name: transaction.id
+    level: extended
+    type: keyword
+    ignore_above: 1024
+    description: 'Unique identifier of the transaction within the scope of its trace.
 
-        A transaction is the highest level of work measured within a service, such
-        as a request to a server.'
-      example: 00f067aa0ba902b7
+      A transaction is the highest level of work measured within a service, such as
+      a request to a server.'
+    example: 00f067aa0ba902b7
   - name: url
     title: URL
     group: 2

generated/ecs/ecs_nested.yml

@@ -9316,10 +9316,14 @@ tls:
   title: TLS
   type: group
 tracing:
-  description: Distributed tracing makes it possible to analyze performance throughout
+  description: 'Distributed tracing makes it possible to analyze performance throughout
     a microservice architecture all in one view. This is accomplished by tracing all
     of the requests - from the initial web request in the front-end service - to queries
     made through multiple back-end services.
+
+    Unlike most field sets in ECS, the tracing fields are *not* nested under the field
+    set name. In other words, the correct field name is `trace.id`, not `tracing.trace.id`,
+    and so on.'
   fields:
     span.id:
       dashed_name: span-id

schemas/tracing.yml

@@ -6,7 +6,12 @@
   short: Fields related to distributed tracing.
   description: >
     Distributed tracing makes it possible to analyze performance throughout a microservice architecture all in one view.
-    This is accomplished by tracing all of the requests - from the initial web request in the front-end service - to queries made through multiple back-end services.
+    This is accomplished by tracing all of the requests - from the initial web
+    request in the front-end service - to queries made through multiple back-end services.
+
+    Unlike most field sets in ECS, the tracing fields are *not* nested under the
+    field set name. In other words, the correct field name is `trace.id`,
+    not `tracing.trace.id`, and so on.
   type: group
   fields:
     - name: trace.id

scripts/generators/beats.py

@@ -17,6 +17,11 @@ def generate(ecs_nested, ecs_version, out_dir):
             continue
         fieldset = ecs_nested[fieldset_name]
 
+        # Handle when `root:true`
+        if fieldset.get('root', False):
+            beats_fields.extend(fieldset_field_array(fieldset['fields'], df_whitelist, fieldset['prefix']))
+            continue
+
         beats_field = ecs_helpers.dict_copy_keys_ordered(fieldset, allowed_fieldset_keys)
         beats_field['fields'] = fieldset_field_array(fieldset['fields'], df_whitelist, fieldset['prefix'])
         beats_fields.append(beats_field)