Define component templates for APM

[simitt]

Fleet creates a custom component template by default for every datastream. Component templates can only overwrite settings and mappings that are not part of the _index_template's settings and mappings. Move all settings that should be changable by users into the component template definition in the apmpackage.

For example in the manifest define dynamic: false (to avoid dynamic indexing for unmapped fields) and add some index setting

elasticsearch:
  index_template:
    settings:
      number_of_replicas: 2
    mappings:
      dynamic: false

With the custom component, users can then define different settings and additional mappings.

---

POC

• Add the above mentioned elasticsearch.index_template to the traces manifest
• Start an environment that is consuming your local package (e.g. ./scripts/compose.py start master --with-elastic-agent --package-registry-url=http://host.docker.internal:8080), set up the apm package and add it to the agent policy
• add a different index setting to the traces-apm@custom component template and add some field mapping
• ingest some data (if you ingested data before, call the rollover API to ensure a new index is created POST traces-apm-default/_rollover)
• check the index settings and mappings (e.g. GET .ds-traces-apm-default-2021.07.14-000003/_settings)

---

related Kibana bug elastic/kibana#105603

[Mpdreamz]

Linking this ES feature request for awareness both ways: elastic/elasticsearch#75031

[axw]

I started looking into this, but it's unclear to me what the expected outcome is. If you specify index_template like in the description, for traces-apm-*, you end up with two additional component templates:

traces-apm@mappings

{
  "dynamic": false
}

traces-apm@settings

{
  "index": {
    "number_of_replicas": "2"
  }
}

• those properties are not defined in the index template anyway, so why are we doing this?
• AFAICT the addition of these component templates does not affect the index template at all, so I don't see how we can "Move all settings that should be changable by users into the component template definition in the apmpackage." without changes to Fleet

What am I missing?

[Mpdreamz]

It should link the following 4 component templates at install time:

https://github.com/elastic/kibana/blob/3864fe1559281b4a2731e3a9439c501125e65669/x-pack/test/fleet_api_integration/apis/epm/install_overrides.ts#L47

AFAICT the addition of these component templates does not affect the index template at all

You don't see these added in the composed_of section of the index template?

[axw]

You don't see these added in the composed_of section of the index template?

I did see the component templates, but they didn't contain the default mappings/settings.

What am I missing?

I was missing the bug linked at the bottom of the description: elastic/kibana#105603

[axw]

Moving this back to blocked for 7.16, until elastic/kibana#105603 is fixed.

[axw]

No longer blocked.

[axw]

Using the latest 8.0.0-SNAPSHOT Kibana image, here's how templates look now.

traces-apm index template settings:

traces-apm@settings component template:

I then edited traces-apm@custom, adding:

• {"index": {"number_of_shards": "2"}} to settings
• span.db.statement as a text field to mappings

After initiating a rollover, I confirm that these have been applied to the new index.

@simitt the fields defined in the package are added to the index template, not to the @mappings component template. Is this what you expected? I think it's fine. I think we need to be able to have some baseline guarantees about the mappings. Users should be free to add mappings, but modifying packaged-defined mappings may break the UI.

[simitt]

Thanks for following up on that. Yes agreed, having the mappings in the index template makes sense. The important part is that users can add mappings and that they can modify settings. So LGTM.

[axw]

Great, I think we can call this done then.