Add final pipeline to base package

[ruflin]

The current idea is that all events will flow through a final pipeline. This pipeline needs to be defined and set on each template with the index.final_pipeline setting.

This final pipeline should be part of the base package and can be empty at first.

[ruflin]

Work is ongoing to add a final pipeline to Beats here: elastic/beats#14001

[ph]

@ruflin We need that for b1 to trace events coming from agents and endpoint.

[ruflin]

@ph How takes a lead on defining this final pipeline? If we want to have it on all events, it must be shipped by Elasticsearch.

[ph]

Will take a stab at it.

cc @dakrone for FYI.

[dakrone]

@ruflin @ph I thought our original consensus was to skip a pipeline for now, since we don't have a need yet? Is this something necessary to add to the default bundled templates in elastic/elasticsearch#57629 ?

[ruflin]

@dakrone I must confess i forgot about the final pipeline. I suggest we get your PR in without it and figure out if we need it later. An option here is also that only if ingest manager updates the base package, the pipeline exists.

[ph]

Lets sync up with endpoint if this is a requirement.

[mtojek]

I think we can resolve this issue now. We don't have the base package.