Cherry-pick #20839 to 7.x: [Elastic Agent] Add support for variable replacement from providers

[blakerouse]

Cherry-pick of PR #20839 to 7.x branch. Original message:

What does this PR do?

Adds support for replacing {{local_dynamic.key|local_dynamic.other|'fallback'}} inside of the Elastic Agent inputs configuration.

Why is it important?

Finishes the core support for #20781

More code is needed to help debugging effort.

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• [ ] I have made corresponding changes to the documentation
• [ ] I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works
• I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

How to test this PR locally

Running the Elastic Agent in local mode, use the following configuration:

outputs:
  default:
    type: elasticsearch
    hosts: [127.0.0.1:9200]
    username: elastic
    password: changeme

providers:
  local_dynamic:
    items:
      - vars:
          key: value1
        processors:
          - add_fields:
              fields:
                custom: match1
              target: dynamic
      - vars:
          key: value2
        processors:
          - add_fields:
              fields:
                custom: match2
              target: dynamic
      - vars:
          key: value3
        processors:
          - add_fields:
              fields:
                custom: match3
              target: dynamic

inputs:
  - type: logfile
    enabled: true
    streams:
      - paths:
          - /var/log/{{local_dynamic.key}}

Then run the following to inspect the generated configuration:

$ ./elastic_agent inspect output -o default
[default] filebeat:
filebeat:
  inputs:
  - index: logs-generic-default
    paths:
    - /var/log/value1
    processors:
    - add_fields:
        fields:
          custom: match1
        target: dynamic
    - add_fields:
        fields:
          dataset: generic
          namespace: default
          type: logs
        target: data_stream
    - add_fields:
        fields:
          dataset: generic
        target: event
    type: log
  - index: logs-generic-default
    paths:
    - /var/log/value2
    processors:
    - add_fields:
        fields:
          custom: match2
        target: dynamic
    - add_fields:
        fields:
          dataset: generic
          namespace: default
          type: logs
        target: data_stream
    - add_fields:
        fields:
          dataset: generic
        target: event
    type: log
  - index: logs-generic-default
    paths:
    - /var/log/value3
    processors:
    - add_fields:
        fields:
          custom: match3
        target: dynamic
    - add_fields:
        fields:
          dataset: generic
          namespace: default
          type: logs
        target: data_stream
    - add_fields:
        fields:
          dataset: generic
        target: event
    type: log
output:
  elasticsearch:
    hosts:
    - 127.0.0.1:9200
    password: changeme
    username: elastic

---
[default] FLEET_MONITORING:
output:
  elasticsearch:
    hosts:
    - 127.0.0.1:9200
    password: changeme
    type: elasticsearch
    username: elastic
programs:
- filebeat

---

Related issues

• Relates [Elastic Agent] Support variable substitution from providers #20781

[elasticmachine]

Pinging @elastic/ingest-management (Team:Ingest Management)

[elasticmachine]

💚 Build Succeeded

Expand to view the summary

Build stats

• Build Cause: [Pull request #20964 updated]

• Start Time: 2020-09-07T13:54:01.458+0000

• Duration: 34 min 42 sec

Test stats 🧪

Test	Results
Failed	0
Passed	1691
Skipped	3
Total	1694

[michalpristas]

Backport looks ok