Remove index management, always use data streams

[axw]

Motivation/summary

• Remove apm-server.ilm.* config
• Remove apm-server.data_streams.enabled config
• Remove setup.* config
• Remove output.elasticsearch.{index,indices} config
• Stop building, including, and using libbeat fields

Checklist

• Update CHANGELOG.asciidoc
  - [ ] Documentation has been updated Doc updates being handled separately

How to test these changes

1. Run apm-server without any special config, other than Elasticsearch host & credentials
2. Install the APM integration package via Kibana
3. Send events to apm-server
4. Check events are indexed into data streams

Related issues

Closes #5961

[apmmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2021-11-16T01:19:07.192+0000

• Duration: 49 min 25 sec

• Commit: ae0b3bd

Test stats 🧪

Test	Results
Failed	0
Passed	5610
Skipped	19
Total	5629

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

• /hey-apm : Run the hey-apm benchmark.

• /package : Generate and publish the docker images.

• run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request is now in conflicts. Could you fix it @axw? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b no-index-management upstream/no-index-management
git merge upstream/master
git push upstream no-index-management

[mergify]

This pull request is now in conflicts. Could you fix it @axw? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b no-index-management upstream/no-index-management
git merge upstream/master
git push upstream no-index-management

[simitt]

Are you going to remove the apm-server.register.ingest.pipeline in a separate PR?

[axw]

Are you going to remove the apm-server.register.ingest.pipeline in a separate PR?

@simitt do you mean remove from apm-server.yml? It was removed in #6575:

642aaee#diff-fbadce52f216b92bda4dab62b076f334502b170af5e51bf3e442e2033b4a4e9d

[simitt]

Yes. Sorry I missed that and still had it in my main branch when I checked.

[marclop]

Verified using 9bb5e4c.