[Telemetry] Report array of well-known technologies

[afharo]

Describe the feature:

We would like to report what well-known technologies are used to ingest data in elasticsearch.

Describe a specific use case for the feature:

In order to build better experiences for our users, we want to understand the most common usages.

[kobelb]

There have been some external discussions to solving the initial implementation of "data source telemetry" which proposed using the cluster state API to determine whether or not various indices exist. It's come to my attention that Elasticsearch makes no guarantees about the response format for cluster-state, and it's not an API which we should be relying upon.

[afharo]

@kobelb I'm implementing a POC, keeping in mind the 2 main different ways of collecting telemetry: OSS/X-Pack vs. Monitoring.

The monitoring way is, initially working as expected. I'll work now on the OSS/X-Pack way and will play around with that API (and possibly the _stats one?) and will try changing permissions to see how it affects it.

Does it sound sensible?

[kobelb]

@afharo sounds completely reasonable! Using monitoring is definitely an option; however, based on some data I was looking at the other day there's a significant number of clusters which don't have monitoring enabled that we'd be missing information from.

[afharo]

Yeah! I checked with @alexfrancoeur and we should prioritise the info we can get from OSS and X-Pack over Monitoring.