Use OpenSearch constants while still supporting OpenDistro

[dlvenable]

Is your feature request related to a problem? Please describe.

Data Prepper's opensearch plugin currently supports both OpenSearch 1.0 clusters and OpenDistro Elasticsearch 7.10 clusters. It supports both by using "opendistro" APIs, indices, and property names.

Examples:

• data-prepper/data-prepper-plugins/opensearch/src/main/java/com/amazon/dataprepper/plugins/sink/opensearch/index/IndexConstants.java

  Lines 27 to 29 in 06d53ae

  	public static final String ISM_ENABLED_SETTING = "opendistro.index_state_management.enabled";
  	public static final String ISM_POLICY_ID_SETTING = "opendistro.index_state_management.policy_id";
  	public static final String ISM_ROLLOVER_ALIAS_SETTING = "opendistro.index_state_management.rollover_alias";

• data-prepper/data-prepper-plugins/opensearch/src/main/java/com/amazon/dataprepper/plugins/sink/opensearch/index/IsmPolicyManagement.java

  Line 39 in 1183a7a

  private static final String POLICY_MANAGEMENT_ENDPOINT = "/_opendistro/_ism/policies/";

• data-prepper/data-prepper-plugins/opensearch/src/test/java/com/amazon/dataprepper/plugins/sink/opensearch/OpenSearchSinkIT.java

  Line 593 in 54f8ce7

  final Request request = new Request(HttpMethod.GET, "/_opendistro/_ism/explain/" + index);

  (several usages in this file)

All the constants are available by searching for opendistro.

These should be replaced with the new values in OpenSearch.

Describe the solution you'd like

Data Prepper can determine the cluster type. It can do this by making an initial request to the main endpoint (/) and getting the distribution value. Then it uses the correct constants.

We would need to verify if this will work for early versions of OpenSearch (say, 1.0.0). Do these versions all use the new strings?

Describe alternatives you've considered (Optional)

Configuration
A relatively simple solution is adding a property to the opensearch plugin which determines if it supports opendistro or opensearch. Then these usages will vary depending on the cluster type.

Example:

sink:
  opensearch:
    hosts: ["https://localhost:9200"]
    distribution: opendistro

In the 1.x series of Data Prepper, the default should be opendistro since this is the current behavior (it will continue to work with OpenSearch 1.x clusters). But, in Data Prepper 2.x, the default should be opensearch.

Split Plugins

The opensearch plugin could be split into both opensearch and opendistro plugins. This might be necessary in the long-term depending on much these implementations must differ. But, I don't believe it is necessary presently.

[dlvenable]

OpenSearch 2.0 is currently scheduled for "early 2022" per the roadmap. I'm unsure if these "opendistro" strings will be removed or not. But, Data Prepper should get ahead of that possibility.

[dlvenable]

This is an interesting use-case in which using the distribution property may not always work: opensearch-project/logstash-output-opensearch#62

Basically, some other products provide OpenSearch-like endpoints, but are neither OpenSearch nor Elasticsearch clusters. As such, the distribution should not be opensearch. Indeed, Data Prepper has a goal to support this as well - #248.

[dlvenable]

OpenSearch is not removing these constant values until OpenSearch 3.0.

opensearch-project/opensearch-plugins#128

Data Prepper thus does not need to resolve this issue to support OpenSearch 2.0.

[dlvenable]

I added the breaking change tag because some solutions may require a breaking change. We can remove this tag if we decide to take a more dynamic approach.

Since OpenSearch 2.0 will continue to support the old values, Data Prepper can continue to use the old values for now.