[FEATURE] Publish older major versions of the Python Client SDK as a separate packages

[chelma]

Is your feature request related to a problem?

Users may want to have both the 1.x and 2.x version of the Python Client SDK available in their application. As an example, I'm currently working on a library for testing upgrades across major versions of the OpenSearch core engine and I want to be able to talk to both the "old" and "new" cluster as part of a snapshot/restore process.

What solution would you like?

Publish the latest minor version of each major version as a separate Python package that can be imported separately with independent namespaces. Elasticsearch does this for its Python client [1].

[1] https://github.com/elastic/elasticsearch-py/blob/7.10/docs/sphinx/index.rst

What alternatives have you considered?

If y'all can offer me an alternative way to use both the Client SDK 1.x and 2.x in my Python app, I'm open to it; but this seems like a pretty straightforward solution

Do you have any additional context?

N/A

[chelma]

Per convo w/ @wbeckler - our team decided to re-implement the client ourselves rather than use the existing Python client, meaning this is not a blocker for us. However, I think this is still worth doing as it could be a blocker for others in the community and appears to be a light lift to set up.

[dblock]

@chelma Is there a discussion somewhere? Another client seems like a lot of extra work.

[chelma]

@wbeckler - Per my read of the updated compatibility matrix, the latest version of the client is compatible with all version of OpenSearch, meaning there is no need to publish historical versions for backwards compatibility reasons. Is that the plan going forward? If so - we can resolve.

@dblock - Our team discussed it internally. @kartg realized that relying on a language-specific client API introduces another variable that we need to consider/test as part of the UTF (e.g. different behavior in the python vs. java client). Combined with indications from our UX research sessions that many users roll their own clients anyways, we decided to test against the REST interface presented by ElasticSearch/OpenSearch rather than go through a language-specific client. So we're not so much making a full client as just making a light wrapper around the REST interface using Python-standard library features for making HTTP calls (the requests library). Hope this clarifies things?

[dblock]

@wbeckler - Per my read of the updated compatibility matrix, the latest version of the client is compatible with all version of OpenSearch, meaning there is no need to publish historical versions for backwards compatibility reasons. Is that the plan going forward? If so - we can resolve.

This is not the plan forever. Beyond server-side compatibility we may be introducing breaking changes in the client itself (for example dropping support for an older version of Python). Generally, clients follow semver, therefore any breaking change (server side API, python version) will be a major increment of the client's version. Related to server APIs we will continue ensuring that clients are compatible with 2 major versions (see opensearch-project/opensearch-clients#17).

@dblock - ... Hope this clarifies things?

Yes, thank you. I do think you'll reimplement many constructs that exist in the client, but c'est la vie.

[dblock]

I'm going to close this. You can get older versions of the client from the package manager, and it doesn't seem like the original ask is useful anymore. @chelma LMK if otherwise!