Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support OpenSearch 2.0 #1311

Closed
4 tasks done
dlvenable opened this issue Apr 12, 2022 · 2 comments
Closed
4 tasks done

Support OpenSearch 2.0 #1311

dlvenable opened this issue Apr 12, 2022 · 2 comments
Assignees
Milestone

Comments

@dlvenable
Copy link
Member

dlvenable commented Apr 12, 2022

Is your feature request related to a problem? Please describe.

OpenSearch 2.0 is releasing in May. Data Prepper should support OpenSearch 2.0.
https://opensearch.org/blog/partners/2022/02/roadmap-proposal/

Describe the solution you'd like

Verify that OpenSearch 2.0 works as a sink and make any necessary changes.

Tasks

@dlvenable dlvenable added this to the v1.4 milestone Apr 12, 2022
@dlvenable
Copy link
Member Author

Data Prepper is using the deprecated master_timeout value:

Per opensearch-project/documentation-website#450, this value will remain available in OpenSearch 2.0.

@dlvenable
Copy link
Member Author

Data Prepper is currently not compatible with OpenSearch.

The bulk request is failing on the assertion below which is in OpenSearch 1.3.1. It currently expects the type to be present, but this was removed along with the removal of mapping types.

https://github.com/opensearch-project/OpenSearch/blob/c4c0672877bf0f787ca857c7c37b775967f93d81/server/src/main/java/org/opensearch/action/DocWriteResponse.java#L140

The snippet below is from the current HEAD of OpenSearch 2.x. It shows that the type is no longer required. (Indeed it is not even a field).

https://github.com/opensearch-project/OpenSearch/blob/ff75b5ffee0ce08894f9670e539c99c5e3ef86b7/server/src/main/java/org/opensearch/action/DocWriteResponse.java#L137-L144

After OpenSearch 2.0.0 RC1 releases, Data Prepper will need to use the 2.0.0 RC1 client. Release task: opensearch-project/opensearch-build#1624

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

1 participant