Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Move bulk API's batch_size parameter to processors #7719

Merged
merged 8 commits into from
Jul 29, 2024

Conversation

chishui
Copy link
Contributor

@chishui chishui commented Jul 16, 2024

Description

  1. Mark batch_size parameter of bulk API to deprecated
  2. Change batch_size parameter's default value from 1 to 2147483647
  3. Add batch_size to text_embedding and sparse_encoding processors

Issues Resolved

Closes #7717

Version

2.16

Frontend features

If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@hdhalter hdhalter added release-notes PR: Include this PR in the automated release notes v2.16.0 labels Jul 16, 2024
@hdhalter
Copy link
Contributor

Thanks, @chishui!! Is this ready for a doc review?

@chishui
Copy link
Contributor Author

chishui commented Jul 17, 2024

@hdhalter I'll post a comment here once this PR is merged and the doc is ready for review, thanks!

@chishui
Copy link
Contributor Author

chishui commented Jul 23, 2024

@hdhalter @Naarcha-AWS The PR this documentation depending on has been merged

@hdhalter hdhalter added the 4 - Doc review PR: Doc review in progress label Jul 23, 2024
_api-reference/document-apis/bulk.md Outdated Show resolved Hide resolved
_ingest-pipelines/processors/sparse-encoding.md Outdated Show resolved Hide resolved
_ingest-pipelines/processors/text-embedding.md Outdated Show resolved Hide resolved
chishui and others added 3 commits July 24, 2024 11:02
Co-authored-by: Naarcha-AWS <[email protected]>
Signed-off-by: Liyun Xiu <[email protected]>
@Naarcha-AWS Naarcha-AWS added 5 - Editorial review PR: Editorial review in progress and removed 4 - Doc review PR: Doc review in progress labels Jul 25, 2024
Copy link
Collaborator

@natebower natebower left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Naarcha-AWS @chishui Just a couple of minor additions. Thanks!

_ml-commons-plugin/remote-models/batch-ingestion.md Outdated Show resolved Hide resolved
_ml-commons-plugin/remote-models/batch-ingestion.md Outdated Show resolved Hide resolved
@Naarcha-AWS Naarcha-AWS merged commit 9f9e6d5 into opensearch-project:main Jul 29, 2024
5 checks passed
@hdhalter hdhalter added 3 - Done Issue is done/complete and removed 5 - Editorial review PR: Editorial review in progress labels Jul 30, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
3 - Done Issue is done/complete release-notes PR: Include this PR in the automated release notes v2.16.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] Deprecate batch_size in bulk and move it to two processors
4 participants