Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix bulk API ignores ingest pipeline for upsert #12883

Merged
merged 7 commits into from
Apr 3, 2024

Conversation

gaobinlong
Copy link
Collaborator

@gaobinlong gaobinlong commented Mar 24, 2024

Description

When there's an ingest pipeline specified in the bulk update action with upsert, bulk API doesn't set the pipeline in upsertRequest correctly, so that the specified pipeline will not be executed. This PR fixes the bug and adds a yaml test case for it.

Related Issues

#12854

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Failing checks are inspected and point to the corresponding known issue(s) (See: Troubleshooting Failing Builds)
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)
  • Public documentation issue/PR created

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Signed-off-by: Gao Binlong <[email protected]>
Copy link
Contributor

github-actions bot commented Mar 24, 2024

Compatibility status:

Checks if related components are compatible with change b19c271

Incompatible components

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/flow-framework.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/opensearch-oci-object-storage.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/neural-search.git, https://github.com/opensearch-project/security-analytics.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/sql.git]

Copy link
Contributor

❌ Gradle check result for 481070f: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

❌ Gradle check result for c2a7820: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

✅ Gradle check result for a26af14: SUCCESS

Copy link

codecov bot commented Mar 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.37%. Comparing base (b15cb0c) to head (b19c271).
Report is 99 commits behind head on main.

Additional details and impacted files
@@             Coverage Diff              @@
##               main   #12883      +/-   ##
============================================
- Coverage     71.42%   71.37%   -0.05%     
- Complexity    59978    60233     +255     
============================================
  Files          4985     5015      +30     
  Lines        282275   283826    +1551     
  Branches      40946    41134     +188     
============================================
+ Hits         201603   202576     +973     
- Misses        63999    64436     +437     
- Partials      16673    16814     +141     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

❕ Gradle check result for 3a18815: UNSTABLE

  • TEST FAILURES:
      1 org.opensearch.remotestore.RemoteStoreStatsIT.testZeroLagOnCreateIndex

Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

Signed-off-by: Gao Binlong <[email protected]>
Copy link
Contributor

❌ Gradle check result for 60ef6a1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link
Contributor

✅ Gradle check result for b19c271: SUCCESS

@gaobinlong
Copy link
Collaborator Author

Could any maintainer help to review this PR? @reta, @andrross, @dblock , thank you!

@reta
Copy link
Collaborator

reta commented Apr 3, 2024

Could any maintainer help to review this PR? @reta, @andrross, @dblock , thank you!

Looking, sorry about that @gaobinlong

@reta reta merged commit f2e8398 into opensearch-project:main Apr 3, 2024
30 checks passed
@reta reta added bug Something isn't working backport 2.x Backport to 2.x branch labels Apr 3, 2024
opensearch-trigger-bot bot pushed a commit that referenced this pull request Apr 3, 2024
* Fix bulk API ignores ingest pipeline for upsert

Signed-off-by: Gao Binlong <[email protected]>

* Modify change log

Signed-off-by: Gao Binlong <[email protected]>

* Use existing pipeline instead

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
(cherry picked from commit f2e8398)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta added a commit that referenced this pull request Apr 3, 2024
* Fix bulk API ignores ingest pipeline for upsert (#12883)

* Fix bulk API ignores ingest pipeline for upsert

Signed-off-by: Gao Binlong <[email protected]>

* Modify change log

Signed-off-by: Gao Binlong <[email protected]>

* Use existing pipeline instead

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
(cherry picked from commit f2e8398)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Update 70_bulk.yml

Signed-off-by: Andriy Redko <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Signed-off-by: Andriy Redko <[email protected]>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Andriy Redko <[email protected]>
shiv0408 pushed a commit to Gaurav614/OpenSearch that referenced this pull request Apr 25, 2024
…2883)

* Fix bulk API ignores ingest pipeline for upsert

Signed-off-by: Gao Binlong <[email protected]>

* Modify change log

Signed-off-by: Gao Binlong <[email protected]>

* Use existing pipeline instead

Signed-off-by: Gao Binlong <[email protected]>

---------

Signed-off-by: Gao Binlong <[email protected]>
Signed-off-by: Shivansh Arora <[email protected]>
@reta reta mentioned this pull request Jul 17, 2024
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch bug Something isn't working
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants