Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Backport 2.x] Make RemoteStoreReplicationSource#getSegmentFiles asynchronous #10771

Merged
merged 1 commit into from
Oct 20, 2023

Conversation

opensearch-trigger-bot[bot]
Copy link
Contributor

Backport 41a12e2 from #10725.

* Make RemoteStoreReplicationSource#getSegmentFiles asynchronous

Also make the remote store download process cancellable in case the
replication event is canceled.

Signed-off-by: Andrew Ross <[email protected]>

* Add ITs ensuring segRep targets are cleaned up on cancellation during metadata and segment fetch steps.

Signed-off-by: Marc Handalian <[email protected]>

* Wrap metadata fetch in cancellableThreads.executeIO

Signed-off-by: Marc Handalian <[email protected]>

* self review

Signed-off-by: Marc Handalian <[email protected]>

* spotless

Signed-off-by: Marc Handalian <[email protected]>

* Add missing node settings when bootstrapping nodes in tests.

Signed-off-by: Marc Handalian <[email protected]>

---------

Signed-off-by: Andrew Ross <[email protected]>
Signed-off-by: Marc Handalian <[email protected]>
Co-authored-by: Andrew Ross <[email protected]>
(cherry picked from commit 41a12e2)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@github-actions
Copy link
Contributor

Compatibility status:

Checks if related components are compatible with change 8fad0ea

Incompatible components

Incompatible components: [https://github.com/opensearch-project/security-analytics.git]

Skipped components

Compatible components

Compatible components: [https://github.com/opensearch-project/security.git, https://github.com/opensearch-project/alerting.git, https://github.com/opensearch-project/index-management.git, https://github.com/opensearch-project/anomaly-detection.git, https://github.com/opensearch-project/asynchronous-search.git, https://github.com/opensearch-project/sql.git, https://github.com/opensearch-project/job-scheduler.git, https://github.com/opensearch-project/observability.git, https://github.com/opensearch-project/common-utils.git, https://github.com/opensearch-project/reporting.git, https://github.com/opensearch-project/k-nn.git, https://github.com/opensearch-project/cross-cluster-replication.git, https://github.com/opensearch-project/custom-codecs.git, https://github.com/opensearch-project/geospatial.git, https://github.com/opensearch-project/ml-commons.git, https://github.com/opensearch-project/performance-analyzer.git, https://github.com/opensearch-project/notifications.git, https://github.com/opensearch-project/performance-analyzer-rca.git, https://github.com/opensearch-project/neural-search.git]

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

  • RESULT: UNSTABLE ❕
  • TEST FAILURES:
      2 org.opensearch.index.shard.RemoteIndexShardTests.classMethod
      1 org.opensearch.index.shard.RemoteIndexShardTests.testSegRepSucceedsOnPreviousCopiedFiles

@codecov
Copy link

codecov bot commented Oct 20, 2023

Codecov Report

Merging #10771 (8fad0ea) into 2.x (822b9d3) will decrease coverage by 0.07%.
Report is 6 commits behind head on 2.x.
The diff coverage is 73.00%.

@@             Coverage Diff              @@
##                2.x   #10771      +/-   ##
============================================
- Coverage     70.98%   70.91%   -0.07%     
+ Complexity    58685    58652      -33     
============================================
  Files          4839     4839              
  Lines        277012   277135     +123     
  Branches      40639    40657      +18     
============================================
- Hits         196629   196540      -89     
- Misses        63664    63851     +187     
- Partials      16719    16744      +25     
Files Coverage Δ
...pensearch/common/settings/FeatureFlagSettings.java 50.00% <ø> (ø)
.../java/org/opensearch/gateway/GatewayMetaState.java 67.59% <100.00%> (-1.88%) ⬇️
...arch/gateway/remote/RemoteClusterStateService.java 69.54% <100.00%> (-0.14%) ⬇️
.../org/opensearch/index/mapper/RangeFieldMapper.java 87.25% <100.00%> (ø)
.../org/opensearch/index/mapper/RootObjectMapper.java 82.74% <100.00%> (+0.07%) ⬆️
...search/index/shard/RemoteStoreRefreshListener.java 87.56% <100.00%> (-2.86%) ⬇️
...ces/replication/PrimaryShardReplicationSource.java 96.00% <ø> (ø)
.../indices/replication/common/ReplicationTarget.java 72.50% <100.00%> (+1.44%) ⬆️
...n/java/org/opensearch/script/ScoreScriptUtils.java 1.50% <100.00%> (ø)
...nsearch/search/aggregations/support/ValueType.java 79.59% <100.00%> (+0.42%) ⬆️
... and 14 more

... and 457 files with indirect coverage changes

@mch2 mch2 merged commit eed9d99 into 2.x Oct 20, 2023
42 of 67 checks passed
@mch2 mch2 deleted the backport/backport-10725-to-2.x branch October 20, 2023 05:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant