Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix IndexShardTests.testRestoreShardFromRemoteStore on Windows. #5399

Merged
merged 1 commit into from
Nov 30, 2022

Conversation

mch2
Copy link
Member

@mch2 mch2 commented Nov 29, 2022

Signed-off-by: Marc Handalian [email protected]

Description

Fix IndexShardTests.testRestoreShardFromRemoteStore on Windows.

Windows has buggy File delete logic where AccessDeniedExceptions are thrown when there is an open file handle on a particular file. FSDirectory attempts to resolve this with hacks by swallowing the exceptions and moving the file to a pending delete state to retry in the future while being filtered from listAll invocations.
However, this logic is also buggy and after the first delete attempt we are left in a state where the file is still on disk and not pending delete. This is because FSDirectory#maybeDeletePendingFiles will execute and attempt to delete the purged files again, which hits a NoSuchFileException that is also swallowed.
A second attempt to delete the file will properly move it to pending deletion, and be filtered from listAll.

Issues Resolved

related #1448
closes #5209

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff
  • Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@mch2 mch2 requested review from a team and reta as code owners November 29, 2022 20:17
@github-actions
Copy link
Contributor

Gradle Check (Jenkins) Run Completed with:

@codecov-commenter
Copy link

Codecov Report

Merging #5399 (fb403b6) into main (e3fd49f) will increase coverage by 0.09%.
The diff coverage is 100.00%.

@@             Coverage Diff              @@
##               main    #5399      +/-   ##
============================================
+ Coverage     70.88%   70.97%   +0.09%     
- Complexity    58062    58101      +39     
============================================
  Files          4711     4711              
  Lines        277533   277534       +1     
  Branches      40169    40169              
============================================
+ Hits         196737   196993     +256     
+ Misses        64612    64385     -227     
+ Partials      16184    16156      -28     
Impacted Files Coverage Δ
server/src/main/java/org/opensearch/Version.java 77.77% <100.00%> (+0.10%) ⬆️
...g/opensearch/index/analysis/CharFilterFactory.java 0.00% <0.00%> (-100.00%) ⬇️
.../java/org/opensearch/node/NodeClosedException.java 50.00% <0.00%> (-50.00%) ⬇️
.../java/org/opensearch/search/dfs/AggregatedDfs.java 51.61% <0.00%> (-41.94%) ⬇️
...ava/org/opensearch/search/dfs/DfsSearchResult.java 47.87% <0.00%> (-39.37%) ⬇️
...h/action/ingest/SimulateDocumentVerboseResult.java 60.71% <0.00%> (-39.29%) ⬇️
...tions/InternalSingleBucketAggregationTestCase.java 59.74% <0.00%> (-36.37%) ⬇️
...n/decider/SnapshotInProgressAllocationDecider.java 34.78% <0.00%> (-34.79%) ⬇️
...arch/search/aggregations/pipeline/LinearModel.java 23.07% <0.00%> (-34.62%) ⬇️
...r/src/main/java/org/opensearch/http/HttpUtils.java 33.33% <0.00%> (-33.34%) ⬇️
... and 495 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@mch2 mch2 merged commit 0210b76 into opensearch-project:main Nov 30, 2022
@mch2 mch2 deleted the 5209 branch November 30, 2022 17:10
@mch2 mch2 added the backport 2.x Backport to 2.x branch label Nov 30, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Nov 30, 2022
(cherry picked from commit 0210b76)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
mch2 pushed a commit that referenced this pull request Dec 1, 2022
… (#5410)

(cherry picked from commit 0210b76)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
mch2 pushed a commit to mch2/OpenSearch that referenced this pull request Dec 9, 2022
…search-project#5399) (opensearch-project#5410)

(cherry picked from commit 0210b76)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
@mch2 mch2 added the backport 2.4 Backport to 2.4 branch label Dec 10, 2022
opensearch-trigger-bot bot pushed a commit that referenced this pull request Dec 10, 2022
(cherry picked from commit 0210b76)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
reta pushed a commit that referenced this pull request Dec 20, 2022
… (#5514)

(cherry picked from commit 0210b76)
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport 2.4 Backport to 2.4 branch skip-changelog
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[BUG] Flaky test - testRestoreShardFromRemoteStore
4 participants