Fix SearchableSnapshotsBlobStoreCacheIntegTests.testBlobStoreCache #78616
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
tlrx
added
>test
elasticmachine
added
the
Team:Distributed (Obsolete)
|
Pinging @elastic/es-distributed (Team:Distributed) |
henningandersen
approved these changes
Oct 6, 2021
| @@ -329,6 +330,10 @@ public Settings onNodeStopped(String nodeName) throws Exception { | |||
|
|
|||
| logger.info("--> deleting indices, maintenance service should clean up snapshot blob cache index"); | |||
| assertAcked(client().admin().indices().prepareDelete("restored-*")); | |||
| // waits for the data node that hosts the .snapshot-blob-cache primary shard | |||
| // to process the cluster state update and to trigger a clean up | |||
| ensureClusterStateConsistency(); | |||
There was a problem hiding this comment.
I am not sure this waits for anything and it seems sort of unrelated to this test?
| @@ -338,7 +343,7 @@ public Settings onNodeStopped(String nodeName) throws Exception { | |||
| .get(), | |||
| 0L | |||
| ); | |||
| }); | |||
| }, 30L, TimeUnit.SECONDS); | |||
There was a problem hiding this comment.
OK, this might help, but can we perhaps log numberOfCachedBlobs before this assert busy such that we can see if it was trying to delete entries but was simply slow or if it did not even start on it?
There was a problem hiding this comment.
Added the numberOfCachedBlobs in the log message before deleting all remaining mounted indices.
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this pull request
Oct 7, 2021
…lastic#78616) This test often fails on CI because it does not wait for the data node that hosts the .snapshot-blob-cache primary shard to process the cluster state update and triggers the clean up. Closes elastic#78512
elasticsearchmachine
pushed a commit
that referenced
this pull request
Oct 7, 2021
wjp719
added a commit
to wjp719/elasticsearch
that referenced
this pull request
Oct 7, 2021
…' into feature/data_stream_support_routing * wjp/feature/data_stream_support_routing: (44 commits) Revert "Adjust /_cat/templates not to request all metadata (elastic#78812)" Allow indices lookup to be built lazily (elastic#78745) [DOCS] Document default security in alpha2 (elastic#78227) Add cluster applier stats (elastic#77552) Fix failing URLDecodeProcessorTests::testProcessor test (elastic#78690) Upgrade to lucene snapshot ba75dc5e6bf (elastic#78817) Adjust /_cat/templates not to request all metadata (elastic#78812) Simplify build plugin license handling (elastic#77009) Fix SearchableSnapshotsBlobStoreCacheIntegTests.testBlobStoreCache (elastic#78616) Improve Docker image caching and testing (elastic#78552) Load knn vectors format with mmapfs (elastic#78724) Fix date math zone test to use negative minutes (elastic#78796) Changing name of shards field in node/stats api to shard_stats (elastic#78531) [DOCS] Fix system index refs in restore tutorial (elastic#78582) Add previously removed settings back for 8.0 (elastic#78784) TSDB: Fix template name in test Add a system property to forcibly format everything (elastic#78768) Revert "Adding config so that some tests will break if over-the-wire encryption fails (elastic#78409)" (elastic#78787) Must date math test failure Adding config so that some tests will break if over-the-wire encryption fails (elastic#78409) ...
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this pull request
Oct 27, 2021
The attempt to fix this test in elastic#78616 wasn't complete: we need to wait for all mounted indices, not only the last restored one, before deleting the indices. Otherwise one of the mounted index might still be initializing and recreates the cached documents in `.snapshot-blob-cache` Closes elastic#78993
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this pull request
Oct 27, 2021
The attempt to fix this test in elastic#78616 wasn't complete: we need to wait for all mounted indices, not only the last restored one, before deleting the indices. Otherwise one of the mounted index might still be initializing and recreates the cached documents in `.snapshot-blob-cache` Closes elastic#78993
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this pull request
Oct 27, 2021
The attempt to fix this test in elastic#78616 wasn't complete: we need to wait for all mounted indices, not only the last restored one, before deleting the indices. Otherwise one of the mounted index might still be initializing and recreates the cached documents in `.snapshot-blob-cache` Closes elastic#78993
tlrx
added a commit
to tlrx/elasticsearch
that referenced
this pull request
Oct 27, 2021
The attempt to fix this test in elastic#78616 wasn't complete: we need to wait for all mounted indices, not only the last restored one, before deleting the indices. Otherwise one of the mounted index might still be initializing and recreates the cached documents in `.snapshot-blob-cache` Closes elastic#78993
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This test often fails on CI because it does not wait for the data node that hosts the
.snapshot-blob-cacheprimary shard to process the cluster state update and triggers the clean up.Closes #78512