Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enhance Shard Level Metdata check in BlobStoreTestUtil #75737

Conversation

original-brownbear
Copy link
Member

Adding check that shard level index metadata actually contains the snapshots
it's supposed to contain. This would have caught a number of recent bugs.

Adding check that shard level index metadata actually contains the snapshots
it's supposed to contain. This would have caught a number of recent bugs.
@original-brownbear original-brownbear added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >test Issues or PRs that are addressing/adding tests v7.15.0 v8.0.0 labels Jul 27, 2021
@elasticmachine elasticmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Jul 27, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@@ -3238,10 +3238,9 @@ public BlobStoreIndexShardSnapshot loadShardSnapshot(BlobContainer shardContaine
* Loads all available snapshots in the repository using the given {@code generation} for a shard. When {@code shardGen}
* is null it tries to load it using the BwC mode, listing the available index- blobs in the shard container.
*/
public BlobStoreIndexShardSnapshots getBlobStoreIndexShardSnapshots(IndexId indexId, ShardId shardId, @Nullable String shardGen)
public BlobStoreIndexShardSnapshots getBlobStoreIndexShardSnapshots(IndexId indexId, int shardId, @Nullable String shardGen)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This API adjustment makes sense in isolation anyway IMO. It's misleading to assume that this logic would filter by ShardId when in fact it merely filters by index id and shard number without reference to the original Index in the ShardId.

Copy link
Contributor

@DaveCTurner DaveCTurner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@original-brownbear
Copy link
Member Author

original-brownbear commented Jul 27, 2021

Thanks David! (and thanks Francisco :), timing)

@original-brownbear original-brownbear merged commit b22180e into elastic:master Jul 27, 2021
@original-brownbear original-brownbear deleted the enhance-blobstore-test-util branch July 27, 2021 16:07
Copy link
Contributor

@fcofdez fcofdez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM2 👍

ywangd pushed a commit to ywangd/elasticsearch that referenced this pull request Jul 30, 2021
Adding check that shard level index metadata actually contains the snapshots
it's supposed to contain. This would have caught a number of recent bugs.
original-brownbear added a commit to original-brownbear/elasticsearch that referenced this pull request Aug 16, 2021
Adding check that shard level index metadata actually contains the snapshots
it's supposed to contain. This would have caught a number of recent bugs.
original-brownbear added a commit that referenced this pull request Aug 16, 2021
Adding check that shard level index metadata actually contains the snapshots
it's supposed to contain. This would have caught a number of recent bugs.
@original-brownbear original-brownbear restored the enhance-blobstore-test-util branch April 18, 2023 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test Issues or PRs that are addressing/adding tests v7.15.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants