Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

skip overriding routing table when it already contains entries with remote recovery source #9962

Merged
merged 6 commits into from
Sep 16, 2023
Merged

skip overriding routing table when it already contains entries with remote recovery source #9962

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
2f9b7fd
skip overriding routing table when it already contains entries with r…
linuxpi Sep 11, 2023
f1da2e2
address PR comments, add check for missing routing table and update U…
linuxpi Sep 12, 2023
257afe3
avoid version increment for remote cluster state restore
linuxpi Sep 13, 2023
cf6e62d
make check for version bump more explicit
linuxpi Sep 15, 2023
707eca8
Empty commit
linuxpi Sep 15, 2023
742ff75
Empty Commit
linuxpi Sep 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 16 additions & 1 deletion server/src/main/java/org/opensearch/gateway/ClusterStateUpdaters.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@
import org.opensearch.cluster.metadata.Metadata;
import org.opensearch.cluster.node.DiscoveryNode;
import org.opensearch.cluster.node.DiscoveryNodes;
import org.opensearch.cluster.routing.RecoverySource;
import org.opensearch.cluster.routing.RoutingTable;
import org.opensearch.common.settings.ClusterSettings;

Expand Down Expand Up @@ -120,7 +121,21 @@ static ClusterState updateRoutingTable(final ClusterState state) {
// initialize all index routing tables as empty
final RoutingTable.Builder routingTableBuilder = RoutingTable.builder(state.routingTable());
for (final IndexMetadata cursor : state.metadata().indices().values()) {
routingTableBuilder.addAsRecovery(cursor);
// Whether IndexMetadata is recovered from local disk or remote it doesn't matter to us at this point.
linuxpi marked this conversation as resolved.
Show resolved Hide resolved
// We are only concerned about index data recovery here. Which is why we only check for remote store enabled and not for remote
// cluster state enabled.
if (cursor.getSettings().getAsBoolean(IndexMetadata.SETTING_REMOTE_STORE_ENABLED, false) == false
|| state.routingTable().hasIndex(cursor.getIndex()) == false
|| state.routingTable()
linuxpi marked this conversation as resolved.
Show resolved Hide resolved
.index(cursor.getIndex())
.shardsMatchingPredicateCount(
shardRouting -> shardRouting.primary()
// We need to ensure atleast one of the primaries is being recovered from remote.
// This ensures we have gone through the RemoteStoreRestoreService and routing table is updated
&& shardRouting.recoverySource() instanceof RecoverySource.RemoteStoreRecoverySource
) == 0) {
linuxpi marked this conversation as resolved.
Show resolved Hide resolved
routingTableBuilder.addAsRecovery(cursor);
}
}
// start with 0 based versions for routing table
routingTableBuilder.version(0);
Expand Down
2 changes: 1 addition & 1 deletion server/src/main/java/org/opensearch/index/recovery/RemoteStoreRestoreService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -187,7 +187,7 @@ private RemoteRestoreResult executeRestore(
IndexMetadata indexMetadata = indexMetadataEntry.getValue().v2();
boolean metadataFromRemoteStore = indexMetadataEntry.getValue().v1();
IndexMetadata updatedIndexMetadata = indexMetadata;
if (restoreAllShards || metadataFromRemoteStore) {
if (restoreAllShards) {
updatedIndexMetadata = IndexMetadata.builder(indexMetadata)
.state(IndexMetadata.State.OPEN)
.version(1 + indexMetadata.getVersion())
linuxpi marked this conversation as resolved.
Show resolved Hide resolved
Expand Down
Loading