Only log warning when actually failing shards #28558
Merged
+7
−3
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ywelsch
reviewed
Feb 8, 2018
| @@ -188,6 +189,7 @@ public ClusterState applyFailedShards(final ClusterState clusterState, final Lis | |||
| if (failedShardEntry.markAsStale()) { | |||
| allocation.removeAllocationId(failedShard); | |||
| } | |||
| logger.warn(new ParameterizedMessage("failing shard [{}]", failedShardEntry), failedShardEntry.getFailure()); | |||
There was a problem hiding this comment.
also add logging information for the "mark as stale" part here as well as in removeStaleIdsWithoutRoutings?
|
@ywelsch, I've added a logging for |
ywelsch
reviewed
Feb 8, 2018
| @@ -238,6 +239,11 @@ public static ClusterState removeStaleIdsWithoutRoutings(ClusterState clusterSta | |||
| } | |||
| indexMetaDataBuilder.putInSyncAllocationIds(shardNumber, remainingInSyncAllocations); | |||
| } | |||
| // Only log the stale shards which have been actually removed. | |||
| if (oldInSyncAllocations.size() != remainingInSyncAllocations.size()) { | |||
There was a problem hiding this comment.
This check is not necessary. idsToRemove already contains the info that's needed (it does not contain non-existing ids, you can even add an assertion for that if you like).
Also I would change the message to "{} marking unavailable shards as stale: {}"
|
Thanks @ywelsch for reviewing. |
dnhatn
changed the title
dnhatn
added a commit
that referenced
this pull request
Feb 8, 2018
Currently the master node logs a warning message whenever it receives a failed shard request. However, this can be noisy because - Multiple failed shard requests can be issued for a single shard - Failed shard requests can be still issued for an already failed shard This commit moves the log-warn to AllocationService in which the failing shard action actually happens. This is another prerequisite step in order to not ignore the shard not-available exceptions in the replication. Relates #28534
lcawl
added
:Search/Search
clintongormley
added
:Distributed Indexing/Distributed
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently the master node logs a warning message whenever it receives a failed shard request. However, this can be noisy because
This commit moves the log-warn to AllocationService in which the failing shard action actually happens. This is another prerequisite step in order to not ignore the shard not-available exceptions in the replication.
Relates #28534