Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Watcher - workaround for potential deadlock #47603

Merged

Conversation

jakelandis
Copy link
Contributor

7.3.0 fixes this issue by changing the locking strategy in #41451.
However, that change is not part of 6.x and the change here is
a minimal workaround to prevent the potential of deadlock.

This change will no longer retry failed bulk requests that go
through the BulkProcessor for Watcher. Specifically this removes
the retry logic when adding Watcher history and Triggered watches
when the Bulk request failed.

Related #47599


Note - this is only for 6.8 since the real fix is in 7.3.0

7.3.0 fixes this issue by changing the locking strategy in elastic#41451.
However, that change is not part of 6.x and the change here is
a minimal workaround to prevent the potential of deadlock.

This change will no longer retry failed bulk requests that go
through the BulkProcessor for Watcher. Specifically this removes
the retry logic when adding Watcher history and Triggered watches
when the Bulk request failed.

Related elastic#47599
@@ -395,6 +396,7 @@ public void afterBulk(long executionId, BulkRequest request, Throwable failure)
.setBulkActions(SETTING_BULK_ACTIONS.get(settings))
.setBulkSize(SETTING_BULK_SIZE.get(settings))
.setConcurrentRequests(SETTING_BULK_CONCURRENT_REQUESTS.get(settings))
.setBackoffPolicy(BackoffPolicy.noBackoff())
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

optionally: add a comment here, why this was added? (OTOH the commit should reveal this as well)

@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/Watcher)

@jakelandis jakelandis merged commit e2448fd into elastic:6.8 Oct 7, 2019
@jakelandis jakelandis deleted the watcher_deadlock_when_bulk_fails branch October 7, 2019 13:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants