Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assert no exceptions during state application #47090

Conversation

DaveCTurner
Copy link
Contributor

Today we log and swallow exceptions during cluster state application, but such
an exception should not occur. This commit adds assertions of this fact, and
updates the Javadocs to explain it.

Relates #47038

Today we log and swallow exceptions during cluster state application, but such
an exception should not occur. This commit adds assertions of this fact, and
updates the Javadocs to explain it.

Relates elastic#47038
@DaveCTurner DaveCTurner added >bug :Distributed Coordination/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. v8.0.0 v7.5.0 v7.4.1 labels Sep 25, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-distributed

Copy link
Member

@original-brownbear original-brownbear left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, this makes a lot of sense to me :)

Copy link
Member

@original-brownbear original-brownbear left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems this needs some test removal/adjustment still though, tests that are simulating exceptions in the CS applier fail now:

FDE 25, 2019 10:43:28 COMME com.carrotsearch.randomizedtesting.RandomizedRunner$QueueUncaughtExceptionsHandler uncaughtException
WARNING: Uncaught exception in thread: Thread[elasticsearch[test_node][clusterApplierService#updateTask][T#1],5,TGRP-ClusterApplierServiceTests]
java.lang.AssertionError
	at __randomizedtesting.SeedInfo.seed([FA241879CEA2DA83]:0)
	at org.elasticsearch.cluster.service.ClusterApplierService.runTask(ClusterApplierService.java:452)
	at org.elasticsearch.cluster.service.ClusterApplierService$UpdateTask.run(ClusterApplierService.java:176)
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:699)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:252)
	at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:215)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)

@DaveCTurner
Copy link
Contributor Author

Thanks @original-brownbear I would wait for a green CI run before looking too hard at this, in case it shakes out anything else.

@DaveCTurner
Copy link
Contributor Author

Failure is unrelated, I opened #47098.

@elasticmachine please run elasticsearch-ci/1

Copy link
Member

@original-brownbear original-brownbear left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM :)

It is a little weird though to test the case that we assert doesn't happen ever, but I can see how we still want to handle those exceptions properly and keep the coverage :)

@DaveCTurner DaveCTurner merged commit 053e95b into elastic:master Sep 25, 2019
@DaveCTurner DaveCTurner deleted the 2019-09-25-assert-no-exceptions-in-application branch September 25, 2019 11:33
DaveCTurner added a commit that referenced this pull request Sep 25, 2019
Today we log and swallow exceptions during cluster state application, but such
an exception should not occur. This commit adds assertions of this fact, and
updates the Javadocs to explain it.

Relates #47038
DaveCTurner added a commit that referenced this pull request Sep 25, 2019
Today we log and swallow exceptions during cluster state application, but such
an exception should not occur. This commit adds assertions of this fact, and
updates the Javadocs to explain it.

Relates #47038
@colings86 colings86 added v7.4.0 and removed v7.4.1 labels Sep 27, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Distributed Coordination/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. v7.4.0 v7.5.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants