Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] CoordinatorTests testDoesNotPerformElectionWhenRestartingFollower failing #89543

Closed
original-brownbear opened this issue Aug 23, 2022 · 2 comments · Fixed by #89876
Closed

[CI] CoordinatorTests testDoesNotPerformElectionWhenRestartingFollower failing #89543

original-brownbear opened this issue Aug 23, 2022 · 2 comments · Fixed by #89876
Assignees
@kingherc
Labels
:Distributed Coordination/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI

Comments

@original-brownbear
Copy link
Member

original-brownbear commented Aug 23, 2022 •
edited by idegtiarenko
Loading

Build scan:
https://gradle-enterprise.elastic.co/s/svflp24u53hec/tests/:server:test/org.elasticsearch.cluster.coordination.CoordinatorTests/testDoesNotPerformElectionWhenRestartingFollower

Reproduction line:
./gradlew ':server:test' --tests "org.elasticsearch.cluster.coordination.CoordinatorTests.testDoesNotPerformElectionWhenRestartingFollower" -Dtests.seed=971F1427747D3FEB -Dtests.locale=sl-SI -Dtests.timezone=Asia/Pyongyang -Druntime.java=17

Applicable branches:
main

Reproduces locally?:
yes

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.cluster.coordination.CoordinatorTests&tests.test=testDoesNotPerformElectionWhenRestartingFollower

Failure excerpt:

java.lang.AssertionError: (No message provided)

  at __randomizedtesting.SeedInfo.seed([971F1427747D3FEB:A6D371A05A76D3E]:0)
  at org.elasticsearch.cluster.coordination.Coordinator.invariant(Coordinator.java:1096)
  at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.runFor(AbstractCoordinatorTestCase.java:765)
  at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:561)
  at org.elasticsearch.cluster.coordination.AbstractCoordinatorTestCase$Cluster.stabilise(AbstractCoordinatorTestCase.java:544)
  at org.elasticsearch.cluster.coordination.CoordinatorTests.testDoesNotPerformElectionWhenRestartingFollower(CoordinatorTests.java:2334)
  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(NativeMethodAccessorImpl.java:-2)
  at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
  at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:568)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:44)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:833)
@original-brownbear original-brownbear added :Distributed Coordination/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. >test-failure Triaged test failures from CI labels Aug 23, 2022
@elasticsearchmachine elasticsearchmachine added the Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. label Aug 23, 2022
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-distributed (Team:Distributed)

@tlrx tlrx self-assigned this Aug 30, 2022
tlrx added a commit to tlrx/elasticsearch that referenced this issue Sep 6, 2022
@tlrx
Fix assertion for single-node cluster and periodic warning w/ seed hosts
d72bc86 
The test CoordinatorTests.testDoesNotPerformElectionWhenRestartingFollower
failed for a 2 nodes cluster after a follower node is restarted (the last
accepted cluster state has 1 node but no periodic checker, which looks OK
to me).

This commit changes the assertion to check the existence of the periodic
checker only when seed hosts have been configured.

Closes elastic#89543
@tlrx tlrx mentioned this issue Sep 6, 2022
Closed
@kingherc
Copy link
Contributor

kingherc commented Sep 7, 2022

Taking over this as mentioned in #89828 (comment) .

@kingherc kingherc assigned kingherc and unassigned tlrx Sep 7, 2022
@kingherc kingherc mentioned this issue Sep 7, 2022
Merged
@DaveCTurner DaveCTurner mentioned this issue Sep 7, 2022
Closed
kingherc added a commit to kingherc/elasticsearch that referenced this issue Sep 7, 2022
@kingherc
Update single-node cluster checker appropriately
a5a848b 
Even when the cluster becomes single-node from multi-node. Or
when the cluster becomes multi-node from single-node.

Fixes elastic#89543
kingherc added a commit that referenced this issue Sep 8, 2022
@kingherc
Update single-node cluster checker appropriately (#89876)
9d27318 
Even when the cluster becomes single-node from multi-node. Or
when the cluster becomes multi-node from single-node.

Fixes #89543
kingherc added a commit to kingherc/elasticsearch that referenced this issue Sep 8, 2022
@kingherc
Update single-node cluster checker appropriately (elastic#89876)
4fb1fc2 
Even when the cluster becomes single-node from multi-node. Or
when the cluster becomes multi-node from single-node.

Fixes elastic#89543
elasticsearchmachine pushed a commit that referenced this issue Sep 8, 2022
@kingherc
Update single-node cluster checker appropriately (#89876) (#89910)
9859ada 
Even when the cluster becomes single-node from multi-node. Or
when the cluster becomes multi-node from single-node.

Fixes #89543
@kingherc kingherc mentioned this issue Sep 13, 2022
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kingherc kingherc

Labels
:Distributed Coordination/Cluster Coordination Cluster formation and cluster state publication, including cluster membership and fault detection. Team:Distributed (Obsolete) Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination. >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
4 participants
@tlrx @kingherc @original-brownbear @elasticsearchmachine