Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] ClusterSearchShardsResponseTests testSerialization failing #100289

Closed
ldematte opened this issue Oct 4, 2023 · 7 comments · Fixed by #100557
Closed

[CI] ClusterSearchShardsResponseTests testSerialization failing #100289

ldematte opened this issue Oct 4, 2023 · 7 comments · Fixed by #100557
Assignees
@benwtrent
Labels
:Core/Infra/Core Core issues without another label low-risk An open issue or test failure that is a low risk to future releases :Search/Search Search-related issues that do not fall into other categories Team:Core/Infra Meta label for core/infra team Team:Search Meta label for search team >test-failure Triaged test failures from CI

Comments

@ldematte
Copy link
Contributor

ldematte commented Oct 4, 2023

After version upgrade to 8.12, 8.11 can be generated as a random version to be passed to DiscoveryNodeUtils.create (by VersionUtils.randomVersion(random())). However from 8.10 forward, minIndexVersion, maxIndexVersion need to be specified too.
The test should be updated to use DiscoveryNode.builder() directly and specify min/max index versions too.

Build scan:
https://gradle-enterprise.elastic.co/s/ib3thvzxhoyo4/tests/:server:test/org.elasticsearch.action.admin.cluster.shards.ClusterSearchShardsResponseTests/testSerialization
Reproduction line:

./gradlew ':server:test' --tests "org.elasticsearch.action.admin.cluster.shards.ClusterSearchShardsResponseTests.testSerialization" -Dtests.seed=194A3CC5493A6758 -Dtests.locale=et -Dtests.timezone=GMT0 -Druntime.java=21

Applicable branches:
main

Reproduces locally?:
Yes

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.action.admin.cluster.shards.ClusterSearchShardsResponseTests&tests.test=testSerialization
Failure excerpt:

java.lang.IllegalArgumentException: Node versions can only be inferred before release version 8.10.0

  at __randomizedtesting.SeedInfo.seed([194A3CC5493A6758:4D925D8D6E685AC4]:0)
  at org.elasticsearch.cluster.node.VersionInformation.inferVersions(VersionInformation.java:42)
  at org.elasticsearch.cluster.node.DiscoveryNodeUtils$Builder.build(DiscoveryNodeUtils.java:144)
  at org.elasticsearch.cluster.node.DiscoveryNodeUtils.create(DiscoveryNodeUtils.java:37)
  at org.elasticsearch.action.admin.cluster.shards.ClusterSearchShardsResponseTests.testSerialization(ClusterSearchShardsResponseTests.java:52)
  at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
  at java.lang.reflect.Method.invoke(Method.java:580)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:1583)
@ldematte ldematte added :Search/Search Search-related issues that do not fall into other categories >test-failure Triaged test failures from CI labels Oct 4, 2023
@elasticsearchmachine elasticsearchmachine added the Team:Search Meta label for search team label Oct 4, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@benwtrent benwtrent mentioned this issue Oct 9, 2023
Closed
@benwtrent benwtrent added the low-risk An open issue or test failure that is a low risk to future releases label Oct 9, 2023
@benwtrent benwtrent self-assigned this Oct 9, 2023
@benwtrent benwtrent mentioned this issue Oct 9, 2023
Merged
benwtrent added a commit that referenced this issue Oct 10, 2023
@benwtrent
Fix ClusterSearchShardsResponseTests#testSerialization (#100557)
0efd101 
Fixes serialization random versioning.

closes: #100289
@ldematte ldematte reopened this Nov 9, 2023
@ldematte
Copy link
Contributor Author

ldematte commented Nov 9, 2023

I see this failing again, with the same exception: https://gradle-enterprise.elastic.co/s/ny6d4ym32nwvy/tests/task/:server:test/details/org.elasticsearch.action.admin.cluster.shards.ClusterSearchShardsResponseTests/testSerialization?top-execution=1

https://gradle-enterprise.elastic.co/s/cltby2i2mi22m/tests/task/:server:test/details/org.elasticsearch.action.admin.cluster.shards.ClusterSearchShardsResponseTests/testSerialization?top-execution=1

Not sure why: maybe related to 8.11 release?

@benwtrent
Copy link
Member

🤔 weird that it failed again. I will look

@mark-vieira
Copy link
Contributor

Yep, this sounds like fallout from 8.11.0. This error is happening across a number of tests. This is likely a core/infra issue. I recall a similar set of errors when 8.10 released.

image

@mark-vieira mark-vieira added the :Core/Infra/Core Core issues without another label label Nov 9, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@elasticsearchmachine elasticsearchmachine added the Team:Core/Infra Meta label for core/infra team label Nov 9, 2023
@ldematte
Copy link
Contributor Author

weird that it failed again. I will look

Yes, the weird thing is that the fix you merged should have fixed it permanently. Maybe it's failing elsewhere - I will also give it a look.

@ldematte
Copy link
Contributor Author

Ah, I see. The test was fixed after 8.11 FF, and now after 8.11 released we are doing 8.11 branch CI - which shows the same error again. I will backport the fix to the 8.11 branch

@ldematte ldematte mentioned this issue Nov 10, 2023
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@benwtrent benwtrent

Labels
:Core/Infra/Core Core issues without another label low-risk An open issue or test failure that is a low risk to future releases :Search/Search Search-related issues that do not fall into other categories Team:Core/Infra Meta label for core/infra team Team:Search Meta label for search team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix ClusterSearchShardsResponseTests#testSerialization benwtrent/elasticsearch
4 participants
@ldematte @mark-vieira @benwtrent @elasticsearchmachine