Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Several BWC failures of MixedClusterClientYamlTestSuiteIT / search.highlight/20_fvh/Highlight query after release of 7.10.1 #66147

Closed
dliappis opened this issue Dec 10, 2020 · 3 comments · Fixed by #66192
Closed

Several BWC failures of MixedClusterClientYamlTestSuiteIT / search.highlight/20_fvh/Highlight query after release of 7.10.1 #66147

dliappis opened this issue Dec 10, 2020 · 3 comments · Fixed by #66192
Assignees
@jtibshirani
Labels
:Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test-failure Triaged test failures from CI

Comments

@dliappis
Copy link
Contributor

Build scan: https://gradle-enterprise.elastic.co/s/jracbd7txe7l2 / https://gradle-enterprise.elastic.co/s/eissbxg7vfko4 and many more

Repro line:

./gradlew ':qa:mixed-cluster:v6.8.14#mixedClusterTest' \
  -Dtests.class="org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT" \
  -Dtests.method="test {p0=search.highlight/20_fvh/Highlight query}" \
  -Dtests.seed=3DBFFE73381C7536 \
  -Dtests.security.manager=true \
  -Dtests.bwc=true \
  -Dbuild.snapshot=false \
  -Dtests.jvm.argline="-Dbuild.snapshot=false" \
  -Dtests.locale=en-AU \
  -Dtests.timezone=Mexico/BajaNorte \
  -Druntime.java=8

Reproduces locally?: yes

Applicable branches: 7.x / 7.10

Failure history:

https://gradle-enterprise.elastic.co/s/cfwfppp3jk5qs
https://gradle-enterprise.elastic.co/s/ze3k7c6oupzq6
https://gradle-enterprise.elastic.co/s/jracbd7txe7l2
https://gradle-enterprise.elastic.co/s/45bjefx3t6opu
https://gradle-enterprise.elastic.co/s/ukymgzhsuscuw
https://gradle-enterprise.elastic.co/s/ek4cgdwmj5p2k

Failure excerpt:

REPRODUCE WITH: ./gradlew ':qa:mixed-cluster:v6.8.14#mixedClusterTest' -Dtests.class="org.elasticsearch.backwards.MixedClusterClientYamlTestSuiteIT" -Dtests.method="test {p0=search.highlight/20_fvh/Highlight query}" -Dtests.seed=3DBFFE73381C7536 -Dtests.security.manager=true -Dtests.bwc=true -Dbuild.snapshot=false -Dtests.jvm.argline="-Dbuild.snapshot=false" -Dtests.locale=en-AU -Dtests.timezone=Mexico/BajaNorte -Druntime.java=8
  2> java.lang.AssertionError: Failure at [search.highlight/20_fvh:60]: field [hits.hits.0.highlight.title.0] is null
        at __randomizedtesting.SeedInfo.seed([3DBFFE73381C7536:B5EBC1A996E018CE]:0)
        at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:414)
        at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.test(ESClientYamlSuiteTestCase.java:391)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1750)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:938)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:974)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:988)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:49)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at org.apache.lucene.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:48)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:817)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:468)
        at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:947)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:832)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:883)
        at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:894)
        at org.apache.lucene.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:45)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:41)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at org.apache.lucene.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
        at org.apache.lucene.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:47)
        at org.apache.lucene.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:64)
        at org.apache.lucene.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:54)
        at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
        at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:368)
        at java.lang.Thread.run(Thread.java:748)

        Caused by:
        java.lang.AssertionError: field [hits.hits.0.highlight.title.0] is null
            at org.junit.Assert.fail(Assert.java:88)
            at org.junit.Assert.assertTrue(Assert.java:41)
            at org.junit.Assert.assertNotNull(Assert.java:712)
            at org.elasticsearch.test.rest.yaml.section.MatchAssertion.doAssert(MatchAssertion.java:79)
            at org.elasticsearch.test.rest.yaml.section.Assertion.execute(Assertion.java:76)
            at org.elasticsearch.test.rest.yaml.ESClientYamlSuiteTestCase.executeSection(ESClientYamlSuiteTestCase.java:407)
            ... 38 more

Relates 3cbe0ea / 5e0b0c7 / d707f26
@dliappis dliappis added the >test-failure Triaged test failures from CI label Dec 10, 2020
@dliappis dliappis mentioned this issue Dec 10, 2020
Merged
@dliappis
Copy link
Contributor Author

This is directly related to the above mentioned commits, muting the test: #66149

dliappis added a commit that referenced this issue Dec 10, 2020
@dliappis
Mute fvh REST tests (#66149)
5b24829 
Relates #66147
dliappis added a commit to dliappis/elasticsearch that referenced this issue Dec 10, 2020
@dliappis
Mute fvh REST tests (elastic#66149)
5e2e07f 
Relates elastic#66147
dliappis added a commit to dliappis/elasticsearch that referenced this issue Dec 10, 2020
@dliappis
Mute fvh REST tests (elastic#66149)
6e9780f 
Relates elastic#66147
This was referenced Dec 10, 2020
Merged
Merged
dliappis added a commit that referenced this issue Dec 10, 2020
@dliappis
[7.x] Mute fvh REST tests (backport of #66149) #66151
23c025e 
Relates #66147
dliappis added a commit that referenced this issue Dec 10, 2020
@dliappis
[7.10] Mute fvh REST tests (backport of #66149) #66152
95dbc6d 
Relates #66147
@jtibshirani jtibshirani mentioned this issue Dec 10, 2020
Merged
@jtibshirani
Copy link
Contributor

Thanks @dliappis for muting this and sorry for the hassle -- I actually pushed those commits as a quick fix, but didn't run the right bwc test to verify! I opened #66192 to address the failures.

@jtibshirani jtibshirani added the :Search Relevance/Highlighting How a query matched a document label Dec 10, 2020
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Dec 10, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@javanna javanna added Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch and removed Team:Search Meta label for search team labels Jul 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jtibshirani jtibshirani

Labels
:Search Relevance/Highlighting How a query matched a document Team:Search Relevance Meta label for the Search Relevance team in Elasticsearch >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix failure in fvh REST tests related to hit order jtibshirani/elasticsearch
4 participants
@javanna @dliappis @jtibshirani @elasticmachine