Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] EarlyDeprecationIndexingIT testEarlyDeprecationIsIndexedAfterTemplateIsLoaded failing #104716

Closed
kingherc opened this issue Jan 24, 2024 · 4 comments
Closed

[CI] EarlyDeprecationIndexingIT testEarlyDeprecationIsIndexedAfterTemplateIsLoaded failing #104716

kingherc opened this issue Jan 24, 2024 · 4 comments
Assignees
@mosche
Labels
:Core/Infra/Logging Log management and logging utilities low-risk An open issue or test failure that is a low risk to future releases Team:Core/Infra Meta label for core/infra team >test-failure Triaged test failures from CI

Comments

@kingherc
Copy link
Contributor

Build scan:
https://gradle-enterprise.elastic.co/s/batvq7lqlwwiu/tests/:x-pack:plugin:deprecation:qa:early-deprecation-rest:javaRestTest/org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT/testEarlyDeprecationIsIndexedAfterTemplateIsLoaded

Reproduction line:

gradlew ':x-pack:plugin:deprecation:qa:early-deprecation-rest:javaRestTest' --tests "org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT.testEarlyDeprecationIsIndexedAfterTemplateIsLoaded" -Dtests.seed=E63B53025A891B68 -Dtests.locale=ga-IE -Dtests.timezone=Etc/GMT-13 -Druntime.java=21

Applicable branches:
8.12

Reproduces locally?:
Didn't try

Failure history:
Failure dashboard for org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT#testEarlyDeprecationIsIndexedAfterTemplateIsLoaded

Failure excerpt:

java.lang.AssertionError: org.elasticsearch.client.ResponseException: method [GET], host [http://127.0.0.1:50229], URI [/.logs-deprecation.elasticsearch-default/_settings], status line [HTTP/1.1 404 Not Found]
{"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [.logs-deprecation.elasticsearch-default]","resource.type":"index_or_alias","resource.id":".logs-deprecation.elasticsearch-default","index_uuid":"_na_","index":".logs-deprecation.elasticsearch-default"}],"type":"index_not_found_exception","reason":"no such index [.logs-deprecation.elasticsearch-default]","resource.type":"index_or_alias","resource.id":".logs-deprecation.elasticsearch-default","index_uuid":"_na_","index":".logs-deprecation.elasticsearch-default"},"status":404}

  at __randomizedtesting.SeedInfo.seed([E63B53025A891B68:7AAB18056B540002]:0)
  at org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT.getIndexSettings(EarlyDeprecationIndexingIT.java:81)
  at org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT.lambda$testEarlyDeprecationIsIndexedAfterTemplateIsLoaded$0(EarlyDeprecationIndexingIT.java:47)
  at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:1278)
  at org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT.testEarlyDeprecationIsIndexedAfterTemplateIsLoaded(EarlyDeprecationIndexingIT.java:46)
  at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
  at java.lang.reflect.Method.invoke(Method.java:580)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
  at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
  at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
  at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
  at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
  at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
  at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
  at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
  at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
  at java.lang.Thread.run(Thread.java:1583)

  Caused by: org.elasticsearch.client.ResponseException: method [GET], host [http://127.0.0.1:50229], URI [/.logs-deprecation.elasticsearch-default/_settings], status line [HTTP/1.1 404 Not Found]
  {"error":{"root_cause":[{"type":"index_not_found_exception","reason":"no such index [.logs-deprecation.elasticsearch-default]","resource.type":"index_or_alias","resource.id":".logs-deprecation.elasticsearch-default","index_uuid":"_na_","index":".logs-deprecation.elasticsearch-default"}],"type":"index_not_found_exception","reason":"no such index [.logs-deprecation.elasticsearch-default]","resource.type":"index_or_alias","resource.id":".logs-deprecation.elasticsearch-default","index_uuid":"_na_","index":".logs-deprecation.elasticsearch-default"},"status":404}

    at org.elasticsearch.client.RestClient.convertResponse(RestClient.java:351)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:317)
    at org.elasticsearch.client.RestClient.performRequest(RestClient.java:292)
    at org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT.getIndexSettings(EarlyDeprecationIndexingIT.java:77)
    at org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT.lambda$testEarlyDeprecationIsIndexedAfterTemplateIsLoaded$0(EarlyDeprecationIndexingIT.java:47)
    at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:1278)
    at org.elasticsearch.xpack.deprecation.EarlyDeprecationIndexingIT.testEarlyDeprecationIsIndexedAfterTemplateIsLoaded(EarlyDeprecationIndexingIT.java:46)
    at jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
    at java.lang.reflect.Method.invoke(Method.java:580)
    at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1758)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:946)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:982)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:996)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at org.apache.lucene.tests.util.TestRuleSetupTeardownChained$1.evaluate(TestRuleSetupTeardownChained.java:48)
    at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
    at org.apache.lucene.tests.util.TestRuleThreadAndTestName$1.evaluate(TestRuleThreadAndTestName.java:45)
    at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
    at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl.forkTimeoutingTask(ThreadLeakControl.java:843)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl$3.evaluate(ThreadLeakControl.java:490)
    at com.carrotsearch.randomizedtesting.RandomizedRunner.runSingleTest(RandomizedRunner.java:955)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$5.evaluate(RandomizedRunner.java:840)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$6.evaluate(RandomizedRunner.java:891)
    at com.carrotsearch.randomizedtesting.RandomizedRunner$7.evaluate(RandomizedRunner.java:902)
    at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at org.apache.lucene.tests.util.TestRuleStoreClassName$1.evaluate(TestRuleStoreClassName.java:38)
    at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
    at com.carrotsearch.randomizedtesting.rules.NoShadowingOrOverridesOnMethodsRule$1.evaluate(NoShadowingOrOverridesOnMethodsRule.java:40)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at org.apache.lucene.tests.util.TestRuleAssertionsRequired$1.evaluate(TestRuleAssertionsRequired.java:53)
    at org.apache.lucene.tests.util.AbstractBeforeAfterRule$1.evaluate(AbstractBeforeAfterRule.java:43)
    at org.apache.lucene.tests.util.TestRuleMarkFailure$1.evaluate(TestRuleMarkFailure.java:44)
    at org.apache.lucene.tests.util.TestRuleIgnoreAfterMaxFailures$1.evaluate(TestRuleIgnoreAfterMaxFailures.java:60)
    at org.apache.lucene.tests.util.TestRuleIgnoreTestSuites$1.evaluate(TestRuleIgnoreTestSuites.java:47)
    at com.carrotsearch.randomizedtesting.rules.StatementAdapter.evaluate(StatementAdapter.java:36)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl$StatementRunner.run(ThreadLeakControl.java:390)
    at com.carrotsearch.randomizedtesting.ThreadLeakControl.lambda$forkTimeoutingTask$0(ThreadLeakControl.java:850)
    at java.lang.Thread.run(Thread.java:1583)
@kingherc kingherc added :Core/Infra/Logging Log management and logging utilities >test-failure Triaged test failures from CI Team:Core/Infra Meta label for core/infra team labels Jan 24, 2024
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (Team:Core/Infra)

@thecoop thecoop added low-risk An open issue or test failure that is a low risk to future releases and removed blocker labels Jan 25, 2024
pgomulka added a commit to pgomulka/elasticsearch that referenced this issue Feb 21, 2024
@pgomulka
Improve EarlyDeprecationindexingIT test reliability
c7c8ee5 
this test intends to test the bulkProcessor2's request consumer before the templates are initialised.
This requires the flush to happen before the templates are loaded.
To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now)

This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured.
It also adds additional trace logging to help with troubleshooting.

relates elastic#104716
@pgomulka pgomulka mentioned this issue Feb 21, 2024
Merged
pgomulka added a commit that referenced this issue Feb 21, 2024
@pgomulka
Improve EarlyDeprecationindexingIT test reliability (#105696)
280fa40 
this test intends to test the bulkProcessor2's request consumer (see DeprecationIndexingComponent#getBulkProcessor) scheduling requests before the startup is completed (flush is enabled). To verify this behaviour the flush has to happen before the templates are loaded. To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now)

This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured. It also adds additional trace logging to help with troubleshooting.

relates #104716
@pgomulka
Copy link
Contributor

I tweaked the test a little (to make sure that it always covers the early deprecation indexing) and enabled trace logging in #105696
I will remove the trace logging once this issue is resolved

@albertzaharovits
Copy link
Contributor

@pgomulka here's a fresh failure for this one https://gradle-enterprise.elastic.co/s/qca37bsec4djs

pgomulka added a commit to pgomulka/elasticsearch that referenced this issue Apr 16, 2024
@pgomulka
Improve EarlyDeprecationindexingIT test reliability (elastic#105696)
c958ea2 
this test intends to test the bulkProcessor2's request consumer (see DeprecationIndexingComponent#getBulkProcessor) scheduling requests before the startup is completed (flush is enabled). To verify this behaviour the flush has to happen before the templates are loaded. To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now)

This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured. It also adds additional trace logging to help with troubleshooting.

relates elastic#104716
@pgomulka pgomulka mentioned this issue Apr 16, 2024
Merged
pgomulka added a commit that referenced this issue Apr 16, 2024
@pgomulka
Improve EarlyDeprecationindexingIT test reliability backport(#105696) #…
8c36f08 
…107540

this test intends to test the bulkProcessor2's request consumer (see DeprecationIndexingComponent#getBulkProcessor) scheduling requests before the startup is completed (flush is enabled). To verify this behaviour the flush has to happen before the templates are loaded. To test this reliably the flush interval in the test should be as small as possible (not hardcoded 5s as of now)

This commit introduces a setting (not meant to be exposed/documented) to allow for the flush interval to be configured. It also adds additional trace logging to help with troubleshooting.

relates #104716
@rjernst rjernst assigned rjernst and mosche and unassigned pgomulka and rjernst Jul 24, 2024
@elasticsearchmachine elasticsearchmachine closed this as not planned Won't fix, can't repro, duplicate, stale Nov 5, 2024
@elasticsearchmachine
Copy link
Collaborator

This issue has been closed because it has been open for too long with no activity.

Any muted tests that were associated with this issue have been unmuted.

If the tests begin failing again, a new issue will be opened, and they may be muted again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mosche mosche

Labels
:Core/Infra/Logging Log management and logging utilities low-risk An open issue or test failure that is a low risk to future releases Team:Core/Infra Meta label for core/infra team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@rjernst @mosche @thecoop @albertzaharovits @kingherc @pgomulka @elasticsearchmachine