[CI] Failure in TimeThrottleIntegrationTests.testTimeThrottle #30003

elasticmachine · 2018-04-11T12:28:37Z

Original comment by @albertzaharovits:

Failure is in this build: LINK REDACTED
Repro cmd which is not working locally is:

./gradlew :x-pack-elasticsearch:plugin:watcher:test \
  -Dtests.seed=FE582B72FA7C67FE \
  -Dtests.class=org.elasticsearch.xpack.watcher.actions.TimeThrottleIntegrationTests \
  -Dtests.method="testTimeThrottle" \
  -Dtests.security.manager=true \
  -Dtests.locale=es-NI \
  -Dtests.timezone=Asia/Khandyga

The symptom is:

1:32:30    > Throwable LINK REDACTED: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 	at __randomizedtesting.SeedInfo.seed([FE582B72FA7C67FE:10F46257D65AAB07]:0)
11:32:30    > 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 	at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:734)
11:32:30    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:708)
11:32:30    > 	at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.startWatcher(AbstractWatcherIntegrationTestCase.java:465)
11:32:30    > 	at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase._setup(AbstractWatcherIntegrationTestCase.java:207)
11:32:30    > 	at java.lang.Thread.run(Thread.java:748)
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STOPPED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTING], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STARTING>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more
11:32:30    > 	Suppressed: java.lang.AssertionError: Expected watcher to be started, but state was [Tuple [v1=node_s1, v2=STARTED], Tuple [v1=node_s0, v2=STOPPED]]
11:32:30    > Expected: every item is is <STARTED>
11:32:30    >      but: an item was <STOPPED>
11:32:30    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
11:32:30    > 		at org.elasticsearch.xpack.watcher.test.AbstractWatcherIntegrationTestCase.lambda$startWatcher$6(AbstractWatcherIntegrationTestCase.java:479)
11:32:30    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:722)
11:32:30    > 		... 39 more

This is similar to others, such as LINK REDACTED and LINK REDACTED so presumably fixing the underlying cause will solve some of them at least.

The text was updated successfully, but these errors were encountered:

elasticmachine · 2018-04-13T12:41:16Z

Original comment by @javanna:

A different test, but exactly the same symptoms:
LINK REDACTED

REPRODUCE WITH: ./gradlew :x-pack-elasticsearch:plugin:watcher:test \
  -Dtests.seed=B7FFE970B7ADF3AF \
  -Dtests.class=org.elasticsearch.xpack.watcher.actions.webhook.WebhookHttpsIntegrationTests \
  -Dtests.method="testHttpsAndBasicAuth" \
  -Dtests.security.manager=true \
  -Dtests.locale=de-AT \
  -Dtests.timezone=Europe/Tirane

REPRODUCE WITH: ./gradlew :x-pack-elasticsearch:plugin:watcher:test \
  -Dtests.seed=B7FFE970B7ADF3AF \
  -Dtests.class=org.elasticsearch.xpack.watcher.actions.webhook.WebhookHttpsIntegrationTests \
  -Dtests.method="testHttpsAndBasicAuth" \
  -Dtests.security.manager=true \
  -Dtests.locale=de-AT \
  -Dtests.timezone=Europe/Tirane

@spinscale I went for posting it here instead of opening another issue, let me know if I should raise another one.

elasticmachine · 2018-04-13T12:46:59Z

Original comment by @spinscale:

all good, I opened LINK REDACTED for this

elasticmachine · 2018-04-17T10:05:04Z

Original comment by @dimitris-athanasiou:

This seems somewhat related as well: LINK REDACTED

REPRODUCE WITH: ./gradlew :x-pack-elasticsearch:qa:smoke-test-watcher-with-security:integTestRunner -Dtests.seed=301BC8944397CEE3 -Dtests.class=org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT -Dtests.method="testSearchInputHasPermissions" -Dtests.security.manager=true -Dtests.locale=es-DO -Dtests.timezone=Asia/Baku

Fails with:

Throwable LINK REDACTED: java.lang.AssertionError: 
09:01:15    > Expected: is "started"
09:01:15    >      but: was "stopped"
09:01:15    > 	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
09:01:15    > 	at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
09:01:15    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:731)
09:01:15    > 	at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:705)
09:01:15    > 	at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.startWatcher(SmokeTestWatcherWithSecurityIT.java:63)
09:01:15    > 	at java.lang.Thread.run(Thread.java:748)
09:01:15    > 	Suppressed: java.lang.AssertionError: 
09:01:15    > Expected: is "started"
09:01:15    >      but: was "stopping"
09:01:15    > 		at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
09:01:15    > 		at org.elasticsearch.smoketest.SmokeTestWatcherWithSecurityIT.lambda$startWatcher$0(SmokeTestWatcherWithSecurityIT.java:74)
09:01:15    > 		at org.elasticsearch.test.ESTestCase.assertBusy(ESTestCase.java:719)
09:01:15    > 		... 38 more

elasticmachine · 2018-04-17T13:04:00Z

Original comment by @dimitris-athanasiou:

More tests failing due to start/stop problems: LINK REDACTED

elasticmachine · 2018-04-19T10:29:20Z

Original comment by @romseygeek:

This also looks like it's due to Watcher start/stop issues: LINK REDACTED

jasontedor · 2018-08-21T14:20:07Z

Closed by #30118

elasticmachine added :Data Management/Watcher >test-failure Triaged test failures from CI labels Apr 25, 2018

elasticmachine assigned spinscale Apr 25, 2018

jasontedor closed this as completed Aug 21, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[CI] Failure in TimeThrottleIntegrationTests.testTimeThrottle #30003

[CI] Failure in TimeThrottleIntegrationTests.testTimeThrottle #30003

elasticmachine commented Apr 11, 2018

elasticmachine commented Apr 13, 2018

elasticmachine commented Apr 13, 2018

elasticmachine commented Apr 17, 2018

elasticmachine commented Apr 17, 2018

elasticmachine commented Apr 19, 2018

jasontedor commented Aug 21, 2018

[CI] Failure in TimeThrottleIntegrationTests.testTimeThrottle #30003

[CI] Failure in TimeThrottleIntegrationTests.testTimeThrottle #30003

Comments

elasticmachine commented Apr 11, 2018

elasticmachine commented Apr 13, 2018

elasticmachine commented Apr 13, 2018

elasticmachine commented Apr 17, 2018

elasticmachine commented Apr 17, 2018

elasticmachine commented Apr 19, 2018

jasontedor commented Aug 21, 2018