[CI] Failure in monitoring/bulk/* #30101
Comments
|
Original comment by @droberts195: There's also a similar failure in a slightly different test in the same suite, namely "Bulk indexing of monitoring data on closed indices should throw an export exception". It failed in this build: LINK REDACTED The error was: The REPRO command was: As with the original one, this doesn't reproduce locally for me. |
|
Original comment by @jkakavas: Another one today As both of the original ones, this doesn't reproduce locally |
|
Original comment by @markharwood: I gave this a quick look and it failed to reproduce locally. I assumed there was a mismatch because test docs are presented with an |
elasticmachine
added
:Data Management/Monitoring
>test-failure
|
another one today this doesn't reproduce locally |
|
Note that there is a duplicate issue #30935 as well with further information from more test failures. |
|
and another today https://elasticsearch-ci.elastic.co/job/elastic+elasticsearch+6.x+intake/127/consoleText also not repro'ing locally |
|
Another case: |
|
Tried again on 6.7 and would not reproduce. Tried random seeds too |
|
This failed again in a PR today and has failed 3 times in the last month Given the latest changes in our muting strategy since we have an issue for this, I'm going to mute the test |
|
I have experienced a lot of failures in ...I found that the test is searching against Here's a patch that I used when looking into this.diff --git a/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/DoSection.java b/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/DoSection.java
index dd2da0d928b..dd71ad654c5 100644
--- a/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/DoSection.java
+++ b/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/DoSection.java
@@ -49,13 +49,16 @@ import java.util.Map;
import java.util.Objects;
import java.util.Set;
import java.util.TreeMap;
+import java.util.concurrent.TimeUnit;
import java.util.regex.Matcher;
import java.util.stream.Collectors;
import static java.util.Collections.emptyList;
+import static java.util.Collections.emptyMap;
import static java.util.Collections.unmodifiableList;
import static org.elasticsearch.common.collect.Tuple.tuple;
import static org.elasticsearch.common.logging.DeprecationLogger.WARNING_HEADER_PATTERN;
+import static org.elasticsearch.test.ESTestCase.assertBusy;
import static org.elasticsearch.test.hamcrest.RegexMatcher.matches;
import static org.hamcrest.Matchers.allOf;
import static org.hamcrest.Matchers.equalTo;
@@ -231,7 +234,7 @@ public class DoSection implements ExecutableSection {
}
@Override
- public void execute(ClientYamlTestExecutionContext executionContext) throws IOException {
+ public void execute(ClientYamlTestExecutionContext executionContext) throws Exception {
if ("param".equals(catchParam)) {
//client should throw validation error before sending request
@@ -240,8 +243,15 @@ public class DoSection implements ExecutableSection {
return;
}
+ final String api = apiCallSection.getApi();
+ final Map<String, String> params = apiCallSection.getParams();
+
+ if (api.equals("search") && params.containsKey("index")) {
+ waitForIndex(executionContext, params.get("index"));
+ }
+
try {
- ClientYamlTestResponse response = executionContext.callApi(apiCallSection.getApi(), apiCallSection.getParams(),
+ ClientYamlTestResponse response = executionContext.callApi(api, params,
apiCallSection.getBodies(), apiCallSection.getHeaders(), apiCallSection.getNodeSelector());
if (Strings.hasLength(catchParam)) {
String catchStatusCode;
@@ -493,4 +503,24 @@ public class DoSection implements ExecutableSection {
return lhs + "." + rhs;
}
}
+
+ private void waitForIndex(ClientYamlTestExecutionContext executionContext, String index) throws Exception {
+ assertBusy(() -> {
+ final Map<String, String> params = new HashMap<>();
+ params.put("h", "index");
+ params.put("s", "index");
+
+ final List<String> actualIndices = executionContext
+ .callApi("cat.indices", params, emptyList(), emptyMap())
+ .getBodyAsString()
+ .lines()
+ .collect(Collectors.toList());
+
+ final String indexAsRegex = "^" + index.replaceAll("\\*", ".*") + "$";
+
+ if (actualIndices.stream().noneMatch(each -> each.matches(indexAsRegex))) {
+ fail("Index [" + index + "] not found in " + actualIndices);
+ }
+ }, 20, TimeUnit.SECONDS);
+ }
}
diff --git a/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/ExecutableSection.java b/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/ExecutableSection.java
index 19690b23528..c4a1992fba1 100644
--- a/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/ExecutableSection.java
+++ b/test/framework/src/main/java/org/elasticsearch/test/rest/yaml/section/ExecutableSection.java
@@ -75,5 +75,5 @@ public interface ExecutableSection {
/**
* Executes the section passing in the execution context
*/
- void execute(ClientYamlTestExecutionContext executionContext) throws IOException;
+ void execute(ClientYamlTestExecutionContext executionContext) throws Exception;
} |
|
Ha, turns out that all you need to enforce the existence of the indices is |
|
As |
ywelsch
changed the title
|
The other test in |
These tests have been muted and an issue open for them for almost two years. This commit unmutes these tests to see if they still fail (beacuse they did not reproduce for me). If they still fail, we can either investigate the failures or rewrite the tests as Java integration tests if the failure is timing-based. If they do not fail, then all the better and the issue will remain closed. Resolves elastic#30101
|
I believe that this is closed by #93397, but this one is so old that it's hard to tell. I've run the tests dozens of times now and haven't seen any failures. |
Original comment by @droberts195:
This test failed in LINK REDACTED
The error was:
The REPRO command was:
This did not reproduce locally for me on the 6.x branch.
The text was updated successfully, but these errors were encountered: