[CI] TasksIT testTasksCancellation failing #109686
Assignees
Labels
:Distributed Coordination/Task Management
Issues for anything around the Tasks API - both persistent and node level.
needs:risk
Requires assignment of a risk label (low, medium, blocker)
Team:Distributed (Obsolete)
Meta label for distributed team (obsolete). Replaced by Distributed Indexing/Coordination.
>test-failure
Triaged test failures from CI
Comments
piergm
added
:Distributed Coordination/Task Management
elasticsearchmachine
added
the
needs:risk
|
Pinging @elastic/es-distributed (Team:Distributed) |
|
I've seen this fail in both |
|
Ah, it's a consequence of #94865, there's now a chance that the task still exists even after the response has been received. This'll reproduce it reliably: diff --git a/server/src/main/java/org/elasticsearch/transport/TaskTransportChannel.java b/server/src/main/java/org/elasticsearch/transport/TaskTransportChannel.java
index d6a5b3436c9..ecc25c0eb77 100644
--- a/server/src/main/java/org/elasticsearch/transport/TaskTransportChannel.java
+++ b/server/src/main/java/org/elasticsearch/transport/TaskTransportChannel.java
@@ -43,6 +43,11 @@ public class TaskTransportChannel implements TransportChannel {
try {
channel.sendResponse(exception);
} finally {
+ try {
+ Thread.sleep(500);
+ } catch (InterruptedException e) {
+ throw new RuntimeException(e);
+ }
onTaskFinished.close();
}
} |
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this issue
Jun 19, 2024
The tasks are removed from the task manager _after_ sending the response, so we cannot reliably assert they're done. With this commit we wait for them to complete properly first. Closes elastic#109686
elasticsearchmachine
pushed a commit
that referenced
this issue
Jun 19, 2024
DaveCTurner
added a commit
to DaveCTurner/elasticsearch
that referenced
this issue
Jun 19, 2024
The tasks are removed from the task manager _after_ sending the response, so we cannot reliably assert they're done. With this commit we wait for them to complete properly first. Closes elastic#109686
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Build scan:
https://gradle-enterprise.elastic.co/s/ioneklonkjxya/tests/:server:internalClusterTest/org.elasticsearch.action.admin.cluster.node.tasks.TasksIT/testTasksCancellation
Reproduction line:
Applicable branches:
main
Reproduces locally?:
No
Failure history:
Failure dashboard for
org.elasticsearch.action.admin.cluster.node.tasks.TasksIT#testTasksCancellationFailure excerpt:
The text was updated successfully, but these errors were encountered: