Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] XPackRestIT class failing #114723

Closed
elasticsearchmachine opened this issue Oct 14, 2024 · 6 comments
Closed

[CI] XPackRestIT class failing #114723

elasticsearchmachine opened this issue Oct 14, 2024 · 6 comments
Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) Team:obs-knowledge Meta label for Observability Knowledge team >test-failure Triaged test failures from CI :UniversalProfiling/Application Elastic Universal Profiling REST APIs and infrastructure

Comments

@elasticsearchmachine
Copy link
Collaborator

elasticsearchmachine commented Oct 14, 2024 •
edited
Loading

Build Scans:

Reproduction Line:

undefined

Applicable branches:
main

Reproduces locally?:
N/A

Failure History:
See dashboard

Failure Message:

undefined

Issue Reasons:

  • [main] 46 failures in class org.elasticsearch.xpack.test.rest.XPackRestIT (4.6% fail rate in 993 executions)
  • [main] 26 failures in step rest-compatibility (12.5% fail rate in 208 executions)
  • [main] 7 failures in step part-3 (3.8% fail rate in 186 executions)
  • [main] 3 failures in step part3 (4.2% fail rate in 72 executions)
  • [main] 3 failures in pipeline elasticsearch-intake (4.0% fail rate in 75 executions)
  • [main] 7 failures in pipeline elasticsearch-periodic-platform-support (53.8% fail rate in 13 executions)
  • [main] 28 failures in pipeline elasticsearch-pull-request (13.2% fail rate in 212 executions)

Note:
This issue was created using new test triage automation. Please report issues or feedback to es-delivery.
@elasticsearchmachine elasticsearchmachine added :ml Machine learning >test-failure Triaged test failures from CI labels Oct 14, 2024
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch 8.x

Mute Reasons:

  • [8.x] 4 failures in class org.elasticsearch.xpack.test.rest.XPackRestIT (0.4% fail rate in 904 executions)
  • [8.x] 2 failures in step amazonlinux-2023_platform-support-aws (33.3% fail rate in 6 executions)
  • [8.x] 3 failures in pipeline elasticsearch-periodic-platform-support (37.5% fail rate in 8 executions)

Build Scans:

elasticsearchmachine added a commit that referenced this issue Oct 14, 2024
@elasticsearchmachine
Mute org.elasticsearch.xpack.test.rest.XPackRestIT org.elasticsearch.…
599ac16 
…xpack.test.rest.XPackRestIT #114723
@elasticsearchmachine elasticsearchmachine added Team:ML Meta label for the ML team needs:risk Requires assignment of a risk label (low, medium, blocker) labels Oct 14, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/ml-core (Team:ML)

@davidkyle davidkyle mentioned this issue Oct 15, 2024
Merged
@davidkyle
Copy link
Member

Many tests failed in this suite because the node died with an uncaught exception.

This log excerpt is from the first link in the comment above elasticsearch-periodic-platform-support #4385 / amazonlinux-2023_platform-support-aws

[2024-10-15T01:08:53,421][ERROR][o.e.b.ElasticsearchUncaughtExceptionHandler] [yamlRestTest-0] fatal error in thread [elasticsearch[yamlRestTest-0][profiling][T#3]], exiting
java.lang.AssertionError: [org.elasticsearch.action.ActionListenerImplementations$RunBeforeActionListener/org.elasticsearch.tasks.TaskManager$1{WrappedActionListener{org.elasticsearch.xpack.profiling.action.TransportGetTopNFunctionsAction$$Lambda/0x00007fd97c8eeba8@fc0ff71}{org.elasticsearch.xpack.profiling.action.TransportGetTopNFunctionsAction$$Lambda/0x00007fd97c8eedd8@69b559a4}}{CancellableTask{Task{id=22182, type='transport', action='indices:data/read/profiling/stack_traces', description='null', parentTask=h0VCn-mGReqDPvG34ulOUw:22181, startTime=1728911333398, headers={}, startTimeNanos=3055553755034}, reason='null', isCancelled=false}}/org.elasticsearch.action.support.TransportAction$$Lambda/0x00007fd97c316a40@45e80b84] org.elasticsearch.ElasticsearchException: executed already
	at org.elasticsearch.action.ActionListener$3.assertFirstRun(ActionListener.java:393) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.ActionListener$3.onFailure(ActionListener.java:409) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.DelegatingActionListener.onFailure(DelegatingActionListener.java:32) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.support.ContextPreservingActionListener.onFailure(ContextPreservingActionListener.java:40) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.ActionListenerImplementations.safeAcceptException(ActionListenerImplementations.java:64) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.ActionListenerImplementations.safeOnFailure(ActionListenerImplementations.java:75) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.DelegatingActionListener.onFailure(DelegatingActionListener.java:32) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.xpack.profiling.action.TransportGetStackTracesAction$DetailsHandler.onStackFramesResponse(TransportGetStackTracesAction.java:770) ~[?:?]
	at org.elasticsearch.action.ActionListener$2.onResponse(ActionListener.java:257) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.action.support.RefCountAwareThreadedActionListener$1.doRun(RefCountAwareThreadedActionListener.java:39) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:992) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:27) ~[elasticsearch-8.16.0-SNAPSHOT.jar:?]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642) ~[?:?]
	at java.lang.Thread.run(Thread.java:1570) ~[?:?]

@davidkyle
Copy link
Member

@elastic/obs-knowledge-team can you take a look at the assertion failure inTransportGetTopNFunctionsAction please. Stack trace in the comment above

@davidkyle davidkyle added the Team:obs-knowledge Meta label for Observability Knowledge team label Oct 15, 2024
@elasticsearchmachine elasticsearchmachine removed the Team:obs-knowledge Meta label for Observability Knowledge team label Oct 15, 2024
davidkyle added a commit that referenced this issue Oct 15, 2024
@davidkyle
Unmute XPackRestIT (#114816)
60443be 
Related to #114723
@davidkyle davidkyle added Team:obs-knowledge Meta label for Observability Knowledge team and removed :ml Machine learning Team:ML Meta label for the ML team labels Oct 22, 2024
@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label and removed Team:obs-knowledge Meta label for Observability Knowledge team labels Oct 22, 2024
@astefan astefan added Team:obs-knowledge Meta label for Observability Knowledge team and removed needs:triage Requires assignment of a team area label labels Oct 25, 2024
@elasticsearchmachine elasticsearchmachine added needs:triage Requires assignment of a team area label and removed Team:obs-knowledge Meta label for Observability Knowledge team labels Oct 25, 2024
@benwtrent benwtrent added the :UniversalProfiling/Application Elastic Universal Profiling REST APIs and infrastructure label Oct 29, 2024
@elasticsearchmachine elasticsearchmachine added the Team:obs-knowledge Meta label for Observability Knowledge team label Oct 29, 2024
@elasticsearchmachine
Copy link
Collaborator Author

Pinging @elastic/obs-knowledge-team (Team:obs-knowledge)

@elasticsearchmachine elasticsearchmachine removed the needs:triage Requires assignment of a team area label label Oct 29, 2024
@elasticsearchmachine
Copy link
Collaborator Author

This has been muted on branch main

Mute Reasons:

  • [main] 45 failures in class org.elasticsearch.xpack.test.rest.XPackRestIT (4.5% fail rate in 1000 executions)
  • [main] 27 failures in step rest-compatibility (12.7% fail rate in 212 executions)
  • [main] 7 failures in step part-3 (3.7% fail rate in 191 executions)
  • [main] 3 failures in step part3 (3.8% fail rate in 78 executions)
  • [main] 7 failures in pipeline elasticsearch-periodic-platform-support (53.8% fail rate in 13 executions)
  • [main] 29 failures in pipeline elasticsearch-pull-request (13.5% fail rate in 215 executions)
  • [main] 3 failures in pipeline elasticsearch-intake (3.8% fail rate in 78 executions)

Build Scans:

elasticsearchmachine added a commit that referenced this issue Nov 6, 2024
@elasticsearchmachine
Mute org.elasticsearch.xpack.test.rest.XPackRestIT org.elasticsearch.…
d21452e 
…xpack.test.rest.XPackRestIT #114723
@kkrik-es kkrik-es mentioned this issue Nov 7, 2024
Merged
@kkrik-es kkrik-es closed this as completed Nov 7, 2024
jozala pushed a commit that referenced this issue Nov 13, 2024
@elasticsearchmachine @jozala
Mute org.elasticsearch.xpack.test.rest.XPackRestIT org.elasticsearch.…
ad61fe4 
…xpack.test.rest.XPackRestIT #114723
alexey-ivanov-es pushed a commit to alexey-ivanov-es/elasticsearch that referenced this issue Nov 28, 2024
@elasticsearchmachine @alexey-ivanov-es
Mute org.elasticsearch.xpack.test.rest.XPackRestIT org.elasticsearch.…
557778a 
…xpack.test.rest.XPackRestIT elastic#114723
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs:risk Requires assignment of a risk label (low, medium, blocker) Team:obs-knowledge Meta label for Observability Knowledge team >test-failure Triaged test failures from CI :UniversalProfiling/Application Elastic Universal Profiling REST APIs and infrastructure
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@astefan @davidkyle @benwtrent @elasticsearchmachine @kkrik-es