Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CI] EsqlDisruptionIT testFromSortWithTieBreakerLimit failing #100105

Closed
stu-elastic opened this issue Sep 29, 2023 · 2 comments · Fixed by #100108
Closed

[CI] EsqlDisruptionIT testFromSortWithTieBreakerLimit failing #100105

stu-elastic opened this issue Sep 29, 2023 · 2 comments · Fixed by #100108
Labels
:Analytics/ES|QL AKA ESQL Team:QL (Deprecated) Meta label for query languages team >test-failure Triaged test failures from CI

Comments

@stu-elastic
Copy link
Contributor

Build scan:
https://gradle-enterprise.elastic.co/s/lfe6he6gbt5g4/tests/:x-pack:plugin:esql:internalClusterTest/org.elasticsearch.xpack.esql.action.EsqlDisruptionIT/testFromSortWithTieBreakerLimit

Reproduction line:

gradlew ':x-pack:plugin:esql:internalClusterTest' --tests "org.elasticsearch.xpack.esql.action.EsqlDisruptionIT.testFromSortWithTieBreakerLimit" -Dtests.seed=9EC73CFBC60F4A -Dtests.locale=ar-AE -Dtests.timezone=Atlantic/Madeira -Druntime.java=21

Applicable branches:
main

Reproduces locally?:
No

Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.xpack.esql.action.EsqlDisruptionIT&tests.test=testFromSortWithTieBreakerLimit

Failure excerpt:

org.elasticsearch.common.breaker.CircuitBreakingException: [request] Data too large, data for [topn] would be [1040/1kb], which is larger than the limit of [1024/1kb]

  at __randomizedtesting.SeedInfo.seed([9EC73CFBC60F4A:828B82120B38619F]:0)
  at org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker.circuitBreak(ChildMemoryCircuitBreaker.java:77)
  at org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker.limit(ChildMemoryCircuitBreaker.java:167)
  at org.elasticsearch.common.breaker.ChildMemoryCircuitBreaker.addEstimateBytesAndMaybeBreak(ChildMemoryCircuitBreaker.java:104)
  at org.elasticsearch.compute.operator.BreakingBytesRefBuilder.grow(BreakingBytesRefBuilder.java:60)
  at org.elasticsearch.compute.operator.topn.SortableTopNEncoder.encodeLong(SortableTopNEncoder.java:20)
  at org.elasticsearch.compute.operator.topn.KeyExtractorForLong.nonNul(KeyExtractorForLong.java:41)
  at org.elasticsearch.compute.operator.topn.KeyExtractorForLong$ForVector.writeKey(KeyExtractorForLong.java:60)
  at org.elasticsearch.compute.operator.topn.TopNOperator$RowFiller.writeKey(TopNOperator.java:148)
  at org.elasticsearch.compute.operator.topn.TopNOperator$RowFiller.row(TopNOperator.java:141)
  at org.elasticsearch.compute.operator.topn.TopNOperator.addInput(TopNOperator.java:327)
  at org.elasticsearch.compute.operator.Driver.runSingleLoopIteration(Driver.java:191)
  at org.elasticsearch.compute.operator.Driver.run(Driver.java:129)
  at org.elasticsearch.compute.operator.Driver$1.doRun(Driver.java:285)
  at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:33)
  at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:983)
  at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:26)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
  at java.lang.Thread.run(Thread.java:1583)
@stu-elastic stu-elastic added :Analytics/ES|QL AKA ESQL >test-failure Triaged test failures from CI labels Sep 29, 2023
@elasticsearchmachine elasticsearchmachine added the Team:QL (Deprecated) Meta label for query languages team label Sep 29, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-ql (Team:QL)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL)

@dnhatn dnhatn mentioned this issue Oct 1, 2023
Merged
dnhatn added a commit that referenced this issue Oct 1, 2023
@dnhatn
Remove global block factory (#100108)
dbb8b7d 
The global BlockFactory should work fine in production, where each 
Elasticsearch node runs in its own JVM process. However, this approach
can lead to issues during testing, especially in IT tests. The same JVM
process might get reused across multiple tests, resulting in situations
where multiple IT tests inadvertently use the same instance of the
global BlockFactory.

For instance, EsqlDisruptionIT fails because it accidentally uses the 
global BlockFactory initialized by EsqlActionBreakerIT, which has a 
limit set to 1KB. Another issue in IT tests is that multiple
Elasticsearch nodes can share the same (single) global instance of the
BlockFactory.

Closes #100105
piergm pushed a commit to piergm/elasticsearch that referenced this issue Oct 2, 2023
@dnhatn @piergm
Remove global block factory (elastic#100108)
2633d93 
The global BlockFactory should work fine in production, where each 
Elasticsearch node runs in its own JVM process. However, this approach
can lead to issues during testing, especially in IT tests. The same JVM
process might get reused across multiple tests, resulting in situations
where multiple IT tests inadvertently use the same instance of the
global BlockFactory.

For instance, EsqlDisruptionIT fails because it accidentally uses the 
global BlockFactory initialized by EsqlActionBreakerIT, which has a 
limit set to 1KB. Another issue in IT tests is that multiple
Elasticsearch nodes can share the same (single) global instance of the
BlockFactory.

Closes elastic#100105
jakelandis pushed a commit to jakelandis/elasticsearch that referenced this issue Oct 2, 2023
@dnhatn @jakelandis
Remove global block factory (elastic#100108)
454b17a 
The global BlockFactory should work fine in production, where each 
Elasticsearch node runs in its own JVM process. However, this approach
can lead to issues during testing, especially in IT tests. The same JVM
process might get reused across multiple tests, resulting in situations
where multiple IT tests inadvertently use the same instance of the
global BlockFactory.

For instance, EsqlDisruptionIT fails because it accidentally uses the 
global BlockFactory initialized by EsqlActionBreakerIT, which has a 
limit set to 1KB. Another issue in IT tests is that multiple
Elasticsearch nodes can share the same (single) global instance of the
BlockFactory.

Closes elastic#100105
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
:Analytics/ES|QL AKA ESQL Team:QL (Deprecated) Meta label for query languages team >test-failure Triaged test failures from CI
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove global block factory dnhatn/elasticsearch
2 participants
@stu-elastic @elasticsearchmachine