Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove global block factory #100108

Merged
merged 2 commits into from
Oct 1, 2023
Merged

Remove global block factory #100108

merged 2 commits into from
Oct 1, 2023

Conversation

dnhatn
Copy link
Member

@dnhatn dnhatn commented Sep 30, 2023

The global BlockFactory should work fine in production, where each Elasticsearch node runs in its own JVM process. However, this approach can lead to issues during testing, especially in IT tests. The same JVM process might get reused across multiple tests, resulting in situations where multiple IT tests inadvertently use the same instance of the global BlockFactory.

For instance, EsqlDisruptionIT fails because it accidentally uses the global BlockFactory initialized by EsqlActionBreakerIT, which has a limit set to 1KB. Another issue in IT tests is that multiple Elasticsearch nodes can share the same (single) global instance of the BlockFactory.

Closes #100105

This PR suggests removing the global BlockFactory since we are already passing it with the DriverContext.

@dnhatn
Copy link
Member Author

dnhatn commented Sep 30, 2023

@nik9000 EsqlDisruptionIT seems to fail more frequently on your machine compared to mine, possibly due to the different number of CPUs. I was able to reproduce reliably the problem on my machine using org.gradle.workers.max=1.

@dnhatn dnhatn marked this pull request as ready for review September 30, 2023 06:41
@elasticsearchmachine elasticsearchmachine added the needs:triage Requires assignment of a team area label label Sep 30, 2023
@dnhatn dnhatn added :Analytics/ES|QL AKA ESQL >non-issue Team:QL (Deprecated) Meta label for query languages team and removed needs:triage Requires assignment of a team area label labels Sep 30, 2023
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-ql (Team:QL)

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL)

Copy link
Member

@costin costin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - less code and easier to track it down.

Copy link
Contributor

@ChrisHegarty ChrisHegarty left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@dnhatn
Copy link
Member Author

dnhatn commented Oct 1, 2023

@costin @ChrisHegarty Thanks for reviewing.

@dnhatn dnhatn merged commit dbb8b7d into elastic:main Oct 1, 2023
@dnhatn dnhatn deleted the block-factory branch October 1, 2023 19:52
piergm pushed a commit to piergm/elasticsearch that referenced this pull request Oct 2, 2023
The global BlockFactory should work fine in production, where each 
Elasticsearch node runs in its own JVM process. However, this approach
can lead to issues during testing, especially in IT tests. The same JVM
process might get reused across multiple tests, resulting in situations
where multiple IT tests inadvertently use the same instance of the
global BlockFactory.

For instance, EsqlDisruptionIT fails because it accidentally uses the 
global BlockFactory initialized by EsqlActionBreakerIT, which has a 
limit set to 1KB. Another issue in IT tests is that multiple
Elasticsearch nodes can share the same (single) global instance of the
BlockFactory.

Closes elastic#100105
jakelandis pushed a commit to jakelandis/elasticsearch that referenced this pull request Oct 2, 2023
The global BlockFactory should work fine in production, where each 
Elasticsearch node runs in its own JVM process. However, this approach
can lead to issues during testing, especially in IT tests. The same JVM
process might get reused across multiple tests, resulting in situations
where multiple IT tests inadvertently use the same instance of the
global BlockFactory.

For instance, EsqlDisruptionIT fails because it accidentally uses the 
global BlockFactory initialized by EsqlActionBreakerIT, which has a 
limit set to 1KB. Another issue in IT tests is that multiple
Elasticsearch nodes can share the same (single) global instance of the
BlockFactory.

Closes elastic#100105
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Analytics/ES|QL AKA ESQL >non-issue Team:QL (Deprecated) Meta label for query languages team v8.11.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[CI] EsqlDisruptionIT testFromSortWithTieBreakerLimit failing
4 participants