[CI] HeapAttackIT testHugeConcat failing #100288
Comments
elasticsearchmachine
added
the
needs:triage
|
In fact, these all fail: |
ChrisHegarty
added
Team:QL (Deprecated)
elasticsearchmachine
removed
the
needs:triage
|
Pinging @elastic/es-ql (Team:QL) |
|
Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL) |
|
Are you sure? Or do they just fail because they run after and the breaker isn't reset? |
|
I mean, both are totally possible at this point. |
|
Or, rather, do they fail because we knock the node over in the huge concat test? |
That could very well be the case. |
|
Running testHugeConcat by itself I see no issues - it passes successfully many many times. If I modify testHugeConcat to loop 10 times - so just do what it is doing 10 times sequentially, I see failures. When I expect to see none. This is odd and kinda makes me think that we're holding on to something from a previous query execution - a real memory leak? Anyway, debugging the x10. run I see failure come mostly from the BytesRefVectorBuilder as it tries to deep copy humungous byte[]'s. A BytesRefArray with a single large entry is hard to extract, since it needs to copy out the internal byte data into a new byte[]. Internally, the BytesRefArray uses bigArrays rather than byte[]. Some screenshots of the failures I see. |
|
Options. In BytesRefVectorBuilder:
|
|
What a lovely catch! go go tests! I think if we're going to deep copy here we should use the |
|
well.. fixing this one place (by just removing the optimisation - just always create a BytesRefArrayVector even for an array with a single value), moves the problem :-) BigByteArray::get allocates a new byte[] to assign to the destination BytesRef, if the 
|
This adds a limit on the bytes length that a concatenated string can get to. The limit is set to 1MB (per entry). When the limit is hit, an exception is returned to the caller (similar to an accounting circuit breaking) and execution halted. Related: #100288.
This adds a limit on the bytes length that a concatenated string can get to. The limit is set to 1MB (per entry). When the limit is hit, an exception is returned to the caller (similar to an accounting circuit breaking) and execution halted. Related: elastic#100288.
|
Adressed in #100360. |
This adds a limit on the bytes length that a concatenated string can get to. The limit is set to 1MB (per entry). When the limit is hit, an exception is returned to the caller (similar to an accounting circuit breaking) and execution halted. Related: #100288.
Build scan:
https://gradle-enterprise.elastic.co/s/raslhxar3ujes/tests/:x-pack:plugin:esql:qa:server:single-node:javaRestTest/org.elasticsearch.xpack.esql.qa.single_node.HeapAttackIT/testHugeConcat
Reproduction line:
Applicable branches:
main
Reproduces locally?:
Yes
Failure history:
https://gradle-enterprise.elastic.co/scans/tests?tests.container=org.elasticsearch.xpack.esql.qa.single_node.HeapAttackIT&tests.test=testHugeConcat
Failure excerpt:
The text was updated successfully, but these errors were encountered: