ESQL: Track memory from values loaded from lucene #101383
Conversation
This adds memory tracking for values loaded from doc values and stored fields.
elasticsearchmachine
added
the
Team:QL (Deprecated)
|
Hi @nik9000, I've created a changelog YAML for you. |
|
Pinging @elastic/es-ql (Team:QL) |
|
Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL) |
…ues' into esql_memory_on_values
|
|
||
| @Override | ||
| public void close() { | ||
| // TODO assert that we close the test blocks |
There was a problem hiding this comment.
I'll take a look at this one now.
| growValuesArray(newSize); | ||
| adjustBreaker(-valuesLength * elementSize()); |
There was a problem hiding this comment.
I made this change while trying to reenable one of the HeapAttack tests - specifically the too many mv fields one. It didn't get it passing, which is genuinely odd to me. Either way, I think it is more correct.
There was a problem hiding this comment.
++ this is more correct as we create a new array of newSize before copying.
| @@ -161,6 +161,7 @@ public static Block[] fromList(BlockFactory blockFactory, List<List<Object>> lis | |||
| public static Block deepCopyOf(Block block, BlockFactory blockFactory) { | |||
| try (Block.Builder builder = block.elementType().newBlockBuilder(block.getPositionCount(), blockFactory)) { | |||
| builder.copyFrom(block, 0, block.getPositionCount()); | |||
| builder.mvOrdering(block.mvOrdering()); | |||
There was a problem hiding this comment.
This was required to get some tests working. It meant I had to make the method a noop on unsupported block types but that seems ok.
| } | ||
| } | ||
| } | ||
| // for (Page p : results) { |
There was a problem hiding this comment.
Woop.s Let me get this back.
| growValuesArray(newSize); | ||
| adjustBreaker(-valuesLength * elementSize()); |
There was a problem hiding this comment.
++ this is more correct as we create a new array of newSize before copying.
It just started failing. Let's suppress it in this PR.
nik9000
added
the
auto-merge-without-approval
This adds memory tracking for values loaded from doc values and stored fields.