Block and Vector creation can start to use a block factory that performs circuit breaking

[ChrisHegarty]

This meta issue lists a number of places in the ESQL runtime where Block and Vector creation can start to use a block factory that performs breaking/accountancy.

• Evaluators @nik9000
• Hash implementations
• aggs
• grouping aggs
• EsqlQueryResponse and exchange sink ESQL: Close pages on response #99876
• operators
  • topn @nik9000 ESQL: Handle allocation errors inside topn #99931
  • project @ChrisHegarty
  • Grok @luigidellaquila ESQL: Add block factory to GROK command #100139
  • Dissect @luigidellaquila ESQL: Add block factory to DISSECT command #100140
  • OrdinalsGroupingOperator @nhat ESQL: Update OrdinalsGroupingOperator to close inbound pages #100141
  • Limit
• Deserialize with the factory ESQL: Read from the BlockFactory #100231
• Handle potential multiple closure of duplicate blocks in pages ESQL: Pages and blocks being closed #100001
• Fix affected tests ESQL: Tracker for tests impacted by the circuit breaker #100147

Working top-down there are cases where the implementation both creates and consumes blocks - do not escape the operator. In theory it should be more straightforward to start with - since their usage is localised.

relates #99173

[elasticsearchmachine]

Pinging @elastic/es-ql (Team:QL)

[elasticsearchmachine]

Pinging @elastic/elasticsearch-esql (:Query Languages/ES|QL)

[nik9000]

So I played a bit with things this afternoon. I believe step 0 is to get the EsqlQueryResponse and ExchangeResponse to properly close Blocks on the way out. this is a kind of hacky way to do that.

[nik9000]

on the way out. this is a kind of hacky way to do that.

And that doesn't quite work, because we let go of the pages before we actually render them. I'll bet there's a way around that. I wonder if we actually decRef after the render. That would work better.

[nik9000]

Looks like we're done here. There is more work to be done in 8.12 in #100528, but this is still work well done!