[Star tree] star tree off heap changes #14815
Labels
enhancement
Enhancement or improvement to existing feature or request
Indexing:Performance
Indexing
Indexing, Bulk Indexing and anything related to indexing
Other
v2.17.0
Is your feature request related to a problem? Please describe
Sort and aggregation of segment documents when building star tree takes up lots of resources if they are stored in in-memory structures like Lists. So this issue tracks the off heap implementation where we store and retrieve the star tree documents in off heap method using temporary files.
Describe the solution you'd like
We will store the offsets in array and swap the offsets when we go through the files. We offload the entire sort and aggregation of segment documents and construction of star tree documents to off-heap.
Related component
Other
Describe alternatives you've considered
No response
Additional context
No response
The text was updated successfully, but these errors were encountered: