[META] Star Tree Index issue list

[bharath-techie]

Please describe the end goal of this project

Implement star tree index to precompute aggregations which helps to enforce upper bound on the aggregation queries ensuring predictable latency and resource usage. It is also storage space efficient and configurable.

Supporting References

• [RFC] Pre Compute Aggregations with Star Tree index #12498

Issues

Milestone 1 - Support for numeric dimensions and basic metrics

Config / codec changes

Covers user/client experience to create star tree indices via specifying star tree config and custom codec changes to support star tree index in experimental mode.

• [Star tree] Star tree index configuration via mappings #14386 - Star tree index config and settings
• [Star tree] Star tree codec changes #14387 - Star tree index codec changes

Flush changes

• [Feature Request] Star Tree Tree Builder [On Heap] #14116 - Basic changes for interfaces to build star tree during flush
• [Feature Request] Star Tree [OnHeap] Min and Max Value Aggregators #14624 - Support min / max aggregations as part of star tree

File format changes

• [Feature Request] Star Tree File Formats #14837
• [Feature Request] Lucene Abstractions for Star Tree #15279
• [Feature Request] Star Tree Meta and Data Writers #15296

Merge / Off heap changes

• [Star tree] Star tree merge changes #14626 - Star tree merge changes - Segment merge changes as part of indexing flow
• [Star tree] star tree off heap changes #14815 - Off heap changes - Build star tree index by performing sort and aggregations off heap as on-heap build is quite heavy

Search changes

• [META] [Star Tree] [Search] Supported Queries to resolve via star tree #15257

Other 2.17 changes

• [Star tree] Rounding off date field in Star tree #15265
• [Star tree] Handle 'unsigned long' as part of star tree #15231
• [Feature Request] Support Scaled Floats with Star Tree #15443

Testing / benchmarks / Bugs

• [Star tree] Star tree test cases / bugs tracker #14813
• [BUG] Transforming Metric Value to Aggregated Value Type during Star Tree Merges #15275
• [BUG] [Star tree] Nodes with -1 as dimension value gets overriden with star node #15097
• [BUG] Construct StarTreeMetadata prior to Writes #15467
• [Feature Request] Support for _doc_count field in star tree to represent _doc_count as part of aggregations response #15288
• [Star tree] Star tree indexing performance optimizations #16218
• [Feature Request] Star tree validations #15491

---

Milestone 2 - Support for keyword fields to support terms / multi-terms aggregations

• [Feature Request] Support for keyword fields as part of star tree index mapping #16232
• [Feature Request] Support for IP fields as part of star tree index mapping #16642

Milestone 3 - Guardrails , handling cases such as high cardinality etc.

Milestone 4 and later is yet to be scoped - But we will have other cases such as deleted docs handling, metrics extension, query/indexing optimizations etc.

Feature requests / Backlog

• [Star tree] Support for timezone and offset for date dimension #15266
• [Feature Request] Generic Numeric Type Converters for Numeric Field Data #14657
• [Star tree] Handle 'unsigned long' as part of star tree #15231

Related component

Other