You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In #85162@martijnvg caused us to skip indices who's @timestamp doesn't overlap the query. It's implemented by reading the index.time_series.start_time and index.time_series.end_time. I wonder if we can set these settings on any index. Like, could we make an ILM action that runs on read-only indices and checks the min and max time in all shards and then sets the settings to those values? Such an action would cause us to skip querying them fairly cheaply.
The text was updated successfully, but these errors were encountered:
Today we already compute IndexLongFieldRange for indices that are readonly (frozen and searchable snapshot backed indices) and store this range in the cluster state. So I think this isn't needed?
Also IndexLongFieldRange is more accurate since it is computed based on min and max value of @timestamp field across all shards of an index. Whereas the index.time_series.start_time and index.time_series.end_time we set can be seen as upper boundaries. Indexing documents with @timestamp outside this range isn't possible (and will fail), but because of various reasons the actual range of the @timestamp may be a bit smaller than what the the range is configured in these index settings. In any way what is defined the index.time_series.start_time and index.time_series.end_time is good enough to decide whether to not query the shards of backing index of a tsdb data stream that fall outside of this range.
Description
In #85162 @martijnvg caused us to skip indices who's
@timestamp
doesn't overlap the query. It's implemented by reading theindex.time_series.start_time
andindex.time_series.end_time
. I wonder if we can set these settings on any index. Like, could we make an ILM action that runs on read-only indices and checks the min and max time in all shards and then sets the settings to those values? Such an action would cause us to skip querying them fairly cheaply.The text was updated successfully, but these errors were encountered: