Track and use dynamic dimensions for routing #102977
Conversation
Dynamic templates allow defining fields with `time_series_dimension` annotations. When these fields are referenced in dynamic template specs at indexing time, they can be used for routing instead of the routing path; the latter can be empty in case all dimensions are dynamically specified. Related to elastic#98384
kkrik-es
added
>enhancement
Team:Analytics
|
Hi @kkrik-es, I've created a changelog YAML for you. |
# Conflicts: # server/src/main/java/org/elasticsearch/search/DefaultSearchContext.java
…ting' into prototype/dynamic_dimensions_setting
| @@ -113,11 +113,23 @@ public Settings getAdditionalIndexSettings( | |||
| builder.put(IndexSettings.TIME_SERIES_START_TIME.getKey(), FORMATTER.format(start)); | |||
| builder.put(IndexSettings.TIME_SERIES_END_TIME.getKey(), FORMATTER.format(end)); | |||
|
|
|||
| if (allSettings.hasValue(IndexMetadata.INDEX_ROUTING_PATH.getKey()) == false | |||
| boolean useDynamicDimensions = allSettings.getAsBoolean(IndexMetadata.TIME_SERIES_DYNAMIC_TEMPLATES.getKey(), false); | |||
There was a problem hiding this comment.
Maybe add a else if() just for dynamic dimensions?
There was a problem hiding this comment.
I think we want to retrieve dynamic dimensions even when there's a routing path, to support migrating existing indexes to the new functionality.
| */ | ||
| private List<String> findRoutingPaths(String indexName, Settings allSettings, List<CompressedXContent> combinedTemplateMappings) { | ||
| private void findRoutingPathsAndDimensions( |
There was a problem hiding this comment.
Maybe keep the existing findRoutingPaths method unchanged and create a findDimensions(...) method?
This keeps the two modes (index routing path & dynamic dimensions) separate.
There was a problem hiding this comment.
I've splitted this in three functions to reuse the MappingService, ptal.
| @@ -233,32 +269,57 @@ public void collectSearchShards(String routing, IntConsumer consumer) { | |||
| } | |||
| } | |||
|
|
|||
| public static class ExtractFromSource extends IndexRouting { | |||
| public static class RoutingPathMatching extends IndexRouting { | |||
There was a problem hiding this comment.
Maybe keep the name the same. In both ways we keep extracting routing information from source and this keeps the diff of this change smaller.
|
@elasticsearchmachine run elasticsearch-ci/part-1 |
|
@elasticsearchmachine run elasticsearch-ci/8.11.2 / bwc-snapshots |
…ting' into prototype/dynamic_dimensions_setting
…ting' into prototype/dynamic_dimensions_setting
| @@ -8,4 +8,4 @@ | |||
| <option name="AUTO_RESTART" value="true" /> | |||
| <method v="2" /> | |||
| </configuration> | |||
| </component> | |||
There was a problem hiding this comment.
Unintended, can't revert :)
Dynamic templates allow defining fields with
time_series_dimensionannotations. When these fields are referenced in dynamic template specs at indexing time, they can be used for routing instead of the routing path; the latter can be empty in case all dimensions are dynamically specified.Related to #98384