Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Templates match indices with date math expressions #71433

Merged
merged 3 commits into from
Apr 8, 2021

Conversation

danhermann
Copy link
Contributor

@danhermann danhermann commented Apr 7, 2021

The root cause of the bug identified in #69727 (reported independently by several users) is that the findV?Templates methods in MetadataIndexTemplateService do not resolve index names with date math expressions before attempting to match them. The reproduction scenario in #69727 demonstrates one flavor of this problem. Before 7.9, an ingest pipeline with a date_index_name processor would (accidentally) work around that problem, but the simple statement below would fail to execute the final pipeline in any ES version (assuming a template that defines a final pipeline for an index pattern of test-*):

# url-encoded form of <test-{now/d}>
POST /%3Ctest-%7Bnow%2Fd%7D%3E/_doc 
{
  "field": "value"
}

A change to the way ingest pipelines work in 7.9 caused the date_index_name scenario to fail as well. This change resolves both problems.

Fixes #69727.

@danhermann danhermann added >bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP v8.0.0 v7.13.0 v7.12.1 labels Apr 7, 2021
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Apr 7, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (Team:Core/Features)

@danhermann
Copy link
Contributor Author

@elasticmachine update branch

Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@danhermann danhermann merged commit 072607a into elastic:master Apr 8, 2021
@danhermann danhermann deleted the 69727_fix_final_pipeline branch April 8, 2021 12:15
@danhermann
Copy link
Contributor Author

Thanks, @dakrone!

@gwbrown
Copy link
Contributor

gwbrown commented Apr 14, 2021

@danhermann Looks like we're good to remove backport pending from this PR, yes?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Data Management/Ingest Node Execution or management of Ingest Pipelines including GeoIP Team:Data Management Meta label for data/management team v7.12.1 v7.13.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Final Pipeline is not applied when routing using a pipeline
5 participants