Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move internal index templates to composable templates #61457

Merged
merged 12 commits into from
Aug 27, 2020
Merged

Move internal index templates to composable templates #61457

merged 12 commits into from
Aug 27, 2020

Conversation

probakowski
Copy link
Contributor

@probakowski probakowski commented Aug 24, 2020 •
edited
Loading

This change moves watcher, ILM history and SLM history templates to composable templates.
Versions are updated to reflect the switch. Only change to the templates themselves is added _meta to mark them as managed

@probakowski
Copy link
Contributor Author

@elasticmachine run elasticsearch-ci/bwc
run elasticsearch-ci/default-distro

@probakowski
Copy link
Contributor Author

@elasticmachine update branch

@probakowski probakowski marked this pull request as ready for review August 26, 2020 10:29
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/ILM+SLM)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Aug 26, 2020
pugnascotia
pugnascotia approved these changes Aug 26, 2020
View reviewed changes
Copy link
Contributor

@pugnascotia pugnascotia left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM but you shouldn't take my word for it 😅

@dakrone?

dakrone
dakrone requested changes Aug 26, 2020
View reviewed changes
Copy link
Member

@dakrone dakrone left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @probakowski, I left some comments about missing _meta maps in some of the templates (we need these to indicate in the UI that a template is managed by ES automatically).

I was thinking though, that we should increment the version to indicate the change, that way a user is not left with a version: 2 legacy template and version: 2 composable index template; they can immediately tell from the version: 3 signifier that the composable index template version is "newer" (since the old one will not be removed) even if they are not familiar with the difference between the types of templates.

x-pack/plugin/core/src/main/resources/triggered-watches.json Show resolved Hide resolved
x-pack/plugin/core/src/main/resources/watch-history-no-ilm.json Show resolved Hide resolved
x-pack/plugin/core/src/main/resources/watch-history.json Show resolved Hide resolved
x-pack/plugin/core/src/main/resources/watches.json Show resolved Hide resolved
@probakowski
Copy link
Contributor Author

@dakrone that makes sense, I've increased version in all templates

dakrone
dakrone approved these changes Aug 27, 2020
View reviewed changes
Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but there is one more missing version increment, in ILMHistoryTemplateRegistry we need to increment the version for the ilm history index template

@probakowski
Copy link
Contributor Author

Good catch, I've updated it as well, thanks!

@probakowski @dakrone
Update x-pack/plugin/ilm/src/main/java/org/elasticsearch/xpack/ilm/hi…
dacba2e 
…story/ILMHistoryTemplateRegistry.java

Co-authored-by: Lee Hinman <[email protected]>
@probakowski probakowski merged commit 3b79d1d into elastic:master Aug 27, 2020
@probakowski probakowski deleted the internal-composable-templates1 branch August 27, 2020 20:38
@probakowski
Copy link
Contributor Author

pinging @elastic/es-ui, FYI this change updates version field in WatcherIndexTemplateRegistryField so probably something around watcher ui will need to be updated as well.
Also versions in SLM and ILM are updated, not sure if this is relevant for UI

@probakowski probakowski mentioned this pull request Aug 27, 2020
Merged
probakowski added a commit to probakowski/elasticsearch that referenced this pull request Aug 27, 2020
@probakowski
Move internal index templates to composable templates (elastic#61457)
b7d6657 
This change moves watcher, ILM history and SLM history templates to composable templates.
Versions are updated to reflect the switch. Only change to the templates themselves is added `_meta` to mark them as managed
@cjcenizal
Copy link
Contributor

Thank you for the ping @probakowski! To help me understand what needs to change in the UI, could you please describe how the watcher, SLM, and ILM APIs have changed?

javanna added a commit that referenced this pull request Sep 1, 2020
@javanna
Adapt to upstream changes
acfaccb 
Possilby after #61457, the priority we use for the index template that converts core yaml tests to use runtime fields is already taken, which causes an error as there ma be multiple matching templates with same priority.
@albertzaharovits albertzaharovits mentioned this pull request Sep 2, 2020
Closed
@probakowski probakowski mentioned this pull request Sep 2, 2020
Merged
probakowski added a commit that referenced this pull request Sep 3, 2020
@probakowski
Fix ILM history index settings (#61880)
969fa53 
This change fixed settings accidentally changed in #61457, this fixes ability of ILM history index to rollover.
It also adds test to make sure we can't hit this kind of bug in the future.

Closes #61863
@probakowski
Copy link
Contributor Author

@cjcenizal I don't know if you consider this API but versions templates and some internal indices got upgraded:

  • ILM history index was ilm-history-2, now it will be ilm-history-3
  • SLM history index was .slm-history-2, now it will be .slm-history-3
  • Watcher history index was .watcher-history-11, now it will be .watcher-history-12

According to note in WatcherIndexTemplateRegistryField UI should be notified when these version changes as UI needs to be updated

probakowski added a commit to probakowski/elasticsearch that referenced this pull request Sep 3, 2020
@probakowski
Fix ILM history index settings (elastic#61880)
995f6b0 
This change fixed settings accidentally changed in elastic#61457, this fixes ability of ILM history index to rollover.
It also adds test to make sure we can't hit this kind of bug in the future.

Closes elastic#61863
probakowski added a commit that referenced this pull request Sep 8, 2020
@probakowski
[7.x] Move internal index templates to composable templates (#61457) (#…
bb357f6 
…61661)

This change moves watcher, ILM history and SLM history templates to composable templates.
Versions are updated to reflect the switch. Only change to the templates themselves is added `_meta` to mark them as managed
@droberts195 droberts195 mentioned this pull request Nov 24, 2020
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dakrone dakrone dakrone approved these changes

@pugnascotia pugnascotia pugnascotia approved these changes

Assignees
No one assigned
Labels
:Data Management/ILM+SLM Index and Snapshot lifecycle management >enhancement Team:Data Management Meta label for data/management team v7.10.0 v8.0.0-alpha1
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
7 participants
@probakowski @elasticmachine @cjcenizal @dakrone @pugnascotia @jakelandis @andreidan