Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow shrink in the hot phase for ILM policies #64008

Merged
merged 13 commits into from
Oct 29, 2020

Conversation

joegallo
Copy link
Contributor

@joegallo joegallo commented Oct 21, 2020

Resolves #56377, very much along the same lines as #52073

This PR changes the shrink action to also be allowed in the hot phase after a rollover. As with #52073, a shrink in the hot phase MUST be accompanied by a rollover, and policy validation has been updated to check for this.

Reviewing commit-by-commit is best, there's a few relatively self-contained cleanup commits.

and lets us drop the static initializer
to allow the possibility that there could be more actions in the hot
phase that require rollover, rather than only just forcemerge.
as long as there's an accompanying rollover.
@joegallo joegallo added the WIP label Oct 21, 2020
Inline forceMergeActionWithCodec in place of its only caller.
Move forcemerge after rollover, since that's when it'll actually
execute. Add shrink to the hot phase. Migrate was missing in the warm
and cold phases, fix that.
@joegallo joegallo added the :Data Management/ILM+SLM Index and Snapshot lifecycle management label Oct 29, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features (:Core/Features/ILM+SLM)

@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Oct 29, 2020
@joegallo joegallo added v7.11.0 v8.0.0 >enhancement and removed Team:Data Management Meta label for data/management team WIP labels Oct 29, 2020
@joegallo joegallo requested a review from jakelandis October 29, 2020 14:31
@joegallo
Copy link
Contributor Author

@elasticmachine update branch

@joegallo joegallo added the Team:Data Management Meta label for data/management team label Oct 29, 2020
Copy link
Member

@dakrone dakrone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, I left one comment about a change we should undo though, other than that, looks good!

@@ -50,14 +50,13 @@
static final Set<String> VALID_WARM_ACTIONS = Sets.newHashSet(ORDERED_VALID_WARM_ACTIONS);
static final Set<String> VALID_COLD_ACTIONS = Sets.newHashSet(ORDERED_VALID_COLD_ACTIONS);
static final Set<String> VALID_DELETE_ACTIONS = Sets.newHashSet(ORDERED_VALID_DELETE_ACTIONS);
private static final Map<String, Set<String>> ALLOWED_ACTIONS = new HashMap<>();
private static final Map<String, Set<String>> ALLOWED_ACTIONS = Map.of(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not compile on backport (which is fine, just wanted to let you know since you asked about this previously)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, I've pregamed that and I'm ready to do the backport dance.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do have these utility classes in the 7.x branch that provide these functions in a Java8-compatible way so that backports are easier. They're also packaged as a multi-release JAR so that in JVMs that are 9 or later, they use the Java libraries instead, so there's no downside to using them when running with later JVMs.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danhermann the utility classes are unfortunately not that helpful, since to use they the Map.of needs to be fully qualified, or java.util.Map<String, Set<String>> has to be specified to fully qualify the jdk classes, which I think is more awkward than just changing on backport.

This reverts commit fe5c390.
@joegallo joegallo merged commit 9986cb8 into elastic:master Oct 29, 2020
@joegallo joegallo deleted the add-shrink-to-hot-phase branch October 29, 2020 18:21
Copy link
Contributor

@andreidan andreidan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for working on this @joegallo

LGTM

@joegallo joegallo added the Team:Deployment Management Meta label for Management Experience - Deployment Management team label Nov 6, 2020
@joegallo
Copy link
Contributor Author

joegallo commented Nov 6, 2020

@elastic/es-ui don't miss #64381 which is very similar to this. Also see some prior art over on elastic/kibana#77193.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Data Management/ILM+SLM Index and Snapshot lifecycle management >enhancement Team:Data Management Meta label for data/management team Team:Deployment Management Meta label for Management Experience - Deployment Management team v7.11.0 v8.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add shrink action to the hot phase
6 participants