Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Always enforce a default tier preference (ENFORCE_DEFAULT_TIER_PREFERENCE must always be true) #79723

Closed

Conversation

joegallo
Copy link
Contributor

Part of #76147, follow up to #79210 and #79275

#79210 added this new setting, and #79275 made it so the value defaults to true. This PR goes one further, and makes it so that the setting is only ever allowed to be true. Mostly it's adjusting the tests to account for that, there were some tests where we were still explicitly setting the value to false in order to test 'what if' scenarios.

It's just always true, these tests need to learn to live with that.
The tests that checked what happens when we don't enforce a tier
preference don't need to exist anymore. Likewise, since we always
enforce we can rename the tests that check what happens when we do.
@joegallo joegallo added >breaking :Data Management/ILM+SLM Index and Snapshot lifecycle management v8.0.0 labels Oct 25, 2021
@elasticmachine elasticmachine added the Team:Data Management Meta label for data/management team label Oct 25, 2021
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-data-management (Team:Data Management)

We'll be removing it one day, and it's only allowed to be true anyway,
so there's no point in having it.
@joegallo joegallo mentioned this pull request Oct 25, 2021
7 tasks
@joegallo
Copy link
Contributor Author

joegallo commented Oct 25, 2021

If we end up going with this version of things, then a follow up PR could short cut a bunch of if/elses, since only the true side of those if/elses would ever get executed.

Alternatively, we could skip this PR and only go with that other PR -- this setting hangs around with a true or false value, and we don't care what it is (or even that it exists), because the behavior doesn't check the value of the setting anymore at all. I talked this approach over with @dakrone and we liked it enough that I'm going to write that up now. I'll leave this PR up though just in case we decide to go this way instead of that way.

edit: that'd be #79751

Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need a way to create indices with no tier preference in tests, since otherwise we essentially disable testing that the tier migration actually works.

new Setting.Validator<>() {
@Override
public void validate(Boolean value) {
if (value == Boolean.FALSE) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I would prefer:

Suggested change
if (value == Boolean.FALSE) {
if (value != Boolean.TRUE) {

then there is no doubt about null being illegal too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

b5e6a5d, sure thing.

Comment on lines -155 to -158
// if we inject a default tier preference, then this test wouldn't be valid anymore,
// so let's turn that off
enforceDefaultTierPreference(false);

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this invalidates the purpose of the test. And on 8.0 we may still see indices with no tier preference (since they could have been created in 7.x). So ideally we would allow the test to create data with no tier preference somehow.

I would be Ok to defer that test to a follow-up though.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

72dee11 should handle these -- I've gone back to the original intent of the tests by manually setting the tier preference back to null.


// we can't have the pre-migration indices getting tier preferences auto-assigned,
// if we did, then we wouldn't really be testing the migration *to* data tiers anymore :D
enforceDefaultTierPreference(false);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another case where we would ideally add such indices with no tier preference (as if they were created in 7.x) and see the migration cure them.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍, see previous.

Comment on lines 188 to 184
assertThat(indexSettings.get(DataTier.TIER_PREFERENCE), is("data_warm,data_hot"));
assertThat(indexSettings.get(DataTier.TIER_PREFERENCE), is(DataTier.DATA_CONTENT));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is an essential part of the migration that is no longer tested in 8.0? I think we should ensure that we can add indices with no tier preference for test purpose and check that the migration fills them in.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍, see previous.

Copy link
Contributor

@henningandersen henningandersen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@joegallo
Copy link
Contributor Author

Closing this one in favor of #79751

@joegallo joegallo closed this Oct 26, 2021
@joegallo joegallo deleted the always-enforce-default-tier-preference branch October 26, 2021 14:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>breaking :Data Management/ILM+SLM Index and Snapshot lifecycle management Team:Data Management Meta label for data/management team v8.0.0-beta1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants