Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validate index settings after applying templates #44612

Conversation

jasontedor
Copy link
Member

Today we validate index settings before applying templates. While this does not cause a problem in general (since if there is a problem we will end up failing on index creation anyway), there is one case where it does cause a problem: checking whether or not the index creation will bring the cluster over the shard limit per node. This is because the only time this limit is checked is when index settings are validated. If we are doing this before applying templates, we would not have the correct number of shards used to check the limit. This commit addresses this by first applying templtes before validating index settings. This addresses the case of checking the shard limit per node, but also will help prevent additional bugs like this in the future. Note that we can not add in any private settings until after the validation is done (otherwise they will fail validation).

Closes #44567
Relates #34021

Today we validate index settings before applying templates. While this
does not cause a problem in general (since if there is a problem we will
end up failing on index creation anyway), there is one case where it
does cause a problem: checking whether or not the index creation will
bring the cluster over the shard limit per node. This is because the
only time this limit is checked is when index settings are validated. If
we are doing this before applying templates, we would not have the
correct number of shards used to check the limit. This commit addresses
this by first applying templtes before validating index settings. This
addresses the case of checking the shard limit per node, but also will
help prevent additional bugs like this in the future. Note that we can
not add in any private settings until after the validation is done
(otherwise they will fail validation).
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-features

@jasontedor
Copy link
Member Author

This approach is flawed because of resizing indices, where we copy settings including private settings, and this is what we would want to validate in an approach like this. Since settings validation is fairly simplistic at this point, I am going to take the simpler approach of merely moving the check shard limit out of validation and after templates have been applied. I will close this PR and open another.

@jasontedor jasontedor closed this Jul 19, 2019
@jasontedor jasontedor deleted the validate-index-settings-after-applying-index-templates branch July 19, 2019 10:31
@jasontedor
Copy link
Member Author

Superseded by #44619

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

cluster.max_shards_per_node derives shard count from default not template settings
3 participants