Disable index.mapper.dynamic index setting validation. #109160
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This setting was removed via elastic#25734, because the setting no longer used since 6.0.0 However, the validation only kicked when trying to set this setting on a closed index. Applying the setting on an open index would just work. With severe consequences later on. For example when upgrading the cluster, nodes would refuse to boot, because the validation would kick in.
martijnvg
added
>bug
:StorageEngine/Mapping
|
Hi @martijnvg, I've created a changelog YAML for you. |
henningandersen
approved these changes
May 29, 2024
|
Pinging @elastic/es-storage-engine (Team:StorageEngine) |
martijnvg
added
the
auto-merge-without-approval
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Jun 3, 2024
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Jun 3, 2024
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Jun 4, 2024
Forward port from elastic#109301 to main branch only, and also checks whether the setting gets archived. Relates to elastic#109160 and elastic#96075
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jun 11, 2024
Currently when upgrading a 7.x cluster to 8.x with `index.mapper.dynamic` index setting defined the following happens: - In case of a full cluster restart upgrade, then the index setting gets archived and after the upgrade the cluster is in a green health. - In case of a rolling cluster restart upgrade, then shards of indices with the index setting fail to allocate as nodes start with 8.x version. The result is that the cluster has a red health and the index setting isn't archived. Closing and opening the index should archive the index setting and allocate the shards. The change is about ensuring the same behavior happens when upgrading a cluster from 7.x to 8.x with indices that have the `index.mapper.dynamic` index setting defined. By re-defining the `index.mapper.dynamic `index setting with `IndexSettingDeprecatedInV7AndRemovedInV8` property, the index is allowed to exist in 7.x indices, but can't be defined in new indices after the upgrade. This way we don't have to rely on index archiving and upgrading via full cluster restart or rolling restart will yield the same outcome. Based on the test in #109301. Relates to #109160 and #96075
martijnvg
added a commit
to martijnvg/elasticsearch
that referenced
this pull request
Jun 11, 2024
Currently when upgrading a 7.x cluster to 8.x with `index.mapper.dynamic` index setting defined the following happens: - In case of a full cluster restart upgrade, then the index setting gets archived and after the upgrade the cluster is in a green health. - In case of a rolling cluster restart upgrade, then shards of indices with the index setting fail to allocate as nodes start with 8.x version. The result is that the cluster has a red health and the index setting isn't archived. Closing and opening the index should archive the index setting and allocate the shards. The change is about ensuring the same behavior happens when upgrading a cluster from 7.x to 8.x with indices that have the `index.mapper.dynamic` index setting defined. By re-defining the `index.mapper.dynamic `index setting with `IndexSettingDeprecatedInV7AndRemovedInV8` property, the index is allowed to exist in 7.x indices, but can't be defined in new indices after the upgrade. This way we don't have to rely on index archiving and upgrading via full cluster restart or rolling restart will yield the same outcome. Based on the test in elastic#109301. Relates to elastic#109160 and elastic#96075
elasticsearchmachine
pushed a commit
that referenced
this pull request
Jun 11, 2024
Currently when upgrading a 7.x cluster to 8.x with `index.mapper.dynamic` index setting defined the following happens: - In case of a full cluster restart upgrade, then the index setting gets archived and after the upgrade the cluster is in a green health. - In case of a rolling cluster restart upgrade, then shards of indices with the index setting fail to allocate as nodes start with 8.x version. The result is that the cluster has a red health and the index setting isn't archived. Closing and opening the index should archive the index setting and allocate the shards. The change is about ensuring the same behavior happens when upgrading a cluster from 7.x to 8.x with indices that have the `index.mapper.dynamic` index setting defined. By re-defining the `index.mapper.dynamic `index setting with `IndexSettingDeprecatedInV7AndRemovedInV8` property, the index is allowed to exist in 7.x indices, but can't be defined in new indices after the upgrade. This way we don't have to rely on index archiving and upgrading via full cluster restart or rolling restart will yield the same outcome. Based on the test in #109301. Relates to #109160 and #96075
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This setting was removed via #25734, because the setting no longer used since 6.0.0
However, the validation only kicked when trying to set this setting on a closed index. Applying the setting on an open index would just work. With severe consequences later on. For example when upgrading the cluster, nodes would refuse to boot, because the validation would kick in.
Relates to #96075