Mappings: Allow to force dots in field names #19937
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
With the changes in 5.0 to allow dots in field names for documents as well as mappings, we now have the possibility to allow a path forward for users which must have dots in fields on 2.x. This change adds an option to override the dot checks, `mapper.allow_dots_in_name` which may be set to `true` to omit the dots in field names error check. The resulting mappings will contain dots in the field names, and on upgrade to 5.x, those will be automatically converted to object mappings on startup. Note that this works because the underlying lucene indexes uses the full path. If a user users this to produce corrupt mappings, for example, two fields `foo` and `foo.bar` which are both text, the upgrade to 5.x will fail (because `foo` cannot be an object as well as a string field).
rjernst
added
:Search Foundations/Mapping
+1 |
|
I created #19956 so we don't forget to add the change to static bwc indexes. |
Closed
|
@rjernst this setting isn't documented anywhere - could you add to the 2.4 docs please? |
|
+1 Yeah! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
With the changes in 5.0 to allow dots in field names for documents as
well as mappings, we now have the possibility to allow a path forward
for users which must have dots in fields on 2.x. This change adds an
option to override the dot checks,
mapper.allow_dots_in_namewhich maybe set to
trueto omit the dots in field names error check. Theresulting mappings will contain dots in the field names, and on upgrade
to 5.x, those will be automatically converted to object mappings on
startup.
Note that this works because the underlying lucene indexes uses the full
path. If a luser uses this to produce corrupt mappings, for example, two
fields
fooandfoo.barwhich are both text, the upgrade to 5.x willfail (because
foocannot be an object as well as a string field).One open question is how to test. I've checked that this upgrade path works manually. I don't think a unit test will work (would depend on a sysprop being set which we should not generally have set in tests). I think the best way would be to add a special case in the bwc generation script, so that 2.4 indexes will have a field name with dot, and then the normal static bwc index tests will check that the upgrade works.