Fix shard splitting for nested
#89351
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
I broke shard splitting when `_routing` is required and you use `nested`
docs. The mapping would look like this:
```
"mappings": {
"_routing": {
"required": true
},
"properties": {
"n": { "type": "nested" }
}
}
```
If you attempt to split an index with a mapping like this it'll blow up
with an exception like this:
```
Caused by: [idx] org.elasticsearch.action.RoutingMissingException: routing is required for [idx]/[0]
at org.elasticsearch.cluster.routing.IndexRouting$IdAndRoutingOnly.checkRoutingRequired(IndexRouting.java:181)
at org.elasticsearch.cluster.routing.IndexRouting$IdAndRoutingOnly.getShard(IndexRouting.java:175)
```
This fixes the problem by entirely avoiding the branch of code. That
branch was trying to find any top level documents that don't have a
`_routing`. But we *know* that there aren't any top level documents
without a routing in this case - the routing is "required". ES wouldn't
have let you index any top level documents without the routing.
This also adds a small pile of REST layer tests for shard splitting that
hit various branches in this area. For extra paranoia.
Closes elastic#88109
nik9000
added
>bug
:Data Management/Indices APIs
elasticsearchmachine
added
the
Team:Data Management
|
Pinging @elastic/es-data-management (Team:Data Management) |
|
Hi @nik9000, I've created a changelog YAML for you. |
romseygeek
approved these changes
Aug 16, 2022
Thanks. It was indeed tricky! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I broke shard splitting when
_routingis required and you usenesteddocs. The mapping would look like this:
If you attempt to split an index with a mapping like this it'll blow up
with an exception like this:
This fixes the problem by entirely avoiding the branch of code. That
branch was trying to find any top level documents that don't have a
_routing. But we know that there aren't any top level documentswithout a routing in this case - the routing is "required". ES wouldn't
have let you index any top level documents without the routing.
This also adds a small pile of REST layer tests for shard splitting that
hit various branches in this area. For extra paranoia.
Closes #88109