Add ability to retrieve document metadata via fields option #78939
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Currently we exclude metadata fields from being looked up using the `fields` option in search. However, as issue like elastic#75836 show, they can still be retrieved e.g. via aliases and then fetching their values causes errors. With this change, we enable retrieval of metadata fields (like `_id`) using the fields option when the field is explicitely asked for. We still continue to exclude any metadata field from matching wildcard patterns, but they should be retrievable via an exact name or if there is an alias definition with a path to a metadata field. This change adds support for the `_id` field in particular.
added 5 commits
October 13, 2021 17:24
…78981) Currently we don't allow retrieving metadata fields through the fields option in search but throw an error on this case. In elastic#78828 we started to enable this for `_id` if the field is explicitely requested. This PR adds `_ignored` and `_routing` metadata fields which are also internally handled as stored fields to the list of fields that can be explicitely retrieved.
cbuescher
changed the title
added 2 commits
October 14, 2021 14:18
) Currently we don't allow retrieving metadata fields through the fields option in search but throw an error on this case. In elastic#78828 we started to enable this for "_id" if the field is explicitely requested. This PR adds _index and _version metadata fields which are internally stored as doc values to the list of fields that can be explicitely retrieved. Relates to elastic#75836
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently we exclude metadata fields from being looked up using the
fieldsoption in search.However, as issue like #75836 show, they can still be retrieved e.g. via aliases and then fetching
their values causes errors.
With this change, we enable retrieval of metadata fields (like
_id,_ignoredetc.) using the fieldsoption when the field is explicitely requested. We still continue to exclude any metadata field from
matching wildcard patterns, but they should be retrievable via an exact name or if there is an alias
definition with a path to a metadata field.
This change adds support for the
_id,_routing,_ignored,_indexand_versionfield in particular.Backport of #78828, #78981 and #79042