-
Notifications
You must be signed in to change notification settings - Fork 25k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Field API fails with _id being an alias #75836
Comments
Pinging @elastic/es-search (Team:Search) |
I looked into this a bit and we came up with a suggestion in the team that we might move away from generally prevent fetching internal fields via the |
Currently we exclude metadata fields from being looked up using the `fields` option in search. However, as issue like #75836 show, they can still be retrieved e.g. via aliases and then fetching their values causes errors. With this change, we enable retrieval of metadata fields (like `_id`) using the fields option when the field is explicitely asked for. We still continue to exclude any metadata field from matching wildcard patterns, but they should be retrievable via an exact name or if there is an alias definition with a path to a metadata field. This change adds support for the `_id` field in particular.
Currently we exclude metadata fields from being looked up using the `fields` option in search. However, as issue like elastic#75836 show, they can still be retrieved e.g. via aliases and then fetching their values causes errors. With this change, we enable retrieval of metadata fields (like `_id`) using the fields option when the field is explicitely asked for. We still continue to exclude any metadata field from matching wildcard patterns, but they should be retrievable via an exact name or if there is an alias definition with a path to a metadata field. This change adds support for the `_id` field in particular.
Currently we don't allow retrieving metadata fields through the fields option in search but throw an error on this case. In #78828 we started to enable this for "_id" if the field is explicitely requested. This PR adds _index and _version metadata fields which are internally stored as doc values to the list of fields that can be explicitely retrieved. Relates to #75836
) Currently we don't allow retrieving metadata fields through the fields option in search but throw an error on this case. In elastic#78828 we started to enable this for "_id" if the field is explicitely requested. This PR adds _index and _version metadata fields which are internally stored as doc values to the list of fields that can be explicitely retrieved. Relates to elastic#75836
Currently we exclude metadata fields from being looked up using the fields option in search. However, as issue like #75836 show, they can still be retrieved e.g. via aliases and then fetching their values causes errors. With this change, we enable retrieval of metadata fields (like `_id`, `_ignored` etc.) using the fields option when the field is explicitely requested. We still continue to exclude any metadata field from matching wildcard patterns, but they should be retrievable via an exact name or if there is an alias definition with a path to a metadata field. This change adds support for the `_id`, `_routing`, `_ignored`, `_index` and `_version` field in particular. Backport of #78828, #78981 and #79042
We implemented the above proposal for the following metadata fields: |
Certain field types like the one for `_seq_no` or `_nested_path` only exist as implementation details of other functionalities. However, we currently don't reject creating an alias field with a path to these field types. This can lead to errors like the one described in elastic#75836 where an alias to an internal field broke the fields API. This change adds a new "isInternalField" method to MappedFieldType that returns "false" by default but can be overridden by implementations that wish to announce that they are of truly internal use only. This information can then, for example, be used in alias field validation to reject creating an alias to those fields. This change adds `_seq_no`, `_field_names`, `_nested_path`, `_source` and `_feature` to the field types that are rejected when an alias is created for them. Relates to elastic#79277
Steps to reproduce:
The last query will now fail with the following exception:
Expected behavior: This request should not fail, but return all fields correctly.
The text was updated successfully, but these errors were encountered: