Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Parse the contents of dynamic objects for [subobjects:false] #117762

Merged
merged 15 commits into from
Dec 3, 2024

Conversation

kkrik-es
Copy link
Contributor

@kkrik-es kkrik-es commented Nov 29, 2024

DocumentParser skips parsing dynamic objects when [dynamic:false] is set in a parent object. When [subobjects:false] is also set, dynamic objects should be flattened instead.

Fixes #117544

@kkrik-es kkrik-es added >bug auto-backport Automatically create backport pull requests when merged Team:StorageEngine :StorageEngine/Mapping The storage related side of mappings v8.17.1 v8.18.0 labels Nov 29, 2024
@kkrik-es kkrik-es self-assigned this Nov 29, 2024
@elasticsearchmachine
Copy link
Collaborator

Hi @kkrik-es, I've created a changelog YAML for you.

@kkrik-es kkrik-es marked this pull request as ready for review November 29, 2024 14:28
@kkrik-es kkrik-es requested review from lkts and martijnvg November 29, 2024 14:28
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-storage-engine (Team:StorageEngine)

Copy link
Member

@martijnvg martijnvg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👍 Thanks for fixing this Kostas!

refresh: true
body:
- '{ "index": { } }'
- '{ "my": { "keyword.field": "abc" } }'
Copy link
Contributor

@lkts lkts Dec 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we have a test that introducing a dynamic field throws in this case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

body:
fields: [ "*" ]

- match: { hits.hits.0.fields: { my.keyword.field: [ abc ] } }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You'll need to sort results for this test to work with multiple shards.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep forgetting that.. Done.


doc = mapper.parse(source("""
{ "metrics": { "service": { "test.other.dots": "foo" } } }"""));
assertNull(doc.rootDoc().getField("metrics.service.test.with.dots"));
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
assertNull(doc.rootDoc().getField("metrics.service.test.with.dots"));
assertNull(doc.rootDoc().getField("metrics.service.test.other.dots"));

here and below?

# Conflicts:
#	server/src/main/java/org/elasticsearch/index/mapper/MapperFeatures.java
@kkrik-es kkrik-es enabled auto-merge (squash) December 3, 2024 15:59
@kkrik-es kkrik-es merged commit f2addbc into elastic:main Dec 3, 2024
16 checks passed
@elasticsearchmachine
Copy link
Collaborator

💔 Backport failed

Status Branch Result
8.17 Commit could not be cherrypicked due to conflicts
8.x

You can use sqren/backport to manually backport by running backport --upstream elastic/elasticsearch --pr 117762

kkrik-es added a commit to kkrik-es/elasticsearch that referenced this pull request Dec 3, 2024
…#117762)

* Parse the contents of dynamic objects for [subobjects:false]

* Update docs/changelog/117762.yaml

* add tests

* tests

* test dynamic field

* test dynamic field

* fix tests
@kkrik-es
Copy link
Contributor Author

kkrik-es commented Dec 3, 2024

💚 All backports created successfully

Status Branch Result
8.17

Questions ?

Please refer to the Backport tool documentation

kkrik-es added a commit to kkrik-es/elasticsearch that referenced this pull request Dec 3, 2024
…#117762)

* Parse the contents of dynamic objects for [subobjects:false]

* Update docs/changelog/117762.yaml

* add tests

* tests

* test dynamic field

* test dynamic field

* fix tests

(cherry picked from commit f2addbc)

# Conflicts:
#	server/src/main/java/org/elasticsearch/index/mapper/MapperFeatures.java
elasticsearchmachine pushed a commit that referenced this pull request Dec 3, 2024
#117919)

* Parse the contents of dynamic objects for [subobjects:false]

* Update docs/changelog/117762.yaml

* add tests

* tests

* test dynamic field

* test dynamic field

* fix tests
elasticsearchmachine pushed a commit that referenced this pull request Dec 3, 2024
#117921)

* Parse the contents of dynamic objects for [subobjects:false]

* Update docs/changelog/117762.yaml

* add tests

* tests

* test dynamic field

* test dynamic field

* fix tests

(cherry picked from commit f2addbc)

# Conflicts:
#	server/src/main/java/org/elasticsearch/index/mapper/MapperFeatures.java
@kkrik-es kkrik-es deleted the fix/117143 branch January 7, 2025 12:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
auto-backport Automatically create backport pull requests when merged backport pending >bug :StorageEngine/Mapping The storage related side of mappings Team:StorageEngine v8.17.1 v8.18.0 v9.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fields not searchable with subobjects: false and dynamic: false when sent as nested objects
4 participants