Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fields API won't extract field that begins with the name of an object #65499

Closed
nik9000 opened this issue Nov 25, 2020 · 5 comments · Fixed by #65539
Closed

Fields API won't extract field that begins with the name of an object #65499

nik9000 opened this issue Nov 25, 2020 · 5 comments · Fixed by #65539
Labels
>bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team

Comments

@nik9000
Copy link
Member

nik9000 commented Nov 25, 2020

Check it out:

curl -XDELETE -HContent-Type:application/json -uelastic:password localhost:9200/test

curl -XPOST -HContent-Type:application/json -uelastic:password localhost:9200/test/_doc?refresh -d'{
  "foo": {
    "cat": "meow"
  },
  "foo.bar": "baz"
}'

curl -XPOST -HContent-Type:application/json -uelastic:password localhost:9200/test/_search?pretty -d'{
  "fields": ["foo.bar"]
}'

Makes this:

{
  "took" : 1,
  "timed_out" : false,
  "_shards" : {
    "total" : 1,
    "successful" : 1,
    "skipped" : 0,
    "failed" : 0
  },
  "hits" : {
    "total" : {
      "value" : 1,
      "relation" : "eq"
    },
    "max_score" : 1.0,
    "hits" : [
      {
        "_index" : "test",
        "_id" : "kyfg_3UBbogJq0WOeKye",
        "_score" : 1.0,
        "_source" : {
          "foo" : {
            "cat" : "meow"
          },
          "foo.bar" : "baz"
        }
      }
    ]
  }
}

Notice no fields in the response. This seems to be because we found the foo object and didn't find bar inside it. I'm not sure though. It seems like we should find foo.bar.

These kinds of documents are confusing in general and folks probably shouldn't be sending them to us. I think generally if we get them it is because someone made a mistake in their application. But we should still handle them.

@nik9000 nik9000 added >bug :Search/Search Search-related issues that do not fall into other categories needs:triage Requires assignment of a team area label labels Nov 25, 2020
@elasticmachine elasticmachine added the Team:Search Meta label for search team label Nov 25, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search (Team:Search)

@nik9000
Copy link
Member Author

nik9000 commented Nov 25, 2020

@romseygeek do you think this might be something you can have a look at? I think you are started to get pretty into how we extract values from xcontent.

@nik9000 nik9000 removed the needs:triage Requires assignment of a team area label label Nov 25, 2020
@romseygeek
Copy link
Contributor

romseygeek commented Nov 25, 2020

Looks like we don't correctly handle object paths in field fetcher (this fails in the same way if you remove the {"foo":{"cat","meow"}} section as well).

@nik9000
Copy link
Member Author

nik9000 commented Nov 25, 2020

Looks like we don't correctly handle object paths in field fetcher (this fails in the same way if you remove the {"foo":{"cat","meow"}} section as well).

Oh! I thought I saw it working locally. I guess I missed it.

@romseygeek
Copy link
Contributor

Ignore my last comment, it does indeed only fail with the extra section. It's a bug in XContentMapValues.extractRawValues() - I'll work on a fix.

romseygeek added a commit that referenced this issue Dec 1, 2020
When we have an object that looks like this:

```
{
  "foo" : {
    "cat": "meow"
  },
  "foo.bar" : "baz"
```

where the inner objects of foo are defined both as json objects and via
dot notation, then XContentMapValues.extractValue() will only descend
through the json object and will not collect values from the dot notated paths.
In these cases, the foo.bar values will not be returned in the fields
section of a search response.

This commit adds the ability to check both paths, ensuring that all relevant
values get returned as part of fields.

Fixes #65499
romseygeek added a commit that referenced this issue Dec 1, 2020
When we have an object that looks like this:

```
{
  "foo" : {
    "cat": "meow"
  },
  "foo.bar" : "baz"
```

where the inner objects of foo are defined both as json objects and via
dot notation, then XContentMapValues.extractValue() will only descend
through the json object and will not collect values from the dot notated paths.
In these cases, the foo.bar values will not be returned in the fields
section of a search response.

This commit adds the ability to check both paths, ensuring that all relevant
values get returned as part of fields.

Fixes #65499
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
>bug :Search/Search Search-related issues that do not fall into other categories Team:Search Meta label for search team
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants