Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add documentation for innerHit on knn nested field #7404

Merged
merged 3 commits into from
Jun 18, 2024
Merged

Add documentation for innerHit on knn nested field #7404

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
a799183
Add documentation for innerHit on knn nested field
heemin32 Jun 18, 2024
35b3d5d
Doc review
kolchfa-aws Jun 18, 2024
cdcb566
Explain excluding source
kolchfa-aws Jun 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
149 changes: 138 additions & 11 deletions _search-plugins/knn/nested-search-knn.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,10 @@ PUT my-knn-index-1
"m": 16
}
}
},
"color": {
"type": "text",
"index": false
}
}
}
Expand All @@ -62,9 +66,9 @@ After you create the index, add some data to it:
```json
PUT _bulk?refresh=true
{ "index": { "_index": "my-knn-index-1", "_id": "1" } }
{"nested_field":[{"my_vector":[1,1,1]},{"my_vector":[2,2,2]},{"my_vector":[3,3,3]}]}
{"nested_field":[{"my_vector":[1,1,1], "color": "blue"},{"my_vector":[2,2,2], "color": "yellow"},{"my_vector":[3,3,3], "color": "white"}]}
{ "index": { "_index": "my-knn-index-1", "_id": "2" } }
{"nested_field":[{"my_vector":[10,10,10]},{"my_vector":[20,20,20]},{"my_vector":[30,30,30]}]}
{"nested_field":[{"my_vector":[10,10,10], "color": "red"},{"my_vector":[20,20,20], "color": "green"},{"my_vector":[30,30,30], "color": "black"}]}
```
{% include copy-curl.html %}

Expand Down Expand Up @@ -94,7 +98,7 @@ Even though all three vectors nearest to the query vector are in document 1, the

```json
{
"took": 23,
"took": 5,
"timed_out": false,
"_shards": {
"total": 1,
Expand All @@ -107,34 +111,37 @@ Even though all three vectors nearest to the query vector are in document 1, the
"value": 2,
"relation": "eq"
},
"max_score": 1,
"max_score": 1.0,
"hits": [
{
"_index": "my-knn-index-1",
"_id": "1",
"_score": 1,
"_score": 1.0,
"_source": {
"nested_field": [
{
"my_vector": [
1,
1,
1
]
],
"color": "blue"
},
{
"my_vector": [
2,
2,
2
]
],
"color": "yellow"
},
{
"my_vector": [
3,
3,
3
]
],
"color": "white"
}
]
}
Expand All @@ -150,21 +157,24 @@ Even though all three vectors nearest to the query vector are in document 1, the
10,
10,
10
]
],
"color": "red"
},
{
"my_vector": [
20,
20,
20
]
],
"color": "green"
},
{
"my_vector": [
30,
30,
30
]
],
"color": "black"
}
]
}
Expand All @@ -174,6 +184,123 @@ Even though all three vectors nearest to the query vector are in document 1, the
}
```

## Inner hits

When you retrieve documents based on matches in nested fields, by default, the response does not contain information about which inner objects matched the query. Thus, it is not apparent why the document is a match. To include information about the matching nested fields in the response, you can provide the `inner_hits` object in your query. To return only certain fields of the matching documents within `inner_hits`, specify the document fields in the `fields` array. Generally, you should also exclude `_source` from the results to avoid returning the whole document. The following example returns only the `color` inner field of the `nested_field`:

```json
GET my-knn-index-1/_search
{
"_source": false,
"query": {
"nested": {
"path": "nested_field",
"query": {
"knn": {
"nested_field.my_vector": {
"vector": [1,1,1],
"k": 2
}
}
},
"inner_hits": {
"_source": false,
"fields":["nested_field.color"]
}
}
}
}
```
{% include copy-curl.html %}

The response contains matching documents. For each matching document, the `inner_hits` object contains only the `nested_field.color` fields of the matched documents in the `fields` array:

```json
{
"took": 4,
"timed_out": false,
"_shards": {
"total": 1,
"successful": 1,
"skipped": 0,
"failed": 0
},
"hits": {
"total": {
"value": 2,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "my-knn-index-1",
"_id": "1",
"_score": 1.0,
"inner_hits": {
"nested_field": {
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 1.0,
"hits": [
{
"_index": "my-knn-index-1",
"_id": "1",
"_nested": {
"field": "nested_field",
"offset": 0
},
"_score": 1.0,
"fields": {
"nested_field.color": [
"blue"
]
}
}
]
}
}
}
},
{
"_index": "my-knn-index-1",
"_id": "2",
"_score": 0.0040983604,
"inner_hits": {
"nested_field": {
"hits": {
"total": {
"value": 1,
"relation": "eq"
},
"max_score": 0.0040983604,
"hits": [
{
"_index": "my-knn-index-1",
"_id": "2",
"_nested": {
"field": "nested_field",
"offset": 0
},
"_score": 0.0040983604,
"fields": {
"nested_field.color": [
"red"
]
}
}
]
}
}
}
}
]
}
}
```

## k-NN search with filtering on nested fields

You can apply a filter to a k-NN search with nested fields. A filter can be applied to either a top-level field or a field inside a nested field.
Expand Down
Loading