Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add doc for binary format support in k-NN #7840

Merged
merged 12 commits into from
Aug 1, 2024
Merged

Add doc for binary format support in k-NN #7840

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
0eea1d3
Add doc for binary format support in k-NN
junqiu-lei Jul 25, 2024
ce3eac4
Resolve tech feedback
junqiu-lei Jul 29, 2024
d877a20
Doc review
kolchfa-aws Aug 1, 2024
cef34e5
Add newline
kolchfa-aws Aug 1, 2024
d693df4
Formatting
kolchfa-aws Aug 1, 2024
604827a
Link fix
kolchfa-aws Aug 1, 2024
d30fe78
Apply suggestions from code review
kolchfa-aws Aug 1, 2024
d1d0272
Add query results to examples
junqiu-lei Aug 1, 2024
9f63324
Rephrased sentences and changed vector field name
kolchfa-aws Aug 1, 2024
27b8295
Editorial review
kolchfa-aws Aug 1, 2024
dae70b1
Remove details from one of the requests
kolchfa-aws Aug 1, 2024
06a104f
Merge branch 'main' into knn-binary-format
kolchfa-aws Aug 1, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 40 additions & 46 deletions _field-types/supported-field-types/knn-vector.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The [k-NN plugin]({{site.url}}{{site.baseurl}}/search-plugins/knn/index/) introd

## Example

For example, to map `my_vector1` as a `knn_vector`, use the following request:
For example, to map `my_vector` as a `knn_vector`, use the following request:

```json
PUT test-index
Expand All @@ -26,7 +26,7 @@ PUT test-index
},
"mappings": {
"properties": {
"my_vector1": {
"my_vector": {
"type": "knn_vector",
"dimension": 3,
"method": {
Expand Down Expand Up @@ -110,7 +110,7 @@ PUT test-index
},
"mappings": {
"properties": {
"my_vector1": {
"my_vector": {
"type": "knn_vector",
"dimension": 3,
"data_type": "byte",
Expand All @@ -135,15 +135,15 @@ Then ingest documents as usual. Make sure each dimension in the vector is in the
```json
PUT test-index/_doc/1
{
"my_vector1": [-126, 28, 127]
"my_vector": [-126, 28, 127]
}
```
{% include copy-curl.html %}

```json
PUT test-index/_doc/2
{
"my_vector1": [100, -128, 0]
"my_vector": [100, -128, 0]
}
```
{% include copy-curl.html %}
Expand All @@ -156,7 +156,7 @@ GET test-index/_search
"size": 2,
"query": {
"knn": {
"my_vector1": {
"my_vector": {
"vector": [26, -120, 99],
"k": 2
}
Expand Down Expand Up @@ -301,7 +301,7 @@ PUT /test-binary-hnsw
},
"mappings": {
"properties": {
"my_vector1": {
"my_vector": {
"type": "knn_vector",
"dimension": 8,
"data_type": "binary",
Expand Down Expand Up @@ -363,11 +363,11 @@ GET /test-binary-hnsw/_search
```
{% include copy-curl.html %}

Query results will be returned:
The response contains two vectors closest to the query vector:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"the" two vectors


<details markdown="block">
<summary>
Results
Response
</summary>
{: .text-delta}

Expand Down Expand Up @@ -413,7 +413,7 @@ Query results will be returned:
]
}
}
```
```
</details>

### Example: IVF
Expand Down Expand Up @@ -442,7 +442,7 @@ Ingest training data containing binary vectors into the training index:

<details markdown="block">
<summary>
Bulk ingest for train index
Bulk ingest request
</summary>
{: .text-delta}

Expand Down Expand Up @@ -588,14 +588,8 @@ PUT test-binary-ivf

Ingest the data containing the binary vectors that you want to search into the created index:

<details markdown="block">
<summary>
Bulk ingest for test-binary-ivf
</summary>
{: .text-delta}

```json
PUT _bulk?refresh=true\
PUT _bulk?refresh=true
{"index": {"_index": "test-binary-ivf", "_id": "1"}}
{"my_vector": [7], "price": 4.4}
{"index": {"_index": "test-binary-ivf", "_id": "2"}}
Expand All @@ -608,7 +602,6 @@ PUT _bulk?refresh=true\
{"my_vector": [80], "price": 16.5}
```
{% include copy-curl.html %}
</details>

Finally, search the data. Be sure to provide a binary vector in the k-NN vector field:

Expand All @@ -618,7 +611,7 @@ GET test-binary-ivf/_search
"size": 2,
"query": {
"knn": {
"my_vector1": {
"my_vector": {
"vector": [8],
"k": 2
}
Expand All @@ -628,15 +621,16 @@ GET test-binary-ivf/_search
```
{% include copy-curl.html %}

Query results will be returned:
The response contains two vectors closest to the query vector:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same


<details markdown="block">
<summary>
Results
Response
</summary>
{: .text-delta}

```json
GET /_plugins/_knn/models/my-model?filter_path=state
{
"took": 7,
"timed_out": false,
Expand All @@ -646,35 +640,35 @@ Query results will be returned:
"skipped": 0,
"failed": 0
},
"hits": {
"hits": {
"total": {
"value": 2,
"relation": "eq"
"value": 2,
"relation": "eq"
},
"max_score": 0.5,
"hits": [
{
"_index": "test-binary-ivf",
"_id": "2",
"_score": 0.5,
"_source": {
"my_vector1": [
10
],
"price": 14.2
}
},
{
"_index": "test-binary-ivf",
"_id": "3",
"_score": 0.25,
"_source": {
"my_vector1": [
15
],
"price": 19.1
}
{
"_index": "test-binary-ivf",
"_id": "2",
"_score": 0.5,
"_source": {
"my_vector": [
10
],
"price": 14.2
}
},
{
"_index": "test-binary-ivf",
"_id": "3",
"_score": 0.25,
"_source": {
"my_vector": [
15
],
"price": 19.1
}
}
]
}
}
Expand Down
Loading