From 90e60e1fd5364ccb956361725b6207360e338168 Mon Sep 17 00:00:00 2001 From: vankichi Date: Wed, 24 Jan 2024 17:46:06 +0900 Subject: [PATCH 1/4] :pencil: Add search optimization document Signed-off-by: vankichi --- docs/performance/tuning-search-performance.md | 61 +++++++++++++++++++ 1 file changed, 61 insertions(+) create mode 100644 docs/performance/tuning-search-performance.md diff --git a/docs/performance/tuning-search-performance.md b/docs/performance/tuning-search-performance.md new file mode 100644 index 0000000000..5f1b881d0c --- /dev/null +++ b/docs/performance/tuning-search-performance.md @@ -0,0 +1,61 @@ +# Tuning Search Performance + +ANN is fast, but sometimes it can be improved more by tuning parameters. +This page shows the essence for improve ANN search by the Vald cluster. + +## Tuning Guideline + +When the search results do NOT satisfy the expected result, it can be improved by tuning parameters. + +First of all, we recommend tuning by following the steps below without doing it blindly. + +
+flowchart TD + A[Perform Linear Search API] + B{Is satisfies?} + C[Tuning Parameters to improve precision] + D[Tuning Embedding Models] + E[Perform Search API] + F[Tuning Parameters to improve latency] + A-->B + B-- Yes -->C + B-- No -->D + C--> E + E--> C + E--> F + F--> E +
+ +The best practice is: + +1. Measure Linear Search performance and use it as a baseline for Search API +1. Repeat tuning to improve precision and measure Search API until the conditions are met +1. Repeat tuning to improve latency and measure Search API until the conditions are met + +
+When the results are not good by Linear Search API, it may need to rethink the embedding model for vectorization. +
+ +## Tuning parameters + +There are two viewpoints, client-side and cluster-side, for improving search performance. + +### Client side + +On the client side, parameters of `Search.Config` will affect the search result. + +| | description | how does it affect? | memo | +| :------------------------------------------------------- | :------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------- | +| radius | the search radius for NGT | Define the search range when NGT searches the nearest neighbors | -1 is recommended. | +| epsilon | the search coefficient for NGT | Expansion factor of the NGT search range. | +| Search operation time increases when the epsilon is big. | recommended value range: `0.01 ~ 0.5`
default value: `0.1` | +| timeout(ns) | max time duration until receiving search results. | An error will be returned if the set `num` search results cannot be obtained within the set time.
By setting `min_num`, the search results will be returned if more than `min_num` can be searched within the time. | default value: `3,000,000,000ns` | + +### Cluster-side + +On the cluster side, these parameters can be set by `values.yaml`, affect the search result. + +| | description | how does it affect? | Memo | +| :--------------------------- | :------------------------------------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------ | +| agent.ngt.creation_edge_size | Number of nodes connected to one node | It helps reduce unreachable edges.
The larger it is, the denser the graph structure will be, but the memory usage, search speed, and index construction time will increase accordingly. | default value: `20` | +| agent.ngt.search_edge_size | Number of nodes to search from the origin node when searching | The number of nodes to search will increase.
Accuracy will be higher, but speed will be lower.
Adjust if adjusting the radius and epsilon does not improve the situation. | default value: `10` | From 91817a8dc2f46998da29ef22c9b9a677a5c6b694 Mon Sep 17 00:00:00 2001 From: vankichi Date: Wed, 24 Jan 2024 17:48:22 +0900 Subject: [PATCH 2/4] :pencil: Fix Signed-off-by: vankichi --- docs/performance/tuning-search-performance.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/docs/performance/tuning-search-performance.md b/docs/performance/tuning-search-performance.md index 5f1b881d0c..a016d4f93f 100644 --- a/docs/performance/tuning-search-performance.md +++ b/docs/performance/tuning-search-performance.md @@ -44,12 +44,11 @@ There are two viewpoints, client-side and cluster-side, for improving search per On the client side, parameters of `Search.Config` will affect the search result. -| | description | how does it affect? | memo | -| :------------------------------------------------------- | :------------------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------- | -| radius | the search radius for NGT | Define the search range when NGT searches the nearest neighbors | -1 is recommended. | -| epsilon | the search coefficient for NGT | Expansion factor of the NGT search range. | -| Search operation time increases when the epsilon is big. | recommended value range: `0.01 ~ 0.5`
default value: `0.1` | -| timeout(ns) | max time duration until receiving search results. | An error will be returned if the set `num` search results cannot be obtained within the set time.
By setting `min_num`, the search results will be returned if more than `min_num` can be searched within the time. | default value: `3,000,000,000ns` | +| | description | how does it affect? | memo | +| :---------- | :------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------ | +| radius | the search radius for NGT | Define the search range when NGT searches the nearest neighbors | -1 is recommended. | +| epsilon | the search coefficient for NGT | Expansion factor of the NGT search range.
Search operation time increases when the epsilon is big. | recommended value range: `0.01 ~ 0.5`
default value: `0.1` | +| timeout(ns) | max time duration until receiving search results. | An error will be returned if the set `num` search results cannot be obtained within the set time.
By setting `min_num`, the search results will be returned if more than `min_num` can be searched within the time. | default value: `3,000,000,000ns` | ### Cluster-side From 0b7d947f4291c546295db7783d9a147ec626a5a8 Mon Sep 17 00:00:00 2001 From: vankichi Date: Fri, 26 Jan 2024 15:36:51 +0900 Subject: [PATCH 3/4] :pencil: Add links Signed-off-by: vankichi --- docs/performance/tuning-search-performance.md | 26 +++++++++++-------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/docs/performance/tuning-search-performance.md b/docs/performance/tuning-search-performance.md index a016d4f93f..bc7f7d58b6 100644 --- a/docs/performance/tuning-search-performance.md +++ b/docs/performance/tuning-search-performance.md @@ -28,7 +28,7 @@ flowchart TD The best practice is: -1. Measure Linear Search performance and use it as a baseline for Search API +1. Measure [Linear Search](../../docs/api/search.md#linearsearch-rpc) performance and use it as a baseline for Search API 1. Repeat tuning to improve precision and measure Search API until the conditions are met 1. Repeat tuning to improve latency and measure Search API until the conditions are met @@ -40,21 +40,25 @@ When the results are not good by Linear Search API, it may need to rethink the e There are two viewpoints, client-side and cluster-side, for improving search performance. +
+There is a trade-off between search speed and accuracy, so tuning accuracy at first is recommended. +
+ ### Client side -On the client side, parameters of `Search.Config` will affect the search result. +On the client side, parameters of [`Search.Config`](../../docs/api/search.md#input) will affect the search result. -| | description | how does it affect? | memo | -| :---------- | :------------------------------------------------ | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------ | -| radius | the search radius for NGT | Define the search range when NGT searches the nearest neighbors | -1 is recommended. | -| epsilon | the search coefficient for NGT | Expansion factor of the NGT search range.
Search operation time increases when the epsilon is big. | recommended value range: `0.01 ~ 0.5`
default value: `0.1` | -| timeout(ns) | max time duration until receiving search results. | An error will be returned if the set `num` search results cannot be obtained within the set time.
By setting `min_num`, the search results will be returned if more than `min_num` can be searched within the time. | default value: `3,000,000,000ns` | +| | description | how does it affect? | memo | +| :---------- | :---------------------------------------------------------------------------------------------------------------------------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------------------------------------------------ | +| radius | the search radius for NGT
[ref: search_radius](https://github.com/yahoojapan/NGT/tree/main/bin/ngt#search) | Define the search range when NGT searches the nearest neighbors | recommended value: `-1` | +| epsilon | the search coefficient for NGT
[ref: search_range_coefficient](https://github.com/yahoojapan/NGT/tree/main/bin/ngt#search) | Expansion factor of the NGT search range.
Search operation time increases when the epsilon is big. | recommended value range: `0.01 ~ 0.3`
default value: `0.1` | +| timeout(ns) | max time duration until receiving search results. | An error will be returned if the set `num` search results cannot be obtained within the set time.
By setting `min_num`, the search results will be returned if more than `min_num` can be searched within the time. | default value: `3,000,000,000ns` | ### Cluster-side On the cluster side, these parameters can be set by `values.yaml`, affect the search result. -| | description | how does it affect? | Memo | -| :--------------------------- | :------------------------------------------------------------ | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------ | -| agent.ngt.creation_edge_size | Number of nodes connected to one node | It helps reduce unreachable edges.
The larger it is, the denser the graph structure will be, but the memory usage, search speed, and index construction time will increase accordingly. | default value: `20` | -| agent.ngt.search_edge_size | Number of nodes to search from the origin node when searching | The number of nodes to search will increase.
Accuracy will be higher, but speed will be lower.
Adjust if adjusting the radius and epsilon does not improve the situation. | default value: `10` | +| | description | how does it affect? | Memo | +| :--------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------- | :----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | :------------------ | +| agent.ngt.creation_edge_size | Number of nodes connected to one node
[ref: no_of_edges](https://github.com/yahoojapan/NGT/tree/main/bin/ngt#create) | It helps reduce unreachable edges.
The larger it is, the denser the graph structure will be, but the memory usage, search speed, and index construction time will increase accordingly. | default value: `20` | +| agent.ngt.search_edge_size | Number of nodes to search from the origin node when searching
[ref: no_of_edges_at_search_at](https://github.com/yahoojapan/NGT/tree/main/bin/ngt#create) | The number of nodes to search will increase.
Accuracy will be higher, but speed will be lower.
Adjust if adjusting the radius and epsilon does not improve the situation. | default value: `10` | From 0f6158c25accf4225625414f63c62e2f51efe89c Mon Sep 17 00:00:00 2001 From: vankichi Date: Mon, 29 Jan 2024 22:46:59 +0900 Subject: [PATCH 4/4] :pencil: Fix tag Signed-off-by: vankichi --- docs/performance/tuning-search-performance.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/performance/tuning-search-performance.md b/docs/performance/tuning-search-performance.md index bc7f7d58b6..b0cb6586df 100644 --- a/docs/performance/tuning-search-performance.md +++ b/docs/performance/tuning-search-performance.md @@ -9,7 +9,7 @@ When the search results do NOT satisfy the expected result, it can be improved b First of all, we recommend tuning by following the steps below without doing it blindly. -
+```mermaid flowchart TD A[Perform Linear Search API] B{Is satisfies?} @@ -24,7 +24,7 @@ flowchart TD E--> C E--> F F--> E -
+``` The best practice is: