Skip to content

Commit

Permalink
fix errors
Browse files Browse the repository at this point in the history
Signed-off-by: zhichao-aws <[email protected]>
  • Loading branch information
zhichao-aws committed Jun 8, 2024
1 parent 60dd58e commit 8db6f92
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
4 changes: 2 additions & 2 deletions _ml-commons-plugin/remote-models/blueprints.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,13 +87,13 @@ The `client_config` parameter supports the following options.

| Field | Data type | Description |
|:---------------------|:----------|:---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `max_connection` | Integer | The maximum number of concurrent connections that the client can establish with the server. Some remote service like SageMaker constraints the max concurrent connections and throw throttling exception if the concurrent connections exceeds the threshold. And the max concurent connections from OpenSearch is `max_connection`*`node_number_for_connector`. To mitigate this issue, please try decrease the value of this parameter, and modify the retry settings in client_config. Default value is 30. |
| `max_connection` | Integer | The maximum number of concurrent connections that the client can establish with the server. Some remote service like SageMaker constraints the max concurrent connections and throw throttling exception if the concurrent connections exceeds the threshold. And the max concurrent connections from OpenSearch is `max_connection`*`node_number_for_connector`. To mitigate this issue, try decrease the value of this parameter, and modify the retry settings in client_config. Default value is 30. |
| `connection_timeout` | Integer | The maximum amount of time (in seconds) that the client will wait while trying to establish a connection to the server. A timeout prevents the client from waiting indefinitely and allows it to recover from unreachable network endpoints. |
| `read_timeout` | Integer | The maximum amount of time (in seconds) that the client will wait for a response from the server after sending a request. Useful when the server is slow to respond or encounters issues while processing a request. |
| `retry_backoff_policy` | String | The backoff policy of retry for remote connector. Useful when there is spike traffic causing throttling exception. Supported policy: `constant`, `exponential_equal_jitter`, `exponential_full_jitter`. Default value is `constant`. |
| `max_retry_times` | Integer | The maximum retry times for a single remote inference request. Useful when there is spike traffic causing throttling exception. When set as 0, retry is disabled. When set as -1, we don't limit the retry by retry times. Set it to positive integers means the max retry times. Default value is 0. |
| `retry_backoff_millis` | Integer | The base backoff time in milliseconds for retry policy. The suspend time during two retries is determined by this parameter and `retry_backoff_policy`. Default value is 200. |
| `retry_timeout_seconds` | Integer | The timeout value in seconds to endup the retry. If the retry can not succeed in this time, the connector will stop retrying and throw exceptions. Default value is 30. |
| `retry_timeout_seconds` | Integer | The timeout value in seconds to end up the retry. If the retry can not succeed in this time, the connector will stop retrying and throw exceptions. Default value is 30. |

## Built-in pre- and post-processing functions

Expand Down
2 changes: 1 addition & 1 deletion _search-plugins/neural-sparse-search.md
Original file line number Diff line number Diff line change
Expand Up @@ -396,7 +396,7 @@ The response contains both documents:

1. When using connectors to call remote service like SageMaker, the ingestion/search fails due to remote connector throttling exception. How to mitigate this?

- Please modify the connector [`client_config`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/#configuration-parameters) to decrease the `max_connection` to prevent max concurrent connection exceeds the threshold of remote service. You can also modify the retry settings to flatten the request spike during ingestion.
- Modify the connector [`client_config`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/blueprints/#configuration-parameters) to decrease the `max_connection` to prevent max concurrent connection exceeds the threshold of remote service. You can also modify the retry settings to flatten the request spike during ingestion.

> **_NOTE:_** For versions before OpenSearch 2.15, the SageMaker throttling exception will be thrown as `"error": {
"type": "status_exception",

Check failure on line 402 in _search-plugins/neural-sparse-search.md

View workflow job for this annotation

GitHub Actions / style-job

[vale] reported by reviewdog 🐶 [OpenSearch.Spelling] Error: _exception. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks. Raw Output: {"message": "[OpenSearch.Spelling] Error: _exception. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_search-plugins/neural-sparse-search.md", "range": {"start": {"line": 402, "column": 27}}}, "severity": "ERROR"}
Expand Down

0 comments on commit 8db6f92

Please sign in to comment.