-
Notifications
You must be signed in to change notification settings - Fork 500
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Refactors JS client helper documentation (#1358)
* Fix Circuit Breaker section in JS client docs Fix #823 Signed-off-by: Robert Da Silva <[email protected]> * Add documentation for JS client bulk helper Signed-off-by: Robert Da Silva <[email protected]> * Refactors js client helper documentation * Incorporated tech review comments Signed-off-by: Fanit Kolchina <[email protected]> * Incorporated doc review comments Signed-off-by: Fanit Kolchina <[email protected]> * Update _clients/javascript/helpers.md Co-authored-by: Alice Williams <[email protected]> * Update _clients/javascript/helpers.md Co-authored-by: Alice Williams <[email protected]> * Update helpers.md * Incorporated tech review feedback Signed-off-by: Fanit Kolchina <[email protected]> * Implemented editorial comments Signed-off-by: Fanit Kolchina <[email protected]> Signed-off-by: Robert Da Silva <[email protected]> Signed-off-by: Fanit Kolchina <[email protected]> Co-authored-by: Robert Da Silva <[email protected]> Co-authored-by: Alice Williams <[email protected]> (cherry picked from commit 89f966a)
- Loading branch information
1 parent
641b749
commit e5968c4
Showing
13 changed files
with
229 additions
and
28 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: OpenSearch CLI | ||
nav_order: 52 | ||
nav_order: 80 | ||
has_children: false | ||
--- | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: Go client | ||
nav_order: 80 | ||
nav_order: 50 | ||
--- | ||
|
||
# Go client | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: Grafana | ||
nav_order: 150 | ||
nav_order: 200 | ||
has_children: false | ||
--- | ||
|
||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: Java client | ||
nav_order: 65 | ||
nav_order: 30 | ||
--- | ||
|
||
# Java client | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,194 @@ | ||
--- | ||
layout: default | ||
title: Helper methods | ||
parent: JavaScript client | ||
nav_order: 2 | ||
--- | ||
|
||
# Helper methods | ||
|
||
Helper methods simplify the use of complicated API tasks. | ||
|
||
## Bulk helper | ||
|
||
The bulk helper simplifies making complex bulk API requests. | ||
|
||
### Usage | ||
|
||
The following code creates a bulk helper instance: | ||
|
||
```javascript | ||
const { Client } = require('@opensearch-project/opensearch') | ||
const documents = require('./docs.json') | ||
|
||
const client = new Client({ ... }) | ||
|
||
const result = await client.helpers.bulk({ | ||
datasource: documents, | ||
onDocument (doc) { | ||
return { | ||
index: { _index: 'example-index' } | ||
} | ||
} | ||
}) | ||
|
||
console.log(result) | ||
``` | ||
|
||
Bulk helper operations return an object with the following fields: | ||
|
||
```json | ||
{ | ||
total: number, | ||
failed: number, | ||
retry: number, | ||
successful: number, | ||
time: number, | ||
bytes: number, | ||
aborted: boolean | ||
} | ||
``` | ||
|
||
#### Bulk helper configuration options | ||
|
||
When creating a new bulk helper instance, you can use the following configuration options. | ||
|
||
| Option | Data Type | Required/Default | Description | ||
| :--- | :--- | :--- | :--- | ||
| `datasource` | An array, async generator or a readable stream of strings or objects | Required | Represents the documents you need to create, delete, index, or update. | ||
| `onDocument` | Function | Required | A function to be invoked with each document in the given `datasource`. It returns the operation to be executed for this document. Optionally, the document can be manipulated for `create` and `index` operations by returning a new document as part of the function's result. | ||
| `concurrency` | Integer | Optional. Default is 5. | The number of requests to be executed in parallel. | ||
| `flushBytes` | Integer | Optional. Default is 5,000,000. | Maximum bulk body size to send in bytes. | ||
| `flushInterval` | Integer | Optional. Default is 30,000. | Time in milliseconds to wait before flushing the body after the last document has been read. | ||
| `onDrop` | Function | Optional. Default is `noop`. | A function to be invoked for every document that can’t be indexed after reaching the maximum number of retries. | ||
| `refreshOnCompletion` | Boolean | Optional. Default is false. | Whether or not a refresh should be run on all affected indexes at the end of the bulk operation. | ||
| `retries` | Integer | Optional. Defaults to the client's `maxRetries` value. | The number of times an operation is retried before `onDrop` is called for that document. | ||
| `wait` | Integer | Optional. Default is 5,000. | Time in milliseconds to wait before retrying an operation. | ||
|
||
### Examples | ||
|
||
The following examples illustrate the index, create, update, and delete bulk helper operations. | ||
|
||
#### Index | ||
|
||
The index operation creates a new document if it doesn’t exist and recreates the document if it already exists. | ||
|
||
The following bulk operation indexes documents into `example-index`: | ||
|
||
```javascript | ||
client.helpers.bulk({ | ||
datasource: arrayOfDocuments, | ||
onDocument (doc) { | ||
return { | ||
index: { _index: 'example-index' } | ||
} | ||
} | ||
}) | ||
``` | ||
|
||
The following bulk operation indexes documents into `example-index` with document overwrite: | ||
|
||
```javascript | ||
client.helpers.bulk({ | ||
datasource: arrayOfDocuments, | ||
onDocument (doc) { | ||
return [ | ||
{ | ||
index: { _index: 'example-index' } | ||
}, | ||
{ ...doc, createdAt: new Date().toISOString() } | ||
] | ||
} | ||
}) | ||
``` | ||
|
||
#### Create | ||
|
||
The create operation creates a new document only if the document does not already exist. | ||
|
||
The following bulk operation creates documents in the `example-index`: | ||
|
||
```javascript | ||
client.helpers.bulk({ | ||
datasource: arrayOfDocuments, | ||
onDocument (doc) { | ||
return { | ||
create: { _index: 'example-index', _id: doc.id } | ||
} | ||
} | ||
}) | ||
``` | ||
|
||
The following bulk operation creates documents in the `example-index` with document overwrite: | ||
|
||
```javascript | ||
client.helpers.bulk({ | ||
datasource: arrayOfDocuments, | ||
onDocument (doc) { | ||
return [ | ||
{ | ||
create: { _index: 'example-index', _id: doc.id } | ||
}, | ||
{ ...doc, createdAt: new Date().toISOString() } | ||
] | ||
} | ||
}) | ||
``` | ||
|
||
#### Update | ||
|
||
The update operation updates the document with the fields being sent. The document must already exist in the index. | ||
|
||
The following bulk operation updates documents in the `arrayOfDocuments`: | ||
|
||
```javascript | ||
client.helpers.bulk({ | ||
datasource: arrayOfDocuments, | ||
onDocument (doc) { | ||
// The update operation always requires a tuple to be returned, with the | ||
// first element being the action and the second being the update options. | ||
return [ | ||
{ | ||
update: { _index: 'example-index', _id: doc.id } | ||
}, | ||
{ doc_as_upsert: true } | ||
] | ||
} | ||
}) | ||
``` | ||
|
||
The following bulk operation updates documents in the `arrayOfDocuments` with document overwrite: | ||
|
||
```javascript | ||
client.helpers.bulk({ | ||
datasource: arrayOfDocuments, | ||
onDocument (doc) { | ||
return [ | ||
{ | ||
update: { _index: 'example-index', _id: doc.id } | ||
}, | ||
{ | ||
doc: { ...doc, createdAt: new Date().toISOString() }, | ||
doc_as_upsert: true | ||
} | ||
] | ||
} | ||
}) | ||
``` | ||
|
||
#### Delete | ||
|
||
The delete operation deletes a document. | ||
|
||
The following bulk operation deletes documents from the `example-index`: | ||
|
||
```javascript | ||
client.helpers.bulk({ | ||
datasource: arrayOfDocuments, | ||
onDocument (doc) { | ||
return { | ||
delete: { _index: 'example-index', _id: doc.id } | ||
} | ||
} | ||
}) | ||
``` |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: PHP client | ||
nav_order: 90 | ||
nav_order: 70 | ||
--- | ||
|
||
# PHP client | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: Python client | ||
nav_order: 70 | ||
nav_order: 10 | ||
--- | ||
|
||
# Python client | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,7 +1,7 @@ | ||
--- | ||
layout: default | ||
title: Ruby client | ||
nav_order: 77 | ||
nav_order: 60 | ||
has_children: false | ||
--- | ||
|
||
|