Skip to content

Commit

Permalink
Adds bulk operations to Go client. Refactors Go client. (#1038)
Browse files Browse the repository at this point in the history
* Adds bulk operations to Go client. Refactors Go client.

Signed-off-by: Fanit Kolchina <[email protected]>

* Incorporated editorial comments

Signed-off-by: Fanit Kolchina <[email protected]>

Signed-off-by: Fanit Kolchina <[email protected]>
(cherry picked from commit 441df4c)
  • Loading branch information
kolchfa-aws authored and github-actions[bot] committed Sep 6, 2022
1 parent 047b9b9 commit 317244b
Showing 1 changed file with 198 additions and 13 deletions.
211 changes: 198 additions & 13 deletions _clients/go.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,21 +11,181 @@ The OpenSearch Go client lets you connect your Go application with the data in y

## Setup

If you're creating a new project:
If you're starting a new project, create a new module by running the following command:

```go
go mod init
go mod init <mymodulename>
```

To add the client to your project, import it like any other module:
To add the Go client to your project, import it like any other module:

```go
go get github.com/opensearch-project/opensearch-go
```
## Connecting to OpenSearch

## Sample code
To connect to the default OpenSearch host, create a client object with the address `https://localhost:9200` if you are using the Security plugin:

This sample code creates a client, adds an index with non-default settings, inserts a document, searches for the document, deletes the document, and finally deletes the index:
```go
client, err := opensearch.NewClient(opensearch.Config{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
Addresses: []string{"https://localhost:9200"},
Username: "admin", // For testing only. Don't store credentials in code.
Password: "admin",
})
```

If you are not using the Security plugin, create a client object with the address `http://localhost:9200`:

```go
client, err := opensearch.NewClient(opensearch.Config{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
Addresses: []string{"http://localhost:9200"},
})
```

The Go client constructor takes an `opensearch.Config{}` type, which can be customized using options such as a list of OpenSearch node addresses or a username and password combination.

To connect to multiple OpenSearch nodes, specify them in the `Addresses` parameter:

```go
var (
urls = []string{"http://localhost:9200", "http://localhost:9201", "http://localhost:9202"}
)

client, err := opensearch.NewClient(opensearch.Config{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
Addresses: urls,
})
```

The Go client retries requests for a maximum of three times by default. To customize the number of retries, set the `MaxRetries` parameter. Additionally, you can change the list of response codes for which a request is retried by setting the `RetryOnStatus` parameter. The following code snippet creates a new Go client with custom `MaxRetries` and `RetryOnStatus` values:

```go
client, err := opensearch.NewClient(opensearch.Config{
Transport: &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
},
Addresses: []string{"http://localhost:9200"},
MaxRetries: 5,
RetryOnStatus: []int{502, 503, 504},
})
```

## Creating an index

To create an OpenSearch index, use the `IndicesCreateRequest` method. You can use the following code to construct a JSON object with custom settings :

```go
settings := strings.NewReader(`{
'settings': {
'index': {
'number_of_shards': 1,
'number_of_replicas': 0
}
}
}`)

res := opensearchapi.IndicesCreateRequest{
Index: "go-test-index1",
Body: settings,
}
```

## Indexing a document

You can index a document into OpenSearch using the `IndexRequest` method:

```go
document := strings.NewReader(`{
"title": "Moneyball",
"director": "Bennett Miller",
"year": "2011"
}`)

docId := "1"
req := opensearchapi.IndexRequest{
Index: "go-test-index1",
DocumentID: docId,
Body: document,
}
insertResponse, err := req.Do(context.Background(), client)
```

## Performing bulk operations

You can perform several operations at the same time by using the `Bulk` method of the client. The operations may be of the same type or of different types.

```go
blk, err := client.Bulk(
strings.NewReader(`
{ "index" : { "_index" : "go-test-index1", "_id" : "2" } }
{ "title" : "Interstellar", "director" : "Christopher Nolan", "year" : "2014"}
{ "create" : { "_index" : "go-test-index1", "_id" : "3" } }
{ "title" : "Star Trek Beyond", "director" : "Justin Lin", "year" : "2015"}
{ "update" : {"_id" : "3", "_index" : "go-test-index1" } }
{ "doc" : {"year" : "2016"} }
`),
)
```

## Searching for documents

The easiest way to search for documents is to construct a query string. The following code uses a `multi_match` query to search for "miller" in the title and director fields. It boosts the documents where "miller" is in the title field.

```go
content := strings.NewReader(`{
"size": 5,
"query": {
"multi_match": {
"query": "miller",
"fields": ["title^2", "director"]
}
}
}`)

search := opensearchapi.SearchRequest{
Index: []string{"go-test-index1"},
Body: content,
}

searchResponse, err := search.Do(context.Background(), client)
```

## Deleting a document

You can delete a document using the `DeleteRequest` method:

```go
delete := opensearchapi.DeleteRequest{
Index: "go-test-index1",
DocumentID: "1",
}

deleteResponse, err := delete.Do(context.Background(), client)
```

## Deleting an index

You can delete an index using the `IndicesDeleteRequest` method:

```go
deleteIndex := opensearchapi.IndicesDeleteRequest{
Index: []string{"go-test-index1"},
}

deleteIndexResponse, err := deleteIndex.Do(context.Background(), client)
```

## Sample program

This sample program creates a client, adds an index with non-default settings, inserts a document, performs bulk operations, searches for the document, deletes the document, and, finally, deletes the index:

```go
package main
Expand Down Expand Up @@ -58,21 +218,23 @@ func main() {
// Print OpenSearch version information on console.
fmt.Println(client.Info())

// Define index mapping.
mapping := strings.NewReader(`{
// Define index settings.
settings := strings.NewReader(`{
'settings': {
'index': {
'number_of_shards': 4
'number_of_shards': 1,
'number_of_replicas': 2
}
}
}`)

// Create an index with non-default settings.
res := opensearchapi.IndicesCreateRequest{
Index: IndexName,
Body: mapping,
Body: settings,
}
fmt.Println("creating index", res)
fmt.Println("Creating index")
fmt.Println(res)

// Add a document to the index.
document := strings.NewReader(`{
Expand All @@ -92,7 +254,27 @@ func main() {
fmt.Println("failed to insert document ", err)
os.Exit(1)
}
fmt.Println("Inserting a document")
fmt.Println(insertResponse)

// Perform bulk operations.
blk, err := client.Bulk(
strings.NewReader(`
{ "index" : { "_index" : "go-test-index1", "_id" : "2" } }
{ "title" : "Interstellar", "director" : "Christopher Nolan", "year" : "2014"}
{ "create" : { "_index" : "go-test-index1", "_id" : "3" } }
{ "title" : "Star Trek Beyond", "director" : "Justin Lin", "year" : "2015"}
{ "update" : {"_id" : "3", "_index" : "go-test-index1" } }
{ "doc" : {"year" : "2016"} }
`),
)

if err != nil {
fmt.Println("failed to perform bulk operations", err)
os.Exit(1)
}
fmt.Println("Performing bulk operations")
fmt.Println(blk)

// Search for the document.
content := strings.NewReader(`{
Expand All @@ -106,6 +288,7 @@ func main() {
}`)

search := opensearchapi.SearchRequest{
Index: []string{IndexName},
Body: content,
}

Expand All @@ -114,6 +297,7 @@ func main() {
fmt.Println("failed to search document ", err)
os.Exit(1)
}
fmt.Println("Searching for a document")
fmt.Println(searchResponse)

// Delete the document.
Expand All @@ -127,10 +311,10 @@ func main() {
fmt.Println("failed to delete document ", err)
os.Exit(1)
}
fmt.Println("deleting document")
fmt.Println("Deleting a document")
fmt.Println(deleteResponse)

// Delete previously created index.
// Delete the previously created index.
deleteIndex := opensearchapi.IndicesDeleteRequest{
Index: []string{IndexName},
}
Expand All @@ -140,6 +324,7 @@ func main() {
fmt.Println("failed to delete index ", err)
os.Exit(1)
}
fmt.Println("deleting index", deleteIndexResponse)
fmt.Println("Deleting the index")
fmt.Println(deleteIndexResponse)
}
```

0 comments on commit 317244b

Please sign in to comment.