Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add cluster awareness and decommission docs #2438

Merged
merged 30 commits into from
Jan 23, 2023
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
ac6879c
Add cluster awareness and decommission docs
Naarcha-AWS Jan 19, 2023
f1a1656
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 19, 2023
2009556
Edit technical feedback
Naarcha-AWS Jan 19, 2023
9ab64cd
Add new cluster awareness examples
Naarcha-AWS Jan 20, 2023
470ea01
Add technical feedback
Naarcha-AWS Jan 23, 2023
4855b78
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
430c8f5
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
d17cf00
Add Caroline's feedback
Naarcha-AWS Jan 23, 2023
fcefe54
Add one more tweak
Naarcha-AWS Jan 23, 2023
6ae0497
Update _ml-commons-plugin/cluster-settings.md
Naarcha-AWS Jan 23, 2023
d225427
Update _ml-commons-plugin/cluster-settings.md
Naarcha-AWS Jan 23, 2023
ab9c864
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
78b17a3
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
dfbd089
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
8834026
Update _ml-commons-plugin/cluster-settings.md
Naarcha-AWS Jan 23, 2023
f1647f6
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
9c528ec
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
ddce7a0
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
164ab0c
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
d571148
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
f8f6a82
Update _ml-commons-plugin/cluster-settings.md
Naarcha-AWS Jan 23, 2023
69859c7
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
17d7c97
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
0ed2074
Update _api-reference/cluster-decommission.md
Naarcha-AWS Jan 23, 2023
88183c8
Update _api-reference/cluster-awareness.md
Naarcha-AWS Jan 23, 2023
306d242
Update _api-reference/cluster-decommission.md
Naarcha-AWS Jan 23, 2023
b8eb313
Add editoiral feedback
Naarcha-AWS Jan 23, 2023
be4b3c4
Fix typos
Naarcha-AWS Jan 23, 2023
94c4a10
Final editorial note
Naarcha-AWS Jan 23, 2023
7254b92
Fix merge conflicts
Naarcha-AWS Jan 23, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
106 changes: 106 additions & 0 deletions _api-reference/cluster-awareness.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
---
layout: default
title: Cluster routing and awareness
nav_order: 16
---

# Cluster routing and awareness

To control the distribution of search or HTTP traffic, you can use the weights per awareness attribute to control the distribution of search or HTTP traffic across zones. This is commonly used for zonal deployments, heterogenous instances, and weighing traffic away from zones during zonal failure.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

## HTTP and Path methods
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

```
PUT /_cluster/routing/awareness/<attribute>/weights
GET /_cluster/routing/awareness/<attribute>/weights?local
GET /_cluster/routing/awareness/<attribute>/weights
```

## Path parameters

Parameter | Type | Description
:--- | :--- | :---
attribute | String | The name of the awareness attribute, usually `zone`. The attribute name must match the values listed in the request body when assigning weights to zones.

## Request body

You can assign weights to zone within the request body of the PUT request. Weights can be set in any ration, for example, 2:3:5. In a 2:3:5 ratio with three zones, for every 100 requests sent to the cluster, each zone would receive either 20, 30, or 50 search requests in a random order.

When assigned a weight of `0`, the zone is cut off from an search traffic.

In the following example request body, `zone_1` and `zone_2` receive 50 requests each, whereas `zone_3` is cut for from receiving requests.

```
{
"zone_1": "5",
"zone_2": "5",
"zone_3": "0",
}
```

## Example: Weighted round robin search

The following sample request creates a round robin shard allocation for search traffic.

### Request

```
PUT /_cluster/routing/awareness/zone/weights
{
"zone_1": "1",
"zone_2": "1",
"zone_3": "0"
}
```

Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
### Response

```
{
"acknowledged": true
}
```

## Example: Get weight for a local node

The following sample request gets the weight for a local node as well as any zones in your cluster. This gives you the ability to see if you local node is weighted over your zones.

### Request

```
GET /_cluster/routing/awareness/zone/weights?local
```

### Response

```json
{
"zone_1": "1",
"zone_2": "1",
"zone_3": "0",
"node_weight" : "0"
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
}
```

## Example: Get weights for all zones

The following sample request gets weights for all zones.

### Request

```
GET /_cluster/routing/awareness/<attribute>/weights
```

### Response

OpenSearch responds with the weight of each zone.

```json
{
"zone_1": "1",
"zone_2": "1",
"zone_3": "0"
}
```
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

79 changes: 79 additions & 0 deletions _api-reference/cluster-decommission.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
layout: default
title: Cluster decommission
nav_order: 20
---

# Cluster decommission

The cluster decommission operation adds support for zonal cluster deployments based on awareness. When a particular zone overloads with traffic, you can decommission the zone to ensure uptime during large search requests. This greatly benefits use cases with multi-zone deployment, since rolling restarts per node can take more time than decommissioning a zone.

Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
For more information on allocation awareness, see [Shard allocation awareness]({{site.url}}{{site.baseurl}}//opensearch/cluster/#shard-allocation-awareness)
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved


## Path and HTTP methods
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is "HTTP and path methods" above. Please align with template in both instances.


```
PUT /_cluster/decommission/awareness/{awareness_attribute_name}/{awareness_attribute_value}
GET /_cluster/decommission/awareness/{awareness_attribute_name}/_status
DELETE /_cluster/decommission/awareness
```

## URL parameters

Parameter | Type | Description
:--- | :--- | :---
awareness_attribute_name | String | The name of awareness attribute, usually `zone`.
awareness_attribute_value | String | The value of the awareness attribute. For example, if you have shards allocated in two different zones, you can give each zone a value of `zone-a` or `zoneb`. The cluster decommission operation decommissions the zone listed in the method.


## Example: Decommission and recommission a zone
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Example: Decommission and recommission a zone
## Example: Decommissioning and recommissioning a zone


You can use the following sample requests to decommission and recommission a zone:
natebower marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
You can use the following sample requests to decommission and recommission a zone:
You can use the following example requests to decommission and recommission a zone:


### Request

The following sample request decommissions `zone-a`:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The following sample request decommissions `zone-a`:
The following example request decommissions `zone-a`:


```
PUT /_cluster/decommission/awareness/<zone>/<zone-a>
```

If you want to recommission a decommissioned zone, you can use the `DELETE` method.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If you want to recommission a decommissioned zone, you can use the `DELETE` method.
If you want to recommission a decommissioned zone, you can use the `DELETE` method:


```
DELETE /_cluster/decommission/awareness
```

### Response

When recommissioning or decommissioning a zone, OpenSearch responds with the following:
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

```json
{
"acknowledged": true
}
```

## Example: Get zone decommission status
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
## Example: Get zone decommission status
## Example: Getting zone decommission status


The following sample requests returns the decommission status of all zones.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The following sample requests returns the decommission status of all zones.
The following example request returns the decommission status of all zones.


### Request

```
GET /_cluster/decommission/awareness/zone/_status
```


### Response

```json
{
"zone-1": "INIT | DRAINING | IN_PROGRESS | SUCCESSFUL | FAILED"
}
```




2 changes: 1 addition & 1 deletion _api-reference/cluster-health.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: Cluster health
nav_order: 16
nav_order: 17
---

# Cluster health
Expand Down
2 changes: 1 addition & 1 deletion _api-reference/cluster-settings.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: Cluster settings
nav_order: 17
nav_order: 18
---

# Cluster settings
Expand Down
2 changes: 1 addition & 1 deletion _api-reference/count.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
layout: default
title: Count
nav_order: 20
nav_order: 21
---

# Count
Expand Down