Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Recovery API #7653

Merged
merged 14 commits into from
Jul 24, 2024
291 changes: 291 additions & 0 deletions _api-reference/index-apis/recover.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,291 @@
---
layout: default
title: Recover index
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
parent: Index APIs
nav_order: 62
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
---

# Recover an index
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Introduced 1.0
{: .label .label-purple }

The Recover index API gives information about any completed or ongoing shard recoveries for one or more indexes. If a data stream is listed, the API returns information about that data stream's backing index.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

Shard recovery is the process of starting a shard copy for restoring a primary shard from a snapshot or syncing a replica shard. Once a shard recovery is complete, the recovered shard becomes available for searching and indexing operations.
hdhalter marked this conversation as resolved.
Show resolved Hide resolved

Shard recovery occurs automatically in the following scenarios:

- Node startup, known as a local store recovery.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
- Replication of a primary shard.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
- Relocation of a shard to a different node within the same cluster.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
- Restoration of a snapshot.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
- Clone, shrink, or split operations.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

The Recover Index API provides information only about completed recoveries for shard copies that currently exist in the cluster. It reports only the most recent recovery for each shard copy and does not include historical information about previous recoveries or information about recoveries of shard copies that no longer exist. Consequently, if a shard copy completes a recovery and is subsequently relocated to a different node, the information about the original recovery will not be displayed in the Recover Index API.


## HTTP and Path methods

Check failure on line 27 in _api-reference/index-apis/recover.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _api-reference/index-apis/recover.md#L27

[OpenSearch.HeadingCapitalization] 'HTTP and Path methods' is a heading and should be in sentence case.
Raw output
{"message": "[OpenSearch.HeadingCapitalization] 'HTTP and Path methods' is a heading and should be in sentence case.", "location": {"path": "_api-reference/index-apis/recover.md", "range": {"start": {"line": 27, "column": 4}}}, "severity": "ERROR"}
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

```
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
GET /_recovery
GET /<index-name>/recovery/
```

## Path parameters

Parameter | Type | Description
:--- | :---
`index-name` | String | The name(s) of the index(es) you want recovery information about.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved


## Query parameters

All of the following query parameters are optional.

Parameter | Type | Description
:--- | :--- | :---
`active_only` | Boolean | When `true`, the response only includes active shard recoveries. Default is `false`.
`detailed` | Boolean | When `true`, gives detailed information about shard recoveries. Default is `false`.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`index` | String | A comma-separated list or wildcard expression of index names used to limit the request.

## Response body
hdhalter marked this conversation as resolved.
Show resolved Hide resolved

The API responds with the following information about the recovery shard:
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

Parameter | Type | Description
:--- | :--- | :---
`id` | Integer | The ID of the shard.
`type` | String | The recovery source for the shard. Returned values include: <br> - `EMPTY_STORE`: An empty store. Indicates a new primary shard or the forced allocation of an empty primary shard using the cluster reroute API. <br> - `EXISTING_STORE`: The store of an existing primary shard. Indicates recovery is related to node startup or the allocation of an existing primary shard. <br> - `LOCAL_SHARDS`: Shards of another index on the same node. Indicates recovery is related to a clone, shrink, or split operation. <br> - `PEER`: A primary shard on another node. Indicates recovery is related to shard replication. <br> - `SNAPSHOT`: A snapshot. Indicates recovery is related to a snapshot restore operation.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`STAGE` | String | The recovery stage. Returned values can include: <br> - `INIT`: Recovery has not started. <br> - `INDEX`: Reading index metadata and copying bytes from source to destination. <br> - `VERIFY_INDEX`: Verifying the integrity of the index. <br> - `TRANSLOG`: Replaying transaction log. <br> - `FINALIZE`: Cleanup. <br> - `DONE`: Complete.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`primary` | Boolean | When `true`, the shard is a primary shard.
`start_time` | String | The timestamp for when the recovery started.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`stop_time` | String | The timestamp for when the recovery finished.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`total_time_in_millis` | String | The total time to recover shard in milliseconds.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`source` | Object | The recovery source. This can include a repository description if recovery is from a snapshot or a description of source node.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`target` | Object | The destination node.
`index` | Object | The statistics about physical index recovery.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`translog` | Object | The statistics about translog recovery.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
`start` | Object | The statistics about time to open and start the index.
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

## Example requests

natebower marked this conversation as resolved.
Show resolved Hide resolved
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
The following examples show how the Recover Index API works.

### Recover information from several or all indexes

The following example request returns recovery information about several indexes in a [human-readable format](https://opensearch.org/docs/latest/api-reference/common-parameters/#human-readable-output):

```
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
GET index1,index2/_recovery?human
```
{% include copy-curl.html %}

The following example returns recovery information about all indexes in a human-readable format:
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved

```
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
GET /_recovery?human
```
{% include copy-curl.html %}

### Recover detailed information

The following example recover detailed recovery information:

```
Naarcha-AWS marked this conversation as resolved.
Show resolved Hide resolved
GET _recovery?human&detailed=true
```
{% include copy-curl.html %}

## Example response

The following response returns detailed recovery information about an index named `shakespeare`:

```json
{
"shakespeare": {
"shards": [
{
"id": 0,
"type": "EXISTING_STORE",
"stage": "DONE",
"primary": true,
"start_time": "2024-07-01T18:06:47.415Z",
"start_time_in_millis": 1719857207415,
"stop_time": "2024-07-01T18:06:47.538Z",
"stop_time_in_millis": 1719857207538,
"total_time": "123ms",
"total_time_in_millis": 123,
"source": {
"bootstrap_new_history_uuid": false
},
"target": {
"id": "uerS7REgRQCbBF3ImY8wOQ",
"host": "172.18.0.3",
"transport_address": "172.18.0.3:9300",
"ip": "172.18.0.3",
"name": "opensearch-node2"
},
"index": {
"size": {
"total": "17.8mb",
"total_in_bytes": 18708764,
"reused": "17.8mb",
"reused_in_bytes": 18708764,
"recovered": "0b",
"recovered_in_bytes": 0,
"percent": "100.0%"
},
"files": {
"total": 7,
"reused": 7,
"recovered": 0,
"percent": "100.0%",
"details": [
{
"name": "_1.cfs",
"length": "9.8mb",
"length_in_bytes": 10325945,
"reused": true,
"recovered": "0b",
"recovered_in_bytes": 0
},
{
"name": "_0.cfe",
"length": "479b",
"length_in_bytes": 479,
"reused": true,
"recovered": "0b",
"recovered_in_bytes": 0
},
{
"name": "_0.si",
"length": "333b",
"length_in_bytes": 333,
"reused": true,
"recovered": "0b",
"recovered_in_bytes": 0
},
{
"name": "_1.cfe",
"length": "479b",
"length_in_bytes": 479,
"reused": true,
"recovered": "0b",
"recovered_in_bytes": 0
},
{
"name": "_1.si",
"length": "333b",
"length_in_bytes": 333,
"reused": true,
"recovered": "0b",
"recovered_in_bytes": 0
},
{
"name": "_0.cfs",
"length": "7.9mb",
"length_in_bytes": 8380790,
"reused": true,
"recovered": "0b",
"recovered_in_bytes": 0
},
{
"name": "segments_3",
"length": "405b",
"length_in_bytes": 405,
"reused": true,
"recovered": "0b",
"recovered_in_bytes": 0
}
]
},
"total_time": "6ms",
"total_time_in_millis": 6,
"source_throttle_time": "-1",
"source_throttle_time_in_millis": 0,
"target_throttle_time": "-1",
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": 0,
"total_time": "113ms",
"total_time_in_millis": 113
},
"verify_index": {
"check_index_time": "0s",
"check_index_time_in_millis": 0,
"total_time": "0s",
"total_time_in_millis": 0
}
},
{
"id": 0,
"type": "PEER",
"stage": "DONE",
"primary": false,
"start_time": "2024-07-01T18:06:47.693Z",
"start_time_in_millis": 1719857207693,
"stop_time": "2024-07-01T18:06:47.744Z",
"stop_time_in_millis": 1719857207744,
"total_time": "50ms",
"total_time_in_millis": 50,
"source": {
"id": "uerS7REgRQCbBF3ImY8wOQ",
"host": "172.18.0.3",
"transport_address": "172.18.0.3:9300",
"ip": "172.18.0.3",
"name": "opensearch-node2"
},
"target": {
"id": "HFYKietmTO6Ud9COgP0k9Q",
"host": "172.18.0.2",
"transport_address": "172.18.0.2:9300",
"ip": "172.18.0.2",
"name": "opensearch-node1"
},
"index": {
"size": {
"total": "0b",
"total_in_bytes": 0,
"reused": "0b",
"reused_in_bytes": 0,
"recovered": "0b",
"recovered_in_bytes": 0,
"percent": "0.0%"
},
"files": {
"total": 0,
"reused": 0,
"recovered": 0,
"percent": "0.0%",
"details": []
},
"total_time": "1ms",
"total_time_in_millis": 1,
"source_throttle_time": "-1",
"source_throttle_time_in_millis": 0,
"target_throttle_time": "-1",
"target_throttle_time_in_millis": 0
},
"translog": {
"recovered": 0,
"total": 0,
"percent": "100.0%",
"total_on_start": -1,
"total_time": "42ms",
"total_time_in_millis": 42
},
"verify_index": {
"check_index_time": "0s",
"check_index_time_in_millis": 0,
"total_time": "0s",
"total_time_in_millis": 0
}
}
]
}
}
```
Loading