Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mark docrep to remote migration as GA and modify settings names #7342

Merged
merged 11 commits into from
Jun 18, 2024
Merged

Mark docrep to remote migration as GA and modify settings names #7342

Changes from 7 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
d411609
Mark docrep to remote migration as GA and modify settings names
gbbafna Jun 10, 2024
337a53b
Update _tuning-your-cluster/availability-and-recovery/remote-store/mi…
gbbafna Jun 10, 2024
060bd40
Merge branch 'main' into migration-ga
gbbafna Jun 10, 2024
4d20c96
Merge branch 'main' into migration-ga
gbbafna Jun 10, 2024
5494924
Merge branch 'main' into migration-ga
gbbafna Jun 12, 2024
1affe48
Merge branch 'main' into migration-ga
vagimeli Jun 17, 2024
957da28
Update migrating-to-remote.md
vagimeli Jun 17, 2024
01b7d0a
Update _tuning-your-cluster/availability-and-recovery/remote-store/mi…
vagimeli Jun 18, 2024
779e037
Update _tuning-your-cluster/availability-and-recovery/remote-store/mi…
vagimeli Jun 18, 2024
4d4721f
Update _tuning-your-cluster/availability-and-recovery/remote-store/mi…
vagimeli Jun 18, 2024
8548c52
Merge branch 'main' into migration-ga
vagimeli Jun 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 23 additions & 21 deletions _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,10 +8,7 @@

# Migrating to remote-backed storage

This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/OpenSearch/issues/7986).
{: .warning}

Introduced 2.14
Introduced 2.15
{: .label .label-purple }

Remote-backed storage offers a new way to protect against data loss by automatically creating backups of all index transactions and sending them to remote storage. To use this feature, [segment replication]({{site.url}}{{site.baseurl}}/opensearch/segment-replication/) must be enabled.
Expand All @@ -24,9 +21,9 @@

Review [Upgrading OpenSearch]({{site.url}}{{site.baseurl}}/upgrade-opensearch/index/) for recommendations about backing up your configuration files and creating a snapshot of the cluster state and indexes before you make any changes to your OpenSearch cluster.

Before migrating to remote-backed storage, upgrade to OpenSearch 2.14.
Before migrating to remote-backed storage, upgrade to OpenSearch 2.15 or later.

As of OpenSearch 2.14, OpenSearch nodes cannot be migrated back to document replication. If you need to revert the migration, then you will need to perform a fresh installation of OpenSearch and restore the cluster from a snapshot. Take a snapshot and store it in a remote repository before beginning the upgrade procedure.
Before upgrading to OpenSearch 2.15, take a cluster snapshot and store it remotely. OpenSearch 2.15 nodes cannot revert to document replication. If migration needs to be undone, perform a fresh OpenSearch installation and restore from the remote snapshot. Securely storing the snapshot remotely allows retrieving and restoring it if issues arise during migration.
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
{: .important}

## Performing the upgrade
Expand Down Expand Up @@ -105,14 +102,14 @@
}
```

1. Set the `remote_store.compatibility_mode` setting to `mixed` to allow remote-backed storage nodes to join the cluster. Then set `migration.direction` to `remote_store`, which allocates new indexes to remote-backed data nodes. The following example updates the aforementioned setting using the Cluster Settings API:
1. Set the `cluster.remote_store.compatibility_mode` setting to `mixed` to allow remote-backed storage nodes to join the cluster. Then set `cluster.migration.direction` to `remote_store`, which allocates new indexes to remote-backed data nodes. The following example updates the aforementioned setting using the Cluster Settings API:

```json
PUT "/_cluster/settings?pretty"
{
"persistent": {
"remote_store.compatibility_mode": "mixed",
"migration.direction" : "remote_store"
"cluster.remote_store.compatibility_mode": "mixed",
"cluster.migration.direction" : "remote_store"
}
}
```
Expand All @@ -121,13 +118,15 @@
```json
{
"acknowledged" : true,
"persistent" : {
"remote_store" : {
"compatibility_mode" : "mixed",
"migration.direction" : "remote_store"
"persistent" : {
"cluster" : {
"remote_store" : {
"compatibility_mode" : "mixed",
"migration.direction" : "remote_store"
}
},
"transient" : { }
}
}
}
```

Expand Down Expand Up @@ -233,8 +232,8 @@
PUT "/_cluster/settings?pretty"
{
"persistent": {
"remote_store.compatibility_mode": strict,
"migration.direction" : none
"cluster.remote_store.compatibility_mode": null,
"cluster.migration.direction" : null
}
}
```
Expand All @@ -243,7 +242,10 @@
```json
{
"acknowledged" : true,
"persistent" : { strict },
"persistent" : {
"cluster.remote_store.compatibility_mode": null,
"cluster.migration.direction" : null
},
"transient" : { }
}
```
Expand All @@ -255,8 +257,8 @@

Use the following cluster settings to enable migration to a remote-backed cluster.

| Field | Data type | Description |
| :--- |:--- |:---|
| `remote_store.compatibility_mode` | String | When set to `strict`, only allows the creation of either non-remote or remote nodes, depending upon the initial cluster type. When set to `mixed`, allows both remote and non-remote nodes to join the cluster. Default is `strict`. |
| `migration.direction` | String | Creates new shards only on remote-backed storage nodes. Default is `None`. |
| Field | Data type | Description |
|:------------------------------------------|:--- |:---|
| `cluster.remote_store.compatibility_mode` | String | When set to `strict`, only allows the creation of either non-remote or remote nodes, depending upon the initial cluster type. When set to `mixed`, allows both remote and non-remote nodes to join the cluster. Default is `strict`. |
vagimeli marked this conversation as resolved.
Show resolved Hide resolved
| `cluster.migration.direction` | String | Creates new shards only on remote-backed storage nodes. Default is `None`. |

Check failure on line 263 in _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md

View workflow job for this annotation

GitHub Actions / style-job 

[vale] reported by reviewdog 🐶
[OpenSearch.SpacingPunctuation] There should be no space before and one space after the punctuation mark in 'nodes. Default'.

Raw Output:
{"message": "[OpenSearch.SpacingPunctuation] There should be no space before and one space after the punctuation mark in 'nodes. Default'.", "location": {"path": "_tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md", "range": {"start": {"line": 263, "column": 106}}}, "severity": "ERROR"}
vagimeli marked this conversation as resolved.
Show resolved Hide resolved

Loading