Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change remote store documentation to reflect all the config options #1245

Merged
merged 3 commits into from
Sep 20, 2022
+33 −19
Merged

Change remote store documentation to reflect all the config options #1245

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
d826793
Change remote store documentation to reflect all the config options
Sep 16, 2022
83f539f
Address review comments
Sep 19, 2022
74e5b80
Address review comments
Sep 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 33 additions & 19 deletions _opensearch/remote.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,30 +12,18 @@ Remote-backed storage is an experimental feature. Therefore, we do not recommend
Remote-backed storage offers OpenSearch users a new way to protect against data loss by automatically creating backups of all index transactions and sending them to remote storage. In order to expose this feature, segment replication must also be enabled. See [Segment replication]({{site.url}}{{site.baseurl}}/opensearch/segment-replication/) for additional information.


## Enable the feature
## Enabling the feature flag
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's switch to "Enable the feature flag".


To create new indexes with remote-backed storage enabled, you must first enable these features by adding the correct properties to `run.gradle` before building OpenSearch. See the [developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md) for information about to use how Gradle to build OpenSearch.

Add the following properties to `run.gradle` to enable the feature:

```bash
testClusters {
runTask {
testDistribution = 'archive'
if (numZones > 1) numberOfZones = numZones
if (numNodes > 1) numberOfNodes = numNodes
systemProperty 'opensearch.experimental.feature.replication_type.enabled', 'true'
systemProperty 'opensearch.experimental.feature.remote_store.enabled', 'true'
}
}
```
There are several methods for enabling remote store feature, depending on the install type. You will also need to enable `remote_store` property when creating the index.

Segment replication must also be enabled to use remote-backed storage.
{: .note}

After building OpenSearch with these properties, you must enable the feature for all nodes in your cluster. This can be done by modifying a `config/jvm.options`, or by defining `OPENSEARCH_JAVA_OPS` from the command line.
### Enable on a node using a tarball install

The flag is toggled using a new jvm parameter that is set either in `OPENSEARCH_JAVA_OPTS` or in config/jvm.options.

### Option 1: Modify jvm.options
1. Option 1: Modify jvm.options
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep these as H3 headers, just for consistency with the gradle option.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These come under H3 header: Enable on a node using a tarball install so changed it to H4 header. Let me know if that is fine.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works for me.


Add the following lines to `config/jvm.options` before starting the OpenSearch process to enable the feature and its dependency:

Expand All @@ -50,7 +38,7 @@ Run OpenSearch
./bin/opensearch
```

### Option 2: Enable from an environment variable
1. Option 2: Enable from an environment variable
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here, let's keep as an h3 heading.


As an alternative to directly modifying `config/jvm.options`, you can define the properties by using an environment variable. This can be done in a single command when you start OpenSearch or by defining the variable with `export`.

Expand All @@ -67,6 +55,32 @@ export OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type.
./bin/opensearch
```

### For developers using Gradle, enable feature flag by updating run.gradle
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's rename this header to "Enable for OpenSearch development".


To create new indexes with remote-backed storage enabled, you must first enable these features by adding the correct properties to `run.gradle` before building OpenSearch. See the [developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md) for information about to use how Gradle to build OpenSearch.

Add the following properties to `run.gradle` to enable the feature:

```bash
testClusters {
runTask {
testDistribution = 'archive'
if (numZones > 1) numberOfZones = numZones
if (numNodes > 1) numberOfNodes = numNodes
systemProperty 'opensearch.experimental.feature.replication_type.enabled', 'true'
systemProperty 'opensearch.experimental.feature.remote_store.enabled', 'true'
}
}
```

### Enable with Docker containers
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's put this above the gradle section with the other two JVM options.


If you're running Docker, add the following line to docker-compose.yml underneath the `opensearch-node` and `environment` section:

````json
OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type.enabled=true -Dopensearch.experimental.feature.remote_store.enabled=true"
````

## Register a remote repository

Now that your deployment is running with the feature flags enabled, the next step is to register a remote repository where backups will be stored. See [Register repository]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#register-repository) for more information.
Expand Down