From d826793853157bbddb94fffbae61ebc3913d0133 Mon Sep 17 00:00:00 2001 From: Sachin Kale Date: Fri, 16 Sep 2022 11:57:04 +0530 Subject: [PATCH 1/3] Change remote store documentation to reflect all the config options Signed-off-by: Sachin Kale --- _opensearch/remote.md | 52 +++++++++++++++++++++++++++---------------- 1 file changed, 33 insertions(+), 19 deletions(-) diff --git a/_opensearch/remote.md b/_opensearch/remote.md index 6a7c4d4dd8..7c566742e3 100644 --- a/_opensearch/remote.md +++ b/_opensearch/remote.md @@ -12,30 +12,18 @@ Remote-backed storage is an experimental feature. Therefore, we do not recommend Remote-backed storage offers OpenSearch users a new way to protect against data loss by automatically creating backups of all index transactions and sending them to remote storage. In order to expose this feature, segment replication must also be enabled. See [Segment replication]({{site.url}}{{site.baseurl}}/opensearch/segment-replication/) for additional information. -## Enable the feature +## Enabling the feature flag -To create new indexes with remote-backed storage enabled, you must first enable these features by adding the correct properties to `run.gradle` before building OpenSearch. See the [developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md) for information about to use how Gradle to build OpenSearch. - -Add the following properties to `run.gradle` to enable the feature: - -```bash -testClusters { - runTask { - testDistribution = 'archive' - if (numZones > 1) numberOfZones = numZones - if (numNodes > 1) numberOfNodes = numNodes - systemProperty 'opensearch.experimental.feature.replication_type.enabled', 'true' - systemProperty 'opensearch.experimental.feature.remote_store.enabled', 'true' - } -} -``` +There are several methods for enabling remote store feature, depending on the install type. You will also need to enable `remote_store` property when creating the index. Segment replication must also be enabled to use remote-backed storage. {: .note} -After building OpenSearch with these properties, you must enable the feature for all nodes in your cluster. This can be done by modifying a `config/jvm.options`, or by defining `OPENSEARCH_JAVA_OPS` from the command line. +### Enable on a node using a tarball install + +The flag is toggled using a new jvm parameter that is set either in `OPENSEARCH_JAVA_OPTS` or in config/jvm.options. -### Option 1: Modify jvm.options +1. Option 1: Modify jvm.options Add the following lines to `config/jvm.options` before starting the OpenSearch process to enable the feature and its dependency: @@ -50,7 +38,7 @@ Run OpenSearch ./bin/opensearch ``` -### Option 2: Enable from an environment variable +1. Option 2: Enable from an environment variable As an alternative to directly modifying `config/jvm.options`, you can define the properties by using an environment variable. This can be done in a single command when you start OpenSearch or by defining the variable with `export`. @@ -67,6 +55,32 @@ export OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type. ./bin/opensearch ``` +### For developers using Gradle, enable feature flag by updating run.gradle + +To create new indexes with remote-backed storage enabled, you must first enable these features by adding the correct properties to `run.gradle` before building OpenSearch. See the [developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md) for information about to use how Gradle to build OpenSearch. + +Add the following properties to `run.gradle` to enable the feature: + +```bash +testClusters { + runTask { + testDistribution = 'archive' + if (numZones > 1) numberOfZones = numZones + if (numNodes > 1) numberOfNodes = numNodes + systemProperty 'opensearch.experimental.feature.replication_type.enabled', 'true' + systemProperty 'opensearch.experimental.feature.remote_store.enabled', 'true' + } +} +``` + +### Enable with Docker containers + +If you're running Docker, add the following line to docker-compose.yml underneath the `opensearch-node` and `environment` section: + +````json +OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type.enabled=true -Dopensearch.experimental.feature.remote_store.enabled=true" +```` + ## Register a remote repository Now that your deployment is running with the feature flags enabled, the next step is to register a remote repository where backups will be stored. See [Register repository]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#register-repository) for more information. From 83f539fcd82869377d9192b1e3507c4723389422 Mon Sep 17 00:00:00 2001 From: Sachin Kale Date: Mon, 19 Sep 2022 10:32:48 +0530 Subject: [PATCH 2/3] Address review comments Signed-off-by: Sachin Kale --- _opensearch/remote.md | 22 +++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/_opensearch/remote.md b/_opensearch/remote.md index 7c566742e3..2128cd9b95 100644 --- a/_opensearch/remote.md +++ b/_opensearch/remote.md @@ -12,7 +12,7 @@ Remote-backed storage is an experimental feature. Therefore, we do not recommend Remote-backed storage offers OpenSearch users a new way to protect against data loss by automatically creating backups of all index transactions and sending them to remote storage. In order to expose this feature, segment replication must also be enabled. See [Segment replication]({{site.url}}{{site.baseurl}}/opensearch/segment-replication/) for additional information. -## Enabling the feature flag +## Enable the feature flag There are several methods for enabling remote store feature, depending on the install type. You will also need to enable `remote_store` property when creating the index. @@ -23,7 +23,7 @@ Segment replication must also be enabled to use remote-backed storage. The flag is toggled using a new jvm parameter that is set either in `OPENSEARCH_JAVA_OPTS` or in config/jvm.options. -1. Option 1: Modify jvm.options +#### Option 1: Modify jvm.options Add the following lines to `config/jvm.options` before starting the OpenSearch process to enable the feature and its dependency: @@ -38,7 +38,7 @@ Run OpenSearch ./bin/opensearch ``` -1. Option 2: Enable from an environment variable +#### Option 2: Enable from an environment variable As an alternative to directly modifying `config/jvm.options`, you can define the properties by using an environment variable. This can be done in a single command when you start OpenSearch or by defining the variable with `export`. @@ -55,6 +55,14 @@ export OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type. ./bin/opensearch ``` +### Enable with Docker containers + +If you're running Docker, add the following line to docker-compose.yml underneath the `opensearch-node` and `environment` section: + +````json +OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type.enabled=true -Dopensearch.experimental.feature.remote_store.enabled=true" +```` + ### For developers using Gradle, enable feature flag by updating run.gradle To create new indexes with remote-backed storage enabled, you must first enable these features by adding the correct properties to `run.gradle` before building OpenSearch. See the [developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md) for information about to use how Gradle to build OpenSearch. @@ -73,14 +81,6 @@ testClusters { } ``` -### Enable with Docker containers - -If you're running Docker, add the following line to docker-compose.yml underneath the `opensearch-node` and `environment` section: - -````json -OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type.enabled=true -Dopensearch.experimental.feature.remote_store.enabled=true" -```` - ## Register a remote repository Now that your deployment is running with the feature flags enabled, the next step is to register a remote repository where backups will be stored. See [Register repository]({{site.url}}{{site.baseurl}}/opensearch/snapshots/snapshot-restore#register-repository) for more information. From 74e5b80c81ed4178978ab18c6f31cb7462cc3053 Mon Sep 17 00:00:00 2001 From: Sachin Kale Date: Mon, 19 Sep 2022 10:34:14 +0530 Subject: [PATCH 3/3] Address review comments Signed-off-by: Sachin Kale --- _opensearch/remote.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_opensearch/remote.md b/_opensearch/remote.md index 2128cd9b95..280440794f 100644 --- a/_opensearch/remote.md +++ b/_opensearch/remote.md @@ -63,7 +63,7 @@ If you're running Docker, add the following line to docker-compose.yml underneat OPENSEARCH_JAVA_OPTS="-Dopensearch.experimental.feature.replication_type.enabled=true -Dopensearch.experimental.feature.remote_store.enabled=true" ```` -### For developers using Gradle, enable feature flag by updating run.gradle +### Enable for OpenSearch development To create new indexes with remote-backed storage enabled, you must first enable these features by adding the correct properties to `run.gradle` before building OpenSearch. See the [developer guide](https://github.com/opensearch-project/OpenSearch/blob/main/DEVELOPER_GUIDE.md) for information about to use how Gradle to build OpenSearch.