Skip to content

Commit

Permalink
Add migrations section (#8873) (#8889)
Browse files Browse the repository at this point in the history
* Add migrations section (#8873)

* Add migrations section.

Signed-off-by: Archer <[email protected]>

* Add back upgrade pages. Make them less migration focused.

Signed-off-by: Archer <[email protected]>

* Add back in upgrade pages.

Signed-off-by: Archer <[email protected]>

* Remove upgrade section. Add redirects.

Signed-off-by: Archer <[email protected]>

* Update migration-console-commands-references.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Implement doc review.

Signed-off-by: Archer <[email protected]>

* Fix getting started headers.

Signed-off-by: Archer <[email protected]>

* Fix parent child relationship for Deploying migration assistant

Signed-off-by: Archer <[email protected]>

* Update _migration-assistant/migration-phases/assessing-your-cluster-for-migration.md

Signed-off-by: Naarcha-AWS <[email protected]>

* Additional feedback

Signed-off-by: Archer <[email protected]>

* Add permalink for migration console

Signed-off-by: Archer <[email protected]>

* Add redirects for index pages.

Signed-off-by: Archer <[email protected]>

* Final edits.

Signed-off-by: Archer <[email protected]>

* Fix headings.

Signed-off-by: Archer <[email protected]>

* Add copy buttons.

Signed-off-by: Archer <[email protected]>

* Last copy button.

Signed-off-by: Archer <[email protected]>

---------

Signed-off-by: Archer <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
(cherry picked from commit 8664051)

* Fix merge conflicts

Signed-off-by: Archer <[email protected]>

* Delete _migrations directory

Signed-off-by: Naarcha-AWS <[email protected]>

---------

Signed-off-by: Archer <[email protected]>
Signed-off-by: Naarcha-AWS <[email protected]>
  • Loading branch information
Naarcha-AWS authored Dec 5, 2024
1 parent ac76760 commit 5b7e5ff
Show file tree
Hide file tree
Showing 29 changed files with 218 additions and 717 deletions.
23 changes: 15 additions & 8 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,6 @@ collections:
install-and-configure:
permalink: /:collection/:path/
output: true
upgrade-to:
permalink: /:collection/:path/
output: true
im-plugin:
permalink: /:collection/:path/
output: true
Expand Down Expand Up @@ -94,6 +91,9 @@ collections:
data-prepper:
permalink: /:collection/:path/
output: true
migration-assistant:
permalink: /:collection/:path/
output: true
tools:
permalink: /:collection/:path/
output: true
Expand Down Expand Up @@ -137,11 +137,6 @@ opensearch_collection:
install-and-configure:
name: Install and upgrade
nav_fold: true
upgrade-to:
name: Migrate to OpenSearch
# nav_exclude: true
nav_fold: true
# search_exclude: true
im-plugin:
name: Managing Indexes
nav_fold: true
Expand Down Expand Up @@ -213,6 +208,12 @@ clients_collection:
name: Clients
nav_fold: true

migration_assistant_collection:
collections:
migration-assistant:
name: Migration Assistant
nav_fold: true

benchmark_collection:
collections:
benchmark:
Expand Down Expand Up @@ -252,6 +253,12 @@ defaults:
values:
section: "benchmark"
section-name: "Benchmark"
-
scope:
path: "_migration-assistant"
values:
section: "migration-assistant"
section-name: "Migration Assistant"

# Enable or disable the site search
# By default, just-the-docs enables its JSON file-based search. We also have an OpenSearch-driven search functionality.
Expand Down
2 changes: 2 additions & 0 deletions _data/top_nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ items:
url: /docs/latest/clients/
- label: Benchmark
url: /docs/latest/benchmark/
- label: Migration Assistant
url: /docs/latest/migration-assistant/
- label: Platform
url: /platform/index.html
children:
Expand Down
8 changes: 7 additions & 1 deletion _includes/cards.html
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,14 @@
<p class="description">Measure performance metrics for your OpenSearch cluster</p>
<p class="last-link">Documentation &#x2192;</p>
</div>

<div class="card">
<a href="{{site.url}}/docs/latest/migration-assistant/" class='card-link'></a>
<p class="heading">Migration Assistant</p>
<p class="description">Migrate to OpenSearch from other platforms</p>
<p class="last-link">Documentation &#x2192;</p>
</div>
</div>

</div>


2 changes: 2 additions & 0 deletions _layouts/default.html
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@
{% assign section = site.clients_collection.collections %}
{% elsif page.section == "benchmark" %}
{% assign section = site.benchmark_collection.collections %}
{% elsif page.section == "migration-assistant" %}
{% assign section = site.migration_assistant_collection.collections %}
{% endif %}

{% if section %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: Configuration options
nav_order: 15
parent: Deploying migration assistant
parent: Deploying Migration Assistant
---

# Configuration options
Expand Down Expand Up @@ -61,6 +61,7 @@ The following CDK performs a backfill migrations using RFS:
}
}
```
{% include copy.html %}

Performing an RFS backfill migration requires an existing snapshot.

Expand Down Expand Up @@ -104,6 +105,7 @@ The following sample CDK performs a live capture migration with C&R:
}
}
```
{% include copy.html %}

Performing a live capture migration requires that a Capture Proxy be configured to capture incoming traffic and send it to the target cluster using the Traffic Replayer service. For arguments available in `captureProxyExtraArgs`, refer to the `@Parameter` fields [here](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficCaptureProxyServer/src/main/java/org/opensearch/migrations/trafficcapture/proxyserver/CaptureProxy.java). For `trafficReplayerExtraArgs`, refer to the `@Parameter` fields [here](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java). At a minimum, no extra arguments may be needed.

Expand All @@ -125,17 +127,18 @@ Both the source and target cluster can use no authentication, authentication lim

### No authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
"auth": {"type": "none"}
}
```
{% include copy.html %}

### Basic authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
Expand All @@ -146,10 +149,11 @@ Both the source and target cluster can use no authentication, authentication lim
}
}
```
{% include copy.html %}

### Signature Version 4 authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
Expand All @@ -160,11 +164,12 @@ Both the source and target cluster can use no authentication, authentication lim
}
}
```
{% include copy.html %}

The `serviceSigningName` can be `es` for an Elasticsearch or OpenSearch domain, or `aoss` for an OpenSearch Serverless collection.

All of these authentication options apply to both source and target clusters.

## Network configuration

The migration tooling expects the source cluster, target cluster, and migration resources to exist in the same VPC. If this is not the case, manual networking setup outside of this documentation is likely required.
The migration tooling expects the source cluster, target cluster, and migration resources to exist in the same VPC. If this is not the case, manual networking setup outside of this documentation is likely required.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: IAM and security groups for existing clusters
nav_order: 20
parent: Deploying migration assistant
parent: Deploying Migration Assistant
---

# IAM and security groups for existing clusters
Expand Down Expand Up @@ -33,7 +33,7 @@ For an OpenSearch Serverless Collection, you will need to configure both network
The Collection should have a network policy that uses the `VPC` access type. This requires creating a VPC endpoint on the VPC used for the solution. The VPC endpoint should be configured for the private subnets of the VPC and should attach the `osClusterAccessSG` security group.

2. **Data Access Policy Configuration**:
The data access policy should grant permission to perform all [index operations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-data-access.html#serverless-data-supported-permissions) (`aoss:*`) for all indexes in the Collection. The IAM task roles of the applicable Migration services (Traffic Replayer, Migration Console, Reindex-from-Snapshot) should be used as the principals for this data access policy.
The data access policy should grant permission to perform all [index operations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-data-access.html#serverless-data-supported-permissions) (`aoss:*`) for all indexes in the Collection. The IAM task roles of the applicable Migration services (Traffic Replayer, migration console, `Reindex-from-Snapshot`) should be used as the principals for this data access policy.

## Capture Proxy on Coordinator Nodes of Source Cluster

Expand Down Expand Up @@ -68,7 +68,3 @@ Before [setting up Capture Proxy instances](https://github.com/opensearch-projec
]
}
```

## Related Links

- [OpenSearch traffic capture setup]
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
---
layout: default
title: Deploying Migration Assistant
nav_order: 10
nav_order: 15
has_children: true
permalink: /deploying-migration-assistant/
redirect-from:
- /deploying-migration-assistant/index/
---

# Deploying Migration Assistant
Expand Down
Loading

0 comments on commit 5b7e5ff

Please sign in to comment.