Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add migrations section (#8873) #8889

Merged
merged 3 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 15 additions & 8 deletions _config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,9 +31,6 @@ collections:
install-and-configure:
permalink: /:collection/:path/
output: true
upgrade-to:
permalink: /:collection/:path/
output: true
im-plugin:
permalink: /:collection/:path/
output: true
Expand Down Expand Up @@ -94,6 +91,9 @@ collections:
data-prepper:
permalink: /:collection/:path/
output: true
migration-assistant:
permalink: /:collection/:path/
output: true
tools:
permalink: /:collection/:path/
output: true
Expand Down Expand Up @@ -137,11 +137,6 @@ opensearch_collection:
install-and-configure:
name: Install and upgrade
nav_fold: true
upgrade-to:
name: Migrate to OpenSearch
# nav_exclude: true
nav_fold: true
# search_exclude: true
im-plugin:
name: Managing Indexes
nav_fold: true
Expand Down Expand Up @@ -213,6 +208,12 @@ clients_collection:
name: Clients
nav_fold: true

migration_assistant_collection:
collections:
migration-assistant:
name: Migration Assistant
nav_fold: true

benchmark_collection:
collections:
benchmark:
Expand Down Expand Up @@ -252,6 +253,12 @@ defaults:
values:
section: "benchmark"
section-name: "Benchmark"
-
scope:
path: "_migration-assistant"
values:
section: "migration-assistant"
section-name: "Migration Assistant"

# Enable or disable the site search
# By default, just-the-docs enables its JSON file-based search. We also have an OpenSearch-driven search functionality.
Expand Down
2 changes: 2 additions & 0 deletions _data/top_nav.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,8 @@ items:
url: /docs/latest/clients/
- label: Benchmark
url: /docs/latest/benchmark/
- label: Migration Assistant
url: /docs/latest/migration-assistant/
- label: Platform
url: /platform/index.html
children:
Expand Down
8 changes: 7 additions & 1 deletion _includes/cards.html
Original file line number Diff line number Diff line change
Expand Up @@ -30,8 +30,14 @@
<p class="description">Measure performance metrics for your OpenSearch cluster</p>
<p class="last-link">Documentation &#x2192;</p>
</div>

<div class="card">
<a href="{{site.url}}/docs/latest/migration-assistant/" class='card-link'></a>
<p class="heading">Migration Assistant</p>
<p class="description">Migrate to OpenSearch from other platforms</p>
<p class="last-link">Documentation &#x2192;</p>
</div>
</div>

</div>


2 changes: 2 additions & 0 deletions _layouts/default.html
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,8 @@
{% assign section = site.clients_collection.collections %}
{% elsif page.section == "benchmark" %}
{% assign section = site.benchmark_collection.collections %}
{% elsif page.section == "migration-assistant" %}
{% assign section = site.migration_assistant_collection.collections %}
{% endif %}

{% if section %}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: Configuration options
nav_order: 15
parent: Deploying migration assistant
parent: Deploying Migration Assistant
---

# Configuration options
Expand Down Expand Up @@ -61,6 +61,7 @@ The following CDK performs a backfill migrations using RFS:
}
}
```
{% include copy.html %}

Performing an RFS backfill migration requires an existing snapshot.

Expand Down Expand Up @@ -104,6 +105,7 @@ The following sample CDK performs a live capture migration with C&R:
}
}
```
{% include copy.html %}

Performing a live capture migration requires that a Capture Proxy be configured to capture incoming traffic and send it to the target cluster using the Traffic Replayer service. For arguments available in `captureProxyExtraArgs`, refer to the `@Parameter` fields [here](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficCaptureProxyServer/src/main/java/org/opensearch/migrations/trafficcapture/proxyserver/CaptureProxy.java). For `trafficReplayerExtraArgs`, refer to the `@Parameter` fields [here](https://github.com/opensearch-project/opensearch-migrations/blob/main/TrafficCapture/trafficReplayer/src/main/java/org/opensearch/migrations/replay/TrafficReplayer.java). At a minimum, no extra arguments may be needed.

Expand All @@ -125,17 +127,18 @@ Both the source and target cluster can use no authentication, authentication lim

### No authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
"auth": {"type": "none"}
}
```
{% include copy.html %}

### Basic authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
Expand All @@ -146,10 +149,11 @@ Both the source and target cluster can use no authentication, authentication lim
}
}
```
{% include copy.html %}

### Signature Version 4 authentication

```
```json
"sourceCluster": {
"endpoint": <SOURCE_CLUSTER_ENDPOINT>,
"version": "ES 7.10",
Expand All @@ -160,11 +164,12 @@ Both the source and target cluster can use no authentication, authentication lim
}
}
```
{% include copy.html %}

The `serviceSigningName` can be `es` for an Elasticsearch or OpenSearch domain, or `aoss` for an OpenSearch Serverless collection.

All of these authentication options apply to both source and target clusters.

## Network configuration

The migration tooling expects the source cluster, target cluster, and migration resources to exist in the same VPC. If this is not the case, manual networking setup outside of this documentation is likely required.
The migration tooling expects the source cluster, target cluster, and migration resources to exist in the same VPC. If this is not the case, manual networking setup outside of this documentation is likely required.
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
layout: default
title: IAM and security groups for existing clusters
nav_order: 20
parent: Deploying migration assistant
parent: Deploying Migration Assistant
---

# IAM and security groups for existing clusters
Expand Down Expand Up @@ -33,7 +33,7 @@
The Collection should have a network policy that uses the `VPC` access type. This requires creating a VPC endpoint on the VPC used for the solution. The VPC endpoint should be configured for the private subnets of the VPC and should attach the `osClusterAccessSG` security group.

2. **Data Access Policy Configuration**:
The data access policy should grant permission to perform all [index operations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-data-access.html#serverless-data-supported-permissions) (`aoss:*`) for all indexes in the Collection. The IAM task roles of the applicable Migration services (Traffic Replayer, Migration Console, Reindex-from-Snapshot) should be used as the principals for this data access policy.
The data access policy should grant permission to perform all [index operations](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless-data-access.html#serverless-data-supported-permissions) (`aoss:*`) for all indexes in the Collection. The IAM task roles of the applicable Migration services (Traffic Replayer, migration console, `Reindex-from-Snapshot`) should be used as the principals for this data access policy.

Check failure on line 36 in _migration-assistant/deploying-migration-assistant/iam-and-security-groups-for-existing-clusters.md

View workflow job for this annotation

GitHub Actions / vale

[vale] _migration-assistant/deploying-migration-assistant/iam-and-security-groups-for-existing-clusters.md#L36

[OpenSearch.Spelling] Error: Replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.
Raw output
{"message": "[OpenSearch.Spelling] Error: Replayer. If you are referencing a setting, variable, format, function, or repository, surround it with tic marks.", "location": {"path": "_migration-assistant/deploying-migration-assistant/iam-and-security-groups-for-existing-clusters.md", "range": {"start": {"line": 36, "column": 332}}}, "severity": "ERROR"}

## Capture Proxy on Coordinator Nodes of Source Cluster

Expand Down Expand Up @@ -68,7 +68,3 @@
]
}
```

## Related Links

- [OpenSearch traffic capture setup]
Original file line number Diff line number Diff line change
@@ -1,7 +1,11 @@
---
layout: default
title: Deploying Migration Assistant
nav_order: 10
nav_order: 15
has_children: true
permalink: /deploying-migration-assistant/
redirect-from:
- /deploying-migration-assistant/index/
---

# Deploying Migration Assistant
Expand Down
Loading
Loading