Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Whitepaper on migration #23468

Merged
merged 11 commits into from
Aug 12, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
86 changes: 29 additions & 57 deletions docs/content/preview/manage/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,65 +13,37 @@ menu:
parent: launch-and-manage
type: indexpage
---
<div class="row">
<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="backup-restore/">
<div class="head">
<img class="icon" src="/images/section_icons/manage/backup.png" aria-hidden="true" />
<div class="title">Back up and restore</div>
</div>
<div class="body">
Back up and restore data in YugabyteDB.
</div>
</a>
</div>

<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="data-migration/">
<div class="head">
<img class="icon" src="/images/section_icons/index/deploy.png" aria-hidden="true" />
<div class="title">Migrate data</div>
</div>
<div class="body">
Bulk data export and import in the context of YugabyteDB.
</div>
</a>
</div>
{{<index/block>}}

<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="./change-cluster-config/">
<div class="head">
<img class="icon" src="/images/section_icons/deploy/enterprise/administer.png" aria-hidden="true" />
<div class="title">Change cluster configuration</div>
</div>
<div class="body">
Move universe data to a new set of machines.
</div>
</a>
</div>
{{<index/item
title="Back up and restore"
body="Back up and restore data in YugabyteDB."
href="backup-restore/"
icon="/images/section_icons/manage/backup.png">}}

<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="./upgrade-deployment/">
<div class="head">
<img class="icon" src="/images/section_icons/deploy/enterprise/administer.png" aria-hidden="true" />
<div class="title">Upgrade a deployment</div>
</div>
<div class="body">
Upgrade the YB software deployment.
</div>
</a>
</div>
{{<index/item
title="Migrate data"
body="Bulk data export and import in the context of YugabyteDB."
href="data-migration/"
icon="/images/section_icons/index/deploy.png">}}

<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="diagnostics-reporting/">
<div class="head">
<img class="icon" src="/images/section_icons/manage/diagnostics.png" aria-hidden="true" />
<div class="title">Diagnostics reporting</div>
</div>
<div class="body">
Configure collection and reporting of diagnostics data.
</div>
</a>
</div>
{{<index/item
title="Change cluster configuration"
body="Move universe data to a new set of machines."
href="change-cluster-config/"
icon="/images/section_icons/deploy/enterprise/administer.png">}}

</div>
{{<index/item
title="Upgrade a deployment"
body="Upgrade the YugabyteDB software."
href="upgrade-deployment/"
icon="/images/section_icons/deploy/enterprise/administer.png">}}

{{<index/item
title="Diagnostics reporting"
body="Configure collection and reporting of diagnostics data."
href="diagnostics-reporting/"
icon="/images/section_icons/manage/diagnostics.png">}}

{{</index/block>}}
59 changes: 22 additions & 37 deletions docs/content/preview/manage/backup-restore/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ menu:
weight: 702
type: indexpage
---

Backup and restoration is the process of creating and storing copies of your data for protection against data loss. With a proper backup strategy, you can always restore your data to a most-recent known working state and minimize application downtime. This in turn guarantees business and application continuity.

Unlike traditional single-instance databases, YugabyteDB is designed for fault tolerance. By maintaining at least three copies of your data across multiple data regions or multiple clouds, it makes sure no losses occur if a single node or single data region becomes unavailable. Thus, with YugabyteDB, you would mainly use backups to:
Expand All @@ -23,40 +24,24 @@ Unlike traditional single-instance databases, YugabyteDB is designed for fault t
* Recover from a disaster scenario, like a full cluster failure or a simultaneous outage of multiple data regions. Even though such scenarios are extremely unlikely, it's still a best practice to maintain a way to recover from them.
* Maintain a remote copy of data, as required by data protection regulations.

<div class="row">
<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="export-import-data/">
<div class="head">
<img class="icon" src="/images/section_icons/manage/export_import.png" aria-hidden="true" />
<div class="title">Export and import</div>
</div>
<div class="body">
Export and import data using SQL or CQL scripts.
</div>
</a>
</div>
<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="snapshot-ysql/">
<div class="head">
<img class="icon" src="/images/section_icons/manage/backup.png" aria-hidden="true" />
<div class="title">Distributed snapshots</div>
</div>
<div class="body">
Back up and restore data using distributed snapshots.
</div>
</a>
</div>
</div>
<div class="row">
<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="point-in-time-recovery/">
<div class="head">
<img class="icon" src="/images/section_icons/manage/pitr.png" aria-hidden="true" />
<div class="title">Point-in-time recovery</div>
</div>
<div class="body">
Restore data to a particular point in time.
</div>
</a>
</div>
</div>
{{<index/block>}}

{{<index/item
title="Export and import"
body="Export and import data using SQL or CQL scripts."
href="export-import-data/"
icon="/images/section_icons/manage/export_import.png">}}

{{<index/item
title="Distributed snapshots"
body="Back up and restore data using distributed snapshots."
href="snapshot-ysql/"
icon="/images/section_icons/manage/backup.png">}}

{{<index/item
title="Point-in-time recovery"
body="Restore data to a particular point in time."
href="point-in-time-recovery/"
icon="/images/section_icons/manage/pitr.png">}}

{{</index/block>}}
72 changes: 45 additions & 27 deletions docs/content/preview/manage/data-migration/_index.md
Original file line number Diff line number Diff line change
@@ -1,38 +1,56 @@
---
title: Migrate data
headerTitle: Migrate data
linkTitle: Migrate data
linkTitle: Migrate
description: Import and export data in YugabyteDB.
image: /images/section_icons/manage/enterprise.png
headcontent: Bulk export and import data from and to YugabyteDB
image: fa-thin fa-cloud-binary
headcontent: Export data and schema from other databases and import into YugabyteDB
menu:
preview:
identifier: manage-bulk-import-export
parent: manage
weight: 703
type: indexpage
---
<div class="row">
<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="bulk-export-ysql/">
<div class="head">
<img class="icon" src="/images/section_icons/index/deploy.png" aria-hidden="true" />
<div class="title">Bulk export</div>
</div>
<div class="body">
This section describes how to bulk export data out of YugabyteDB.
</div>
</a>
</div>
<div class="col-12 col-md-6 col-lg-12 col-xl-6">
<a class="section-link icon-offset" href="bulk-import-ysql/">
<div class="head">
<img class="icon" src="/images/section_icons/index/deploy.png" aria-hidden="true" />
<div class="title">Bulk import</div>
</div>
<div class="body">
This section describes how to bulk import data into YugabyteDB.
</div>
</a>
</div>
</div>

Migrating to YugabyteDB, a high-performance distributed SQL database, involves a series of carefully planned steps to ensure a seamless transition from your existing database environment, such as PostgreSQL, to a scalable, fault-tolerant, and globally distributed system. Whether you're moving from a monolithic setup or another distributed system, understanding the migration strategies, tools, and best practices is crucial to achieving a successful migration.

## YugabyteDB Voyager

Yugabyte ships [YugabyteDB Voyager](/preview/yugabyte-voyager/), a comprehensive data migration tool designed specifically to help users migrate from traditional databases like PostgreSQL, Oracle, MySQL, and others to YugabyteDB. It offers a variety of features that make the migration process smoother, more reliable, and less error-prone.

{{<lead link="/preview/yugabyte-voyager/migrate/">}}
To learn more about how to methodically export using YB Voyager, see [Migrate](/preview/yugabyte-voyager/migrate/).
{{</lead>}}

## Export your data

If you need to export your data manually, you can use the [ysql_dump](../../admin/ysql-dump/) tool or use the [COPY TO](../../api/ysql/the-sql-language/statements/cmd_copy/) command to export your tables into CSV files.

{{<lead link="../../manage/data-migration/bulk-export-ysql/">}}
To learn more about how to use ysql_dump and the `COPY` command to export your data, see [Export data](../../manage/data-migration/bulk-export-ysql/).
{{</lead>}}

## Import your data

To import data into a YSQL database manually, you can use [ysqlsh](../../admin/ysqlsh/) tool or use the [COPY FROM](../../api/ysql/the-sql-language/statements/cmd_copy/) command to import CSV files into YugabyteDB.

{{<lead link="../../manage/data-migration/bulk-import-ysql/">}}
To learn more about how to import your data manually and best practices, see [Import data](../../manage/data-migration/bulk-import-ysql/).
{{</lead>}}

## Verify migration

After the data has been imported into the newly set up YugabyteDB cluster, you need to verify and validate that the data and the schema have been migrated correctly to ensure smooth functioning of your services.

{{<lead link="">}}
To understand the various steps involved in validating your data, see [Verify migration](./verify-migration-ysql).
{{</lead>}}

## Guide to migration from PostgreSQL

Migrating from one system to another is no trivial task. It involves pre-planning, exporting and importing schema, analyzing and modifying schema, exporting and importing data, and more. Follow these precautions and steps to ensure a smooth migration.

{{<lead link="../../manage/data-migration/migrate-from-postgres/">}}
To understand the various steps involved in migrating from PostgreSQL and best practices, see [Migrate from PostgreSQL](../../manage/data-migration/migrate-from-postgres/).
{{</lead>}}
22 changes: 5 additions & 17 deletions docs/content/preview/manage/data-migration/bulk-export-ycql.md
Original file line number Diff line number Diff line change
@@ -1,32 +1,20 @@
---
title: Bulk export YCQL
headerTitle: Bulk export for YCQL
linkTitle: Bulk export
headerTitle: Export data
linkTitle: Export data
description: Bulk export for YCQL using cassandra-loader and cassandra-unloader.
badges: ycql
aliases:
- /preview/manage/data-migration/ycql/bulk-export/
menu:
preview:
identifier: manage-bulk-export-ycql
parent: manage-bulk-import-export
weight: 705
weight: 720
type: docs
---

<ul class="nav nav-tabs-alt nav-tabs-yb">
<li >
<a href="../bulk-export-ysql/" class="nav-link">
<i class="icon-postgres" aria-hidden="true"></i>
YSQL
</a>
</li>
<li >
<a href="../bulk-export-ycql/" class="nav-link active">
<i class="icon-cassandra" aria-hidden="true"></i>
YCQL
</a>
</li>
</ul>
{{<api-tabs>}}

Bulk export is available for YugabyteDB's [Cassandra-compatible YCQL API](../../../api/ycql/). To export data from a YugabyteDB or an Apache Cassandra table, you can use the [`cassandra-unloader`](https://github.com/yugabyte/cassandra-loader#cassandra-unloader) tool.

Expand Down
Loading