-
Notifications
You must be signed in to change notification settings - Fork 2
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added backup pages for S3 / DynamoDb
* Added backup pages for S3 / DynamoDb * fixed typo
- Loading branch information
Showing
4 changed files
with
230 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,10 @@ | ||
{ | ||
"label": "Backup", | ||
"collapsible": true, | ||
"collapsed": true, | ||
"position": 8, | ||
"link": { | ||
"type": "generated-index", | ||
"description": "Service Backup" | ||
} | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,128 @@ | ||
# DynamoDB Backup | ||
|
||
## Introduction | ||
This backup policy outlines the procedures for creating, maintaining, and restoring backups of DynamoDB tables to ensure data durability, business continuity, and quick recovery in case of data loss, corruption, or unintentional modification. | ||
|
||
Depending on your companies Recovery Time Objective (RTO), Recovery Point Objective (RPO) objectives, regulatory, and business requirements for data retention the default FormKiQ configuration may be enough or you might need to enable additional backup strategies. | ||
|
||
## Point-in-Time Recovery | ||
|
||
By default FormKiQ sets up all DynamoDb tables that store document data with Point-in-Time Recovery (PITR) enabled. This ensures continuous backups of these tables and allows for the recovery from human errors, data corruption, or accidental table modifications. | ||
|
||
With Point-in-Time Recovery (PITR), enabled this will provide continuous backups for up to **35 days** window. | ||
|
||
## Additional Backup Strategy | ||
|
||
### On-Demand Backups | ||
|
||
On-demand backups create a snapshot of the entire DynamoDB table, which is retained until explicitly deleted. On-demand backups are not continuous but can be created at specific points in time. | ||
|
||
#### When to Implement: | ||
- Before making significant schema changes or data migrations. | ||
- Prior to any major deployments or upgrades that could potentially cause data corruption. | ||
- To comply with long-term retention policies that exceed the 35-day limit of PITR. | ||
- For tables that are not frequently updated but need long-term, stable backup points. | ||
|
||
#### Why to Implement | ||
|
||
On-demand backups provide a stable snapshot for long-term retention and disaster recovery outside the 35-day PITR window. This is especially useful for audit purposes or meeting compliance requirements where longer retention periods are necessary. | ||
|
||
#### How to Implement | ||
- Use the AWS Management Console, AWS CLI, or SDK to create on-demand backups. | ||
- Schedule on-demand backups using automation (e.g., AWS Lambda, AWS Backup, or a CI/CD pipeline). | ||
- Retain these backups for a specified time, typically **90 days**. | ||
|
||
### AWS Backup Integration | ||
|
||
AWS Backup is a centralized backup service for managing backups across multiple AWS services, including DynamoDB. | ||
|
||
#### When to Implement | ||
- If your organization manages multiple AWS resources and you require centralized backup management. | ||
- For cross-region or cross-account backup needs. | ||
- To comply with governance and compliance policies that require backup automation and reporting. | ||
|
||
#### Why to Implement | ||
|
||
AWS Backup provides a unified backup management system, automating the backup creation, retention, and deletion processes for multiple AWS services. It ensures compliance, simplifies cross-region backups, and supports disaster recovery. | ||
|
||
#### How to Implement | ||
|
||
- Define backup plans using AWS Backup, specifying the DynamoDB tables to back up and the frequency of backups. | ||
- Use AWS Backup for **cross-region** backups to ensure resilience against regional outages. | ||
- Schedule backups and automate retention policies using AWS Backup’s lifecycle rules. | ||
|
||
### Export to Amazon S3 | ||
|
||
Exporting DynamoDB table data to Amazon S3 allows for the extraction of table data for archival, analysis, and integration with other systems. | ||
|
||
#### When to Implement | ||
|
||
- When you need an archive of data beyond the 35-day PITR window. | ||
- When you need to perform long-term auditing or compliance reporting that requires storing data outside of DynamoDB. | ||
- If integration with other analytics tools is necessary (e.g., Amazon Athena, Amazon Redshift). | ||
|
||
#### Why to Implement | ||
|
||
Exporting to S3 provides flexibility for long-term data retention, offline analysis, and third-party integration. It offers cost-effective storage for data that is no longer needed in DynamoDB but must be retained. | ||
|
||
#### How to Implement | ||
|
||
- Use the **Export to S3** feature in DynamoDB to export data to Amazon S3 without affecting table performance. | ||
- Schedule regular exports (e.g., monthly) to capture historical data snapshots. | ||
- Implement lifecycle policies in Amazon S3 to manage the retention and archival of exported data. | ||
|
||
## Restoring from Backups | ||
|
||
### Restoring Using Point-in-Time Recovery (PITR) | ||
|
||
#### When to Use | ||
|
||
PITR should be used when you need to restore a table due to human errors such as accidental data deletion, corruption, or modification. | ||
|
||
#### How to Restore | ||
* In the AWS Management Console, navigate to **DynamoDB**. | ||
* Select the table you wish to restore. | ||
* Choose **Actions** > **Restore Table to Point in Time**. | ||
* Specify the date and exact time (to the second) to restore the table to. | ||
* Confirm the restoration process. A new table will be created with the restored data. | ||
|
||
**Limitations**: PITR can restore data only within the last 35 days. | ||
|
||
### Restoring Using On-Demand Backups | ||
|
||
On-demand backups should be used for restoring a table after a major incident, such as a complete table loss or failure during a schema update, or when restoring from a long-term retention backup. | ||
|
||
#### How to Restore | ||
|
||
* In the AWS Management Console, go to **DynamoDB** and select **Backups**. | ||
* Find the backup you wish to restore from. | ||
* Select the **Restore** option and choose the appropriate backup snapshot. | ||
* Specify the table name for the restored data (it will create a new table). | ||
* Confirm the restoration process. | ||
|
||
**Limitations**: Restores are limited by the time and date the backup was taken. | ||
|
||
### Restoring Using AWS Backup | ||
|
||
Use AWS Backup when performing cross-region or cross-account disaster recovery, or when centralizing recovery across multiple AWS services. | ||
|
||
#### How to Restore | ||
|
||
* Open the **AWS Backup** console. | ||
* Navigate to **Protected Resources** and select **DynamoDB**. | ||
* Select the backup plan and the specific backup to restore. | ||
* Click on **Restore** and specify the region and account where the restore should take place. | ||
* AWS Backup will initiate the restore, and a new table will be created with the restored data. | ||
|
||
**Limitations**: Ensure that the destination region/account has proper permissions for restoring resources. | ||
|
||
### Restoring Using Export to Amazon S3 | ||
|
||
Use S3 exports when you need to restore data outside the normal DynamoDB operations, such as for historical analysis or long-term audit requirements. | ||
|
||
#### How to Restore | ||
* In the AWS Management Console, go to **Amazon S3** and find the bucket where the data was exported. | ||
* Retrieve the exported data files (they will be in a format such as **.csv** or **.json**). | ||
* Depending on the purpose, re-import the data back into DynamoDB (if needed) using scripts or AWS Glue ETL jobs, or analyze the data directly in Amazon S3 using tools like Amazon Athena. | ||
|
||
**Limitations**: Restoring data from S3 is not a direct DynamoDB recovery process; additional ETL workflows are needed for re-ingestion. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,91 @@ | ||
# S3 Backup | ||
|
||
## Introduction | ||
Amazon S3 (Simple Storage Service) is designed to provide highly reliable and redundant storage for critical data. S3 automatically replicates data across multiple devices and facilities within a region, ensuring **99.999999999% (11 9's) durability** of objects over a given year. With its **built-in redundancy** and ability to store data across multiple Availability Zones (AZs), S3 ensures high availability and protection against hardware failures, providing **99.99% availability**. | ||
|
||
This backup and restore policy outlines additional procedures to complement S3’s reliability and redundancy, ensuring business continuity and quick recovery in case of accidental data loss, corruption, or intentional deletions. The policy also addresses strategies for long-term data archiving and lifecycle management for optimizing costs while maintaining data durability. | ||
|
||
|
||
## S3 Versioning | ||
|
||
By default FormKiQ configures S3 buckets to have S3 versioning enabled. Versioning allows for the preservation of, retrieval, and restoration of every version of every object stored in an S3 bucket. This allows for near real-time protection against accidental deletions or overwrites and gives the ability to restore specific versions immediately. | ||
|
||
## Additional Backup Strategies | ||
|
||
### Cross-Region Replication (CRR) | ||
|
||
Cross-Region Replication (CRR) automatically replicates S3 bucket data to another region. | ||
|
||
#### When to Implement | ||
- To protect data from regional outages or disasters. | ||
- For compliance with geographical data storage regulations that require data to be stored in multiple locations. | ||
|
||
#### Why to Implement | ||
|
||
CRR provides disaster recovery capabilities and ensures data is available in a different region in case of regional failures or major outages. | ||
|
||
#### How to Implement | ||
1. Enable Versioning on both the source and destination buckets. | ||
2. Set up CRR in the AWS Management Console or via the AWS CLI. | ||
3. Monitor replication status to ensure successful data transfer between regions. | ||
|
||
### AWS Backup | ||
|
||
AWS Backup is a centralized backup solution that supports automated backup and restore for S3 buckets. | ||
|
||
#### When to Implement | ||
- For managing backups across multiple AWS services and resources in a unified manner. | ||
- To automate the backup process for S3, ensuring regular and consistent snapshots of your data. | ||
|
||
#### Why to Implement | ||
|
||
AWS Backup simplifies the process of managing and automating backups across different services and ensures compliance with backup policies for S3 and other AWS resources. | ||
|
||
#### How to Implement | ||
1. Configure backup plans in AWS Backup, specifying the S3 buckets and frequency of backups. | ||
2. Schedule automatic backups based on your RPO requirements. | ||
3. Define retention policies within AWS Backup to control the duration for which backups are kept. | ||
|
||
## Restoring from Backups | ||
|
||
### Restoring Using S3 Versioning | ||
|
||
Use versioning to restore previous versions of objects that have been accidentally deleted or overwritten. | ||
|
||
#### How to Restore | ||
|
||
* Navigate to the S3 bucket in the AWS Management Console. | ||
* Enable **Show Versions** to display all object versions. | ||
* Identify the version you want to restore. | ||
* Copy or download the specific object version, or set the desired version as the latest by removing the "Delete Marker" (if deleted). | ||
|
||
**RTO**: Immediate, depending on the size of the object. | ||
**Limitations**: Only available if Versioning is enabled on the bucket before the deletion or overwrite occurred. | ||
|
||
### Restoring Using Cross-Region Replication (CRR) | ||
|
||
Use CRR when you need to recover data from a replicated bucket in a different region due to a regional outage or disaster. | ||
|
||
#### How to Restore | ||
|
||
* Navigate to the destination bucket in the secondary region. | ||
* Access the replicated objects from the CRR-enabled bucket. | ||
* Download the necessary objects or replicate them back to the primary region as needed. | ||
|
||
**RTO**: Immediate to a few hours, depending on the size of the data and replication status. | ||
|
||
**Limitations**: Replication is asynchronous; there could be slight delays in data replication across regions. | ||
|
||
### Restoring Using AWS Backup | ||
|
||
Use AWS Backup to restore an entire S3 bucket or specific objects in case of accidental deletion, data corruption, or disaster recovery. | ||
|
||
#### How to Restore | ||
|
||
* Open the **AWS Backup** console and navigate to **Protected Resources**. | ||
* Find the S3 bucket backup you want to restore. | ||
* Click **Restore**, and specify the bucket or object you want to restore. | ||
* Optionally, choose a different bucket for the restore location. | ||
|
||
**RTO**: Typically within **minutes to hours**, depending on the size and number of objects being restored. | ||
**Limitations**: Full restoration may take longer for very large datasets. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters