Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add DynamoDB Global Table Support #2482

Closed
bflad opened this issue Nov 30, 2017 · 8 comments · Fixed by #2517
Closed

Add DynamoDB Global Table Support #2482

bflad opened this issue Nov 30, 2017 · 8 comments · Fixed by #2517
Labels
new-resource Introduces a new resource. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Milestone

Comments

@bflad
Copy link
Contributor

bflad commented Nov 30, 2017

AWS has announced support for managing globally replicating DynamoDB tables: https://aws.amazon.com/about-aws/whats-new/2017/11/aws-launches-amazon-dynamodb-global-tables/

Prerequisite: aws-sdk-go v1.12.36 (#2474)

Terraform Version

terraform 0.10+
terraform-provider-aws 1.5.0

Affected Resource(s)

  • aws_dynamodb_table
  • Potentially new: aws_dynamodb_global_table

Expected Behavior

Create, update, delete, and import DynamoDB global tables. Potentially:

provider "aws" {
  alias  = "us-east-1"
  region = "us-east-1"
}

provider "aws" {
  alias  = "us-west-2"
  region = "us-west-2"
}

resource "aws_dynamodb_table" "us-east-1" {
  provider = "aws.us-east-1"

  hash_key         = "myAttribute"
  name             = "myTable"
  stream_enabled   = true
  stream_view_type = "NEW_AND_OLD_IMAGES"
  read_capacity    = 1
  write_capacity   = 1

  attribute {
    name = "myAttribute"
    type = "S"
  }
}

resource "aws_dynamodb_table" "us-west-2" {
  provider = "aws.us-west-2"

  hash_key         = "myAttribute"
  name             = "myTable"
  stream_enabled   = true
  stream_view_type = "NEW_AND_OLD_IMAGES"
  read_capacity    = 1
  write_capacity   = 1

  attribute {
    name = "myAttribute"
    type = "S"
  }
}

# New resource
resource "aws_dynamodb_global_table" "" {
  provider = "aws.us-east-1"

  name = "myTable"

  replication_group {
    regions = [
      "us-east-1",
      "us-west-2",
    ]
  }
}

Important Factoids

From the docs:

The following is a conceptual overview of how a global table is created.

  1. Create an ordinary DynamoDB table, with DynamoDB Streams enabled, in an AWS region.
  2. Repeat step 1 for every other AWS region where you want to replicate your data.
  3. Define a DynamoDB global table, based upon the tables that you have created.

There are a lot of restrictions on how these are to be setup at the moment. From the docs:

If you want to add a new replica table to a global table, each of the following conditions must be true:

  • The table must have the same primary key as all of the other replicas.
  • The table must have the same name as all of the other replicas.
  • The table must have DynamoDB Streams enabled, with the stream containing both the new and the old images of the item.
  • None of the replica tables in the global table can contain any data.

I presume we'll want to follow the AWS API here and layer the global table management on top of our existing aws_dynamodb_table resource rather than trying to reinvent the wheel with managing all the replica tables consistently in a single resource.

References

http://docs.aws.amazon.com/amazondynamodb/latest/developerguide/globaltables_HowItWorks.html

@randomcamel randomcamel added the enhancement Requests to existing resources that expand the functionality or scope. label Nov 30, 2017
@bflad
Copy link
Contributor Author

bflad commented Nov 30, 2017

I'm going to start on an implementation. More soon. 🚀

@bflad
Copy link
Contributor Author

bflad commented Dec 2, 2017

PR submitted for new resource: #2517

@jonathan-kosgei
Copy link

jonathan-kosgei commented Dec 7, 2017

Hey @bflad awesome PR! Thanks! I'm not sure how long it'll take to get accepted upstream and to be added to a release, I just wanted to confirm that I can fork terraform, apply your pr and this should then just work?

Any example on usage would be great, do the old app autoscaling resources for dynamodb need to be created as well? Any examples? Thanks!

@radeksimko radeksimko added new-resource Introduces a new resource. service/dynamodb Issues and PRs that pertain to the dynamodb service. and removed enhancement Requests to existing resources that expand the functionality or scope. labels Jan 22, 2018
@t-pascal
Copy link

We are also eagerly anticipating this PR to be merged so we can use and test it.

@bflad
Copy link
Contributor Author

bflad commented Jan 25, 2018

Hi, everyone! This has landed in master and will ship with the v1.8.0 release of the provider, expected tomorrow. Happy Terraform'ing! 🎉

@bflad bflad added this to the v1.8.0 milestone Jan 25, 2018
@bflad
Copy link
Contributor Author

bflad commented Jan 29, 2018

This has been released in terraform-provider-aws version 1.8.0. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@akshaybadekar29
Copy link

akshaybadekar29 commented Apr 28, 2018

Hello,

I have tried using the same solution it gives me below error
Error: Error asking for user input: Error parsing address 'aws_dynamodb_global_table': invalid resource address "aws_dynamodb_global_table"

Also, I tried to update my provider to provider.aws v1.8.0
Still same error can you please help me

Terraform v0.11.7

  • provider.aws v1.8.0

@ghost
Copy link

ghost commented Apr 6, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 6, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
new-resource Introduces a new resource. service/dynamodb Issues and PRs that pertain to the dynamodb service.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants