Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

r/aws_s3_bucket: Support S3 Cross-Region Replication filtering based on S3 object tags #6095

Merged
merged 1 commit into from
Oct 31, 2018
Merged

r/aws_s3_bucket: Support S3 Cross-Region Replication filtering based on S3 object tags #6095

merged 1 commit into from
Oct 31, 2018

Conversation

ewbankkit
Copy link
Contributor

@ewbankkit ewbankkit commented Oct 8, 2018 •
edited by bflad
Loading

Fixes #828
Fixes #5940.

Acceptance tests:

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_Replication'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3Bucket_Replication -timeout 120m
=== RUN   TestAccAWSS3Bucket_Replication
=== PAUSE TestAccAWSS3Bucket_Replication
=== RUN   TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
=== PAUSE TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
=== RUN   TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== PAUSE TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== RUN   TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== PAUSE TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== RUN   TestAccAWSS3Bucket_ReplicationSchemaV2
--- PASS: TestAccAWSS3Bucket_ReplicationSchemaV2 (183.92s)
=== CONT  TestAccAWSS3Bucket_Replication
=== CONT  TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError
=== CONT  TestAccAWSS3Bucket_ReplicationWithoutStorageClass
=== CONT  TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation
--- PASS: TestAccAWSS3Bucket_ReplicationExpectVersioningValidationError (33.54s)
--- PASS: TestAccAWSS3Bucket_ReplicationWithoutStorageClass (57.85s)
--- PASS: TestAccAWSS3Bucket_ReplicationConfiguration_Rule_Destination_AccessControlTranslation (126.77s)
--- PASS: TestAccAWSS3Bucket_Replication (154.13s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	338.080s

@ghost ghost added size/XL Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. service/s3 Issues and PRs that pertain to the s3 service. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. labels Oct 8, 2018
@ewbankkit ewbankkit mentioned this pull request Oct 14, 2018
Closed
@ewbankkit ewbankkit changed the title [WIP] r/aws_s3_bucket: Support S3 Cross-Region Replication filtering based on S3 object tags r/aws_s3_bucket: Support S3 Cross-Region Replication filtering based on S3 object tags Oct 20, 2018
ewbankkit
ewbankkit commented Oct 20, 2018
View reviewed changes
aws/resource_aws_s3_bucket.go
err := resource.Retry(1*time.Minute, func() *resource.RetryError {
if _, err := s3conn.PutBucketReplication(i); err != nil {
if isAWSErr(err, s3.ErrCodeNoSuchBucket, "") ||
isAWSErr(err, "InvalidRequest", "Versioning must be 'Enabled' on the bucket") {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check fixes #828.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

ewbankkit
ewbankkit commented Oct 20, 2018
View reviewed changes
aws/resource_aws_s3_bucket.go
Prefix: aws.String(rr["prefix"].(string)),
Status: aws.String(rr["status"].(string)),
rcRule := &s3.ReplicationRule{}
if status, ok := rr["status"]; ok && status != "" {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check and the subsequent continue is required for updates when the bucket is using CRR schema V2 or else the PutBucketReplication API call fails with missing Bucket Name errors.

ewbankkit
ewbankkit commented Oct 20, 2018
View reviewed changes
aws/resource_aws_s3_bucket_test.go
@@ -1577,6 +1758,14 @@ func testAccCheckAWSS3BucketReplicationRules(n string, providerF func() *schema.
}
}
}
// Sort filter tags by key.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to sort both the specified and returned filter tags as they are in non-deterministic order and I was getting errors like

$ make testacc TEST=./aws/ TESTARGS='-run=TestAccAWSS3Bucket_ReplicationSchemaV2'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws/ -v -parallel 20 -run=TestAccAWSS3Bucket_ReplicationSchemaV2 -timeout 120m
=== RUN   TestAccAWSS3Bucket_ReplicationSchemaV2
--- FAIL: TestAccAWSS3Bucket_ReplicationSchemaV2 (167.52s)
    testing.go:538: Step 3 error: Check failed: Check 6/6 error: bad replication rules, expected: [{
          DeleteMarkerReplication: {
            Status: "Disabled"
          },
          Destination: {
            Bucket: "arn:aws:s3:::tf-test-bucket-destination-5528715964644708672",
            StorageClass: "STANDARD"
          },
          Filter: {
            And: {
              Tags: [{
                  Key: "ReplicateMe",
                  Value: "Yes"
                },{
                  Key: "AnotherTag",
                  Value: "OK"
                },{
                  Key: "Foo",
                  Value: "Bar"
                }]
            }
          },
          ID: "foobar",
          Priority: 0,
          Status: "Enabled"
        }], got [{
          DeleteMarkerReplication: {
            Status: "Disabled"
          },
          Destination: {
            Bucket: "arn:aws:s3:::tf-test-bucket-destination-5528715964644708672",
            StorageClass: "STANDARD"
          },
          Filter: {
            And: {
              Prefix: "",
              Tags: [{
                  Key: "AnotherTag",
                  Value: "OK"
                },{
                  Key: "ReplicateMe",
                  Value: "Yes"
                },{
                  Key: "Foo",
                  Value: "Bar"
                }]
            }
          },
          ID: "foobar",
          Priority: 0,
          Status: "Enabled"
        }]
FAIL
FAIL	github.com/terraform-providers/terraform-provider-aws/aws	167.532s
GNUmakefile:20: recipe for target 'testacc' failed
make: *** [testacc] Error 1

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@ewbankkit
Copy link
Contributor Author

Removed WIP, ready for review.

bflad
bflad approved these changes Oct 31, 2018
View reviewed changes
Copy link
Contributor

@bflad bflad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, excellent job @ewbankkit! 🚀 I've run the acceptance times a few times and passes just fine.

aws/resource_aws_s3_bucket.go
err := resource.Retry(1*time.Minute, func() *resource.RetryError {
if _, err := s3conn.PutBucketReplication(i); err != nil {
if isAWSErr(err, s3.ErrCodeNoSuchBucket, "") ||
isAWSErr(err, "InvalidRequest", "Versioning must be 'Enabled' on the bucket") {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

aws/resource_aws_s3_bucket_test.go
@@ -1577,6 +1758,14 @@ func testAccCheckAWSS3BucketReplicationRules(n string, providerF func() *schema.
}
}
}
// Sort filter tags by key.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

@bflad bflad added this to the v1.42.0 milestone Oct 31, 2018
@bflad bflad merged commit b32213e into hashicorp:master Oct 31, 2018
bflad added a commit that referenced this pull request Oct 31, 2018
@bflad
Update CHANGELOG for #6095
5ee23a2
@bflad
Copy link
Contributor

bflad commented Nov 1, 2018

This has been released in version 1.42.0 of the AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

@ewbankkit ewbankkit deleted the issue-5940 branch November 1, 2018 12:00
@ewbankkit ewbankkit mentioned this pull request Nov 2, 2018
Closed
@ghost
Copy link

ghost commented Apr 2, 2020

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

@ghost ghost locked and limited conversation to collaborators Apr 2, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Reviewers

@bflad bflad bflad approved these changes

Assignees
No one assigned
Labels
documentation Introduces or discusses updates to documentation. service/s3 Issues and PRs that pertain to the s3 service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Milestone
v1.42.0
2 participants
@ewbankkit @bflad