Add support for metadata on s3 objects #1945

jocgir · 2017-10-17T16:25:20Z

No description provided.

scott2449 · 2018-05-24T19:59:59Z

Not sure how the pipeline works here. How long before this will hit the release version?

jocgir · 2018-05-24T20:20:41Z

Not sure neither. One of our PR have been merged within minutes while another took months. We just cross our fingers and wait.

scott2449 · 2018-05-24T20:27:48Z

We have some Hashicorp licenses/support @ Dow Jones. Let me poke them from that direction.

jocgir · 2018-05-24T20:30:35Z

Cool, thanks!

julienduchesne · 2018-07-04T13:10:18Z

This PR would be ready to merge. We have been using it for a while now. Added an acceptance test.

make testacc TESTARGS='-run=TestAccAWSS3BucketObject_metadata'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./... -v -run=TestAccAWSS3BucketObject_metadata -timeout 120m
?       github.com/terraform-providers/terraform-provider-aws   [no test files]
=== RUN   TestAccAWSS3BucketObject_metadata
--- PASS: TestAccAWSS3BucketObject_metadata (28.45s)
PASS
ok      github.com/terraform-providers/terraform-provider-aws/aws       28.461s

bflad

Hi @jocgir 👋 Thanks for submitting this. Please see the below initial feedback and let us know if you have any questions or do not have time to implement the items.

bflad · 2019-03-26T18:08:13Z

aws/resource_aws_s3_bucket_object.go

@@ -77,6 +77,12 @@ func resourceAwsS3BucketObject() *schema.Resource {
 				Optional: true,
 			},

+			"metadata": {


Can you please add the Elem configuration for this new TypeMap attribute?

Elem: schema.Schema{Type: schema.TypeString},

bflad · 2019-03-26T18:09:32Z

aws/resource_aws_s3_bucket_object.go

+			meta[key] = &valueStr
+		}
+
+		putInput.Metadata = meta


The above conversion logic can be removed with using the existing stringMapToPointers helper function:

Suggested change

putInput.Metadata = meta

putInput.Metadata = stringMapToPointers(v.(map[string]interface{}))

bflad · 2019-03-26T18:09:48Z

aws/resource_aws_s3_bucket_object.go

@@ -299,6 +315,15 @@ func resourceAwsS3BucketObjectRead(d *schema.ResourceData, meta interface{}) err
 	d.Set("content_encoding", resp.ContentEncoding)
 	d.Set("content_language", resp.ContentLanguage)
 	d.Set("content_type", resp.ContentType)
+	metadata := pointersMapToStringList(resp.Metadata)
+
+	// AWS Go SDK capitalizes metadata, this is a workaround. https://github.com/aws/aws-sdk-go/issues/445


Thanks for this context!

aws/resource_aws_s3_bucket_object.go

aws/resource_aws_s3_bucket_object_test.go

bflad · 2019-03-26T18:13:05Z

aws/resource_aws_s3_bucket_object_test.go

+		CheckDestroy: testAccCheckAWSS3BucketObjectDestroy,
+		Steps: []resource.TestStep{
+			{
+				PreConfig: func() {},


Extraneous PreConfig can be removed 👍

website/docs/r/s3_bucket_object.html.markdown

bflad · 2019-03-26T18:15:01Z

aws/resource_aws_s3_bucket_object_test.go

+		PreCheck:     func() { testAccPreCheck(t) },
+		Providers:    testAccProviders,
+		CheckDestroy: testAccCheckAWSS3BucketObjectDestroy,
+		Steps: []resource.TestStep{


Since the new attribute does not include ForceNew: true, this test should include a second TestStep that updates the metadata information.

…on-s3-objects

Co-Authored-By: jocgir <[email protected]>

…etadata-on-s3-objects

bflad · 2019-07-24T03:46:27Z

Hi folks 👋 Since we haven't seen anything in awhile regarding the remaining feedback items, they were finished up in a followup commit. Adding the additional test steps showed that metadata updates were not occurring correctly, so that is also fixed.

Output from acceptance testing:

--- PASS: TestAccAWSS3BucketObject_contentBase64 (30.69s)
--- PASS: TestAccAWSS3BucketObject_content (31.18s)
--- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (31.51s)
--- PASS: TestAccAWSS3BucketObject_empty (31.68s)
--- PASS: TestAccAWSS3BucketObject_sse (31.84s)
--- PASS: TestAccAWSS3BucketObject_source (32.12s)
--- PASS: TestAccAWSS3BucketObject_updates (50.09s)
--- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (51.04s)
--- PASS: TestAccAWSS3BucketObject_kms (54.27s)
--- PASS: TestAccAWSS3BucketObject_metadata (69.32s)
--- PASS: TestAccAWSS3BucketObject_acl (72.38s)
--- PASS: TestAccAWSS3BucketObject_tags (89.73s)
--- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (90.66s)
--- PASS: TestAccAWSS3BucketObject_storageClass (107.45s)

Reference: #1945 (review) Output from acceptance testing: ``` --- PASS: TestAccAWSS3BucketObject_contentBase64 (30.69s) --- PASS: TestAccAWSS3BucketObject_content (31.18s) --- PASS: TestAccAWSS3BucketObject_withContentCharacteristics (31.51s) --- PASS: TestAccAWSS3BucketObject_empty (31.68s) --- PASS: TestAccAWSS3BucketObject_sse (31.84s) --- PASS: TestAccAWSS3BucketObject_source (32.12s) --- PASS: TestAccAWSS3BucketObject_updates (50.09s) --- PASS: TestAccAWSS3BucketObject_updatesWithVersioning (51.04s) --- PASS: TestAccAWSS3BucketObject_kms (54.27s) --- PASS: TestAccAWSS3BucketObject_metadata (69.32s) --- PASS: TestAccAWSS3BucketObject_acl (72.38s) --- PASS: TestAccAWSS3BucketObject_tags (89.73s) --- PASS: TestAccAWSS3BucketObject_tagsLeadingSlash (90.66s) --- PASS: TestAccAWSS3BucketObject_storageClass (107.45s) ```

julienduchesne · 2019-07-24T11:10:18Z

Oh wow. Completely forgot about updating this PR for merge. Thanks for making the changes yourself and merging @bflad 🎉 ❤️

bflad · 2019-07-26T00:41:46Z

This has been released in version 2.21.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests, documentation updates, or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

ghost · 2019-11-02T15:11:03Z

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!

Add support for metadata on s3 objects

990fb3c

Ninir added the enhancement Requests to existing resources that expand the functionality or scope. label Oct 18, 2017

julienduchesne and others added 2 commits November 10, 2017 18:44

Write metadata as lower case

8165453

Adjust the documentation.

6ac55c8

jocgir mentioned this pull request Nov 11, 2017

New ssm parameters features #1520

Merged

radeksimko added the size/M Managed by automation to categorize the size of a PR. label Nov 15, 2017

bflad added the service/s3 Issues and PRs that pertain to the s3 service. label Jan 28, 2018

julienduchesne deleted the support-metadata-on-s3-objects branch February 26, 2018 11:53

Merge branch 'upstream' into support-metadata-on-s3-objects

97ff4c2

ghost added the size/M Managed by automation to categorize the size of a PR. label Apr 7, 2018

Merge branch 'upstream' into support-metadata-on-s3-objects

788dbd5

Merge branch 'master' into support-metadata-on-s3-objects

d878a5f

julienduchesne force-pushed the support-metadata-on-s3-objects branch from c0e0ff6 to d878a5f Compare July 4, 2018 12:19

ghost added the size/M Managed by automation to categorize the size of a PR. label Jul 4, 2018

Add acceptance test for s3 object metadata

b78710a

ghost added tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. documentation Introduces or discusses updates to documentation. labels Jan 18, 2019

julienduchesne force-pushed the support-metadata-on-s3-objects branch from f1fffa3 to b78710a Compare January 18, 2019 16:28

Merge branch 'master' into support-metadata-on-s3-objects

1b6be88

julienduchesne force-pushed the support-metadata-on-s3-objects branch from fea83d3 to 1b6be88 Compare January 18, 2019 16:44

ewbankkit mentioned this pull request Mar 9, 2019

aws_s3_bucket_object metadata key name incorrect #7810

Closed

bflad reviewed Mar 26, 2019

View reviewed changes

bflad added the waiting-response Maintainers are waiting on response from community or contributor. label Mar 26, 2019

jocgir and others added 2 commits April 25, 2019 17:38

Merge remote-tracking branch 'upstream/master' into support-metadata-…

a1c8433

…on-s3-objects

Update website/docs/r/s3_bucket_object.html.markdown

87effba

Co-Authored-By: jocgir <[email protected]>

bflad and others added 3 commits April 25, 2019 18:08

Update aws/resource_aws_s3_bucket_object.go

cecac74

Co-Authored-By: jocgir <[email protected]>

Update aws/resource_aws_s3_bucket_object_test.go

0db5475

Co-Authored-By: jocgir <[email protected]>

Merge remote-tracking branch 'upstream/stable-website' into support-m…

a3931ef

…etadata-on-s3-objects

aeschright requested a review from a team June 28, 2019 18:17

bflad added this to the v2.21.0 milestone Jul 24, 2019

bflad merged commit a3931ef into hashicorp:master Jul 24, 2019

bflad added a commit that referenced this pull request Jul 24, 2019

Update CHANGELOG for #1945

47a6e20

ghost removed the waiting-response Maintainers are waiting on response from community or contributor. label Jul 24, 2019

ghost locked and limited conversation to collaborators Nov 2, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for metadata on s3 objects #1945

Add support for metadata on s3 objects #1945

jocgir commented Oct 17, 2017

scott2449 commented May 24, 2018

jocgir commented May 24, 2018

scott2449 commented May 24, 2018

jocgir commented May 24, 2018

julienduchesne commented Jul 4, 2018

bflad left a comment

bflad Mar 26, 2019

bflad Mar 26, 2019

bflad Mar 26, 2019

bflad Mar 26, 2019

bflad Mar 26, 2019

bflad commented Jul 24, 2019

julienduchesne commented Jul 24, 2019

bflad commented Jul 26, 2019

ghost commented Nov 2, 2019

	putInput.Metadata = meta
	putInput.Metadata = stringMapToPointers(v.(map[string]interface{}))

Add support for metadata on s3 objects #1945

Add support for metadata on s3 objects #1945

Conversation

jocgir commented Oct 17, 2017

scott2449 commented May 24, 2018

jocgir commented May 24, 2018

scott2449 commented May 24, 2018

jocgir commented May 24, 2018

julienduchesne commented Jul 4, 2018

bflad left a comment

Choose a reason for hiding this comment

bflad Mar 26, 2019

Choose a reason for hiding this comment

bflad Mar 26, 2019

Choose a reason for hiding this comment

bflad Mar 26, 2019

Choose a reason for hiding this comment

bflad Mar 26, 2019

Choose a reason for hiding this comment

bflad Mar 26, 2019

Choose a reason for hiding this comment

bflad commented Jul 24, 2019

julienduchesne commented Jul 24, 2019

bflad commented Jul 26, 2019

ghost commented Nov 2, 2019