Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

s3: Enable use of AWS SDK for Go v2 #33304

Merged
merged 19 commits into from
Sep 5, 2023
Merged

s3: Enable use of AWS SDK for Go v2 #33304

merged 19 commits into from
Sep 5, 2023

Conversation

ewbankkit
Copy link
Contributor

@ewbankkit ewbankkit commented Sep 5, 2023

Description

Enable use of AWS SDK for Go v2 for S3 resources.
Migrate the aws_canonical_user_id and aws_s3_objects data sources.

Relations

Closes #26569.
Relates #33028.

Output from Acceptance Testing

Verified (using the aws_canonical_user_id data source) that the defaults for S3 API endpoints have not changed:

# SDK v1 us-west-2,s3_us_east_1_regional_endpoint="": http.url=https://s3.us-west-2.amazonaws.com/
# SDK v1 us-west-2,s3_us_east_1_regional_endpoint="legacy": http.url=https://s3.us-west-2.amazonaws.com/
# SDK v1 us-west-2,s3_us_east_1_regional_endpoint="regional": http.url=https://s3.us-west-2.amazonaws.com/
# SDK v1 us-east-1,s3_us_east_1_regional_endpoint="": http.url=https://s3.amazonaws.com/
# SDK v1 us-east-1,s3_us_east_1_regional_endpoint="legacy": http.url=https://s3.amazonaws.com/
# SDK v1 us-east-1,s3_us_east_1_regional_endpoint="regional": http.url=https://s3.us-east-1.amazonaws.com/


# SDK v2 us-west-2,s3_us_east_1_regional_endpoint="": http.url=https://s3.us-west-2.amazonaws.com/
# SDK v2 us-west-2,s3_us_east_1_regional_endpoint="legacy": http.url=https://s3.us-west-2.amazonaws.com/
# SDK v2 us-west-2,s3_us_east_1_regional_endpoint="regional": http.url=https://s3.us-west-2.amazonaws.com/
# SDK v2 us-east-1,s3_us_east_1_regional_endpoint="": http.url=https://s3.amazonaws.com/
# SDK v2 us-east-1,s3_us_east_1_regional_endpoint="legacy": http.url=https://s3.amazonaws.com/
# SDK v2 us-east-1,s3_us_east_1_regional_endpoint="regional": http.url=https://s3.us-east-1.amazonaws.com/
% make testacc TESTARGS='-run=TestAccS3CanonicalUserIDDataSource_' PKG=s3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 20  -run=TestAccS3CanonicalUserIDDataSource_ -timeout 180m
=== RUN   TestAccS3CanonicalUserIDDataSource_basic
=== PAUSE TestAccS3CanonicalUserIDDataSource_basic
=== CONT  TestAccS3CanonicalUserIDDataSource_basic
--- PASS: TestAccS3CanonicalUserIDDataSource_basic (31.20s)
PASS
ok      github.com/hashicorp/terraform-provider-aws/internal/service/s3 37.281s
% make testacc TESTARGS='-run=TestAccS3ObjectsDataSource_' PKG=s3 ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 3  -run=TestAccS3ObjectsDataSource_ -timeout 180m
=== RUN   TestAccS3ObjectsDataSource_basic
=== PAUSE TestAccS3ObjectsDataSource_basic
=== RUN   TestAccS3ObjectsDataSource_basicViaAccessPoint
=== PAUSE TestAccS3ObjectsDataSource_basicViaAccessPoint
=== RUN   TestAccS3ObjectsDataSource_prefixes
=== PAUSE TestAccS3ObjectsDataSource_prefixes
=== RUN   TestAccS3ObjectsDataSource_encoded
=== PAUSE TestAccS3ObjectsDataSource_encoded
=== RUN   TestAccS3ObjectsDataSource_maxKeys
=== PAUSE TestAccS3ObjectsDataSource_maxKeys
=== RUN   TestAccS3ObjectsDataSource_startAfter
=== PAUSE TestAccS3ObjectsDataSource_startAfter
=== RUN   TestAccS3ObjectsDataSource_fetchOwner
=== PAUSE TestAccS3ObjectsDataSource_fetchOwner
=== CONT  TestAccS3ObjectsDataSource_basic
=== CONT  TestAccS3ObjectsDataSource_maxKeys
=== CONT  TestAccS3ObjectsDataSource_fetchOwner
--- PASS: TestAccS3ObjectsDataSource_basic (48.69s)
=== CONT  TestAccS3ObjectsDataSource_startAfter
--- PASS: TestAccS3ObjectsDataSource_fetchOwner (48.82s)
=== CONT  TestAccS3ObjectsDataSource_prefixes
--- PASS: TestAccS3ObjectsDataSource_maxKeys (87.02s)
=== CONT  TestAccS3ObjectsDataSource_encoded
--- PASS: TestAccS3ObjectsDataSource_startAfter (46.36s)
=== CONT  TestAccS3ObjectsDataSource_basicViaAccessPoint
--- PASS: TestAccS3ObjectsDataSource_prefixes (46.24s)
--- PASS: TestAccS3ObjectsDataSource_encoded (43.09s)
--- PASS: TestAccS3ObjectsDataSource_basicViaAccessPoint (43.34s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	145.202s
% make testacc TESTARGS='-run=TestAccS3ObjectsDataSource_maxKeys' PKG=s3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 20  -run=TestAccS3ObjectsDataSource_maxKeys -timeout 180m
=== RUN   TestAccS3ObjectsDataSource_maxKeysSmall
=== PAUSE TestAccS3ObjectsDataSource_maxKeysSmall
=== RUN   TestAccS3ObjectsDataSource_maxKeysLarge
=== PAUSE TestAccS3ObjectsDataSource_maxKeysLarge
=== CONT  TestAccS3ObjectsDataSource_maxKeysSmall
=== CONT  TestAccS3ObjectsDataSource_maxKeysLarge
--- PASS: TestAccS3ObjectsDataSource_maxKeysSmall (76.54s)
--- PASS: TestAccS3ObjectsDataSource_maxKeysLarge (232.58s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	238.411s

% make testacc TESTARGS='-run=TestAccS3CanonicalUserIDDataSource_' PKG=s3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 20  -run=TestAccS3CanonicalUserIDDataSource_ -timeout 180m
=== RUN   TestAccS3CanonicalUserIDDataSource_basic
=== PAUSE TestAccS3CanonicalUserIDDataSource_basic
=== CONT  TestAccS3CanonicalUserIDDataSource_basic
--- PASS: TestAccS3CanonicalUserIDDataSource_basic (31.20s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	37.281s
% make testacc TESTARGS='-run=TestAccS3ObjectsDataSource_' PKG=s3 ACCTEST_PARALLELISM=3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 3  -run=TestAccS3ObjectsDataSource_ -timeout 180m
=== RUN   TestAccS3ObjectsDataSource_basic
=== PAUSE TestAccS3ObjectsDataSource_basic
=== RUN   TestAccS3ObjectsDataSource_basicViaAccessPoint
=== PAUSE TestAccS3ObjectsDataSource_basicViaAccessPoint
=== RUN   TestAccS3ObjectsDataSource_prefixes
=== PAUSE TestAccS3ObjectsDataSource_prefixes
=== RUN   TestAccS3ObjectsDataSource_encoded
=== PAUSE TestAccS3ObjectsDataSource_encoded
=== RUN   TestAccS3ObjectsDataSource_maxKeys
=== PAUSE TestAccS3ObjectsDataSource_maxKeys
=== RUN   TestAccS3ObjectsDataSource_startAfter
=== PAUSE TestAccS3ObjectsDataSource_startAfter
=== RUN   TestAccS3ObjectsDataSource_fetchOwner
=== PAUSE TestAccS3ObjectsDataSource_fetchOwner
=== CONT  TestAccS3ObjectsDataSource_basic
=== CONT  TestAccS3ObjectsDataSource_maxKeys
=== CONT  TestAccS3ObjectsDataSource_fetchOwner
--- PASS: TestAccS3ObjectsDataSource_basic (48.69s)
=== CONT  TestAccS3ObjectsDataSource_startAfter
--- PASS: TestAccS3ObjectsDataSource_fetchOwner (48.82s)
=== CONT  TestAccS3ObjectsDataSource_prefixes
--- PASS: TestAccS3ObjectsDataSource_maxKeys (87.02s)
=== CONT  TestAccS3ObjectsDataSource_encoded
--- PASS: TestAccS3ObjectsDataSource_startAfter (46.36s)
=== CONT  TestAccS3ObjectsDataSource_basicViaAccessPoint
--- PASS: TestAccS3ObjectsDataSource_prefixes (46.24s)
--- PASS: TestAccS3ObjectsDataSource_encoded (43.09s)
--- PASS: TestAccS3ObjectsDataSource_basicViaAccessPoint (43.34s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	145.202s
… than `1000`.

Acceptance test output:

% make testacc TESTARGS='-run=TestAccS3ObjectsDataSource_maxKeys' PKG=s3
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./internal/service/s3/... -v -count 1 -parallel 20  -run=TestAccS3ObjectsDataSource_maxKeys -timeout 180m
=== RUN   TestAccS3ObjectsDataSource_maxKeysSmall
=== PAUSE TestAccS3ObjectsDataSource_maxKeysSmall
=== RUN   TestAccS3ObjectsDataSource_maxKeysLarge
=== PAUSE TestAccS3ObjectsDataSource_maxKeysLarge
=== CONT  TestAccS3ObjectsDataSource_maxKeysSmall
=== CONT  TestAccS3ObjectsDataSource_maxKeysLarge
--- PASS: TestAccS3ObjectsDataSource_maxKeysSmall (76.54s)
--- PASS: TestAccS3ObjectsDataSource_maxKeysLarge (232.58s)
PASS
ok  	github.com/hashicorp/terraform-provider-aws/internal/service/s3	238.411s
@github-actions
Copy link

github-actions bot commented Sep 5, 2023

Community Note

Voting for Prioritization

  • Please vote on this pull request by adding a 👍 reaction to the original post to help the community and maintainers prioritize this pull request.
  • Please see our prioritization guide for information on how we prioritize.
  • Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request.

For Submitters

  • Review the contribution guide relating to the type of change you are making to ensure all of the necessary steps have been taken.
  • For new resources and data sources, use skaff to generate scaffolding with comments detailing common expectations.
  • Whether or not the branch has been rebased will not impact prioritization, but doing so is always a welcome surprise.

@github-actions github-actions bot added size/XL Managed by automation to categorize the size of a PR. documentation Introduces or discusses updates to documentation. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure. service/s3 Issues and PRs that pertain to the s3 service. client-connections Pertains to the AWS Client and service connections. generators Relates to code generators. labels Sep 5, 2023
@terraform-aws-provider terraform-aws-provider bot added the prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. label Sep 5, 2023
@ewbankkit ewbankkit merged commit 5abfb89 into main Sep 5, 2023
80 checks passed
@ewbankkit ewbankkit deleted the f-s3-aws-sdk-v2 branch September 5, 2023 13:20
@github-actions github-actions bot added this to the v5.16.0 milestone Sep 5, 2023
@github-actions
Copy link

github-actions bot commented Sep 8, 2023

This functionality has been released in v5.16.0 of the Terraform AWS Provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template. Thank you!

@github-actions
Copy link

github-actions bot commented Oct 8, 2023

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 8, 2023
@justinretzolk justinretzolk added the aws-sdk-go-migration Issues that are related to the providers migration to AWS SDK for Go v2. label Mar 27, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
aws-sdk-go-migration Issues that are related to the providers migration to AWS SDK for Go v2. client-connections Pertains to the AWS Client and service connections. documentation Introduces or discusses updates to documentation. generators Relates to code generators. prioritized Part of the maintainer teams immediate focus. To be addressed within the current quarter. service/s3 Issues and PRs that pertain to the s3 service. size/XL Managed by automation to categorize the size of a PR. tests PRs: expanded test coverage. Issues: expanded coverage, enhancements to test infrastructure.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Allow request_payer argument for aws_s3_objects data source
2 participants