Skip to content

Commit

Permalink
Upgrade AWS SDK Go to v2
Browse files Browse the repository at this point in the history
Although aws-sdk-go-v2 was released a long time ago, historically until
now tfmigrate has used v1 as well as the old Terraform.

aws-sdk-go and aws-sdk-go-v2 have different priorities for loading
credentials. Specifically, aws-sdk-go v1 prioritizes reading environment
variables over profiles. This is a problem when switching between
multiple accounts in profile while OIDC authenticating to AWS from
GitHub Actions.

Up to Terraform v1.5, aws-sdk-go v1 was used for s3 backend
authentication via hashicorp/aws-sdk-go-base, but Terraform / OpenTofu
v1.6 and later, aws-sdk-go-v2 is used.

Up to tfmigrate v0.3, we have been using aws-sdk-go v1 via
hashicorp/aws-sdk-go-base for history s3 storage authentication, but
will be using aws-sdk-go-v2 after the next tfmigrate v0.4. This is a
breaking change, but the goal is to align with the behavior of
Terraform / OpenTofu v1.6 and later, so if you are affected, please
adjust your AWS authentication settings.

At the time of this writing, v2 of hashicorp/aws-sdk-go-base, which uses
aws-sdk-go-v2, is still beta, but it is already used in the stable
releases of Terraform and OpenTofu. Since there are multiple beta
versions of hashicorp/aws-sdk-go-base and slightly different versions in
use, the results of my investigation are shown below.

- tfmigrate: v0.3.24: v1.1.0
- terraform v1.5.7: v0.7.1
- terraform v1.6.0: v2.0.0-beta.35
- terraform v1.7.0: v2.0.0-beta.43
- terraform v1.8.0: v2.0.0-beta.45
- terraform v1.9.8 (latest): v2.0.0-beta.45
- opentofu v1.6.0: v2.0.0-beta.43
- opentofu v1.8.5 (latest): v2.0.0-beta.43
- terraform-provider-aws v3.76.1: v1.0.0
- terraform-provider-aws v4.0.0: v2.0.0-beta.5
- terraform-provider-aws v5.0.0: v2.0.0-beta.27
- terraform-provider-aws v5.29.0: v2.0.0-beta.43
- terraform-provider-aws v5.31.0: v2.0.0-beta.45
- terraform-provider-aws v5.74.0 (latest): v2.0.0-beta.59

As a special note, use_legacy_workflow has been removed in terraform
v1.8.0 and opentofu v1.8.0, but UseLegacyWorkflow has been removed in
aws-sdk-go-base in v2.0.0-beta.49.

Considering the above, I think v2.0.0-beta.43 or v2.0.0-beta.45 would be
reasonable. At this point, I don't see any strong reason to go with
v2.0.0-beta.45, so I'm going to go with v2.0.0-beta.43 to align with
opentofu.

The s3 history storage implementation was rewritten with the
aws-sdk-go-v2 migration guide.

https://aws.github.io/aws-sdk-go-v2/docs/migrating/

As a tfmigrate user, there are no breaking changes at the configuration
file level, but it should be noted that, as mentioned above, AWS
credentials have a higher priority in profile than in environment
variables. Also, as a side effect, the path to the file that sets the
AWS profile is ~/.aws/config by default, but it can now be specified
with the environment variable AWS_CONFIG_FILE.
  • Loading branch information
minamijoyo committed Nov 7, 2024
1 parent 3f4250a commit 7798416
Show file tree
Hide file tree
Showing 7 changed files with 211 additions and 97 deletions.
41 changes: 33 additions & 8 deletions go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,15 @@ go 1.22

require (
cloud.google.com/go/storage v1.36.0
github.com/aws/aws-sdk-go v1.55.5
github.com/aws/aws-sdk-go-v2 v1.32.3
github.com/aws/aws-sdk-go-v2/config v1.28.1
github.com/aws/aws-sdk-go-v2/credentials v1.17.42
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.16.18
github.com/aws/aws-sdk-go-v2/feature/s3/manager v1.17.35
github.com/aws/aws-sdk-go-v2/service/s3 v1.66.2
github.com/davecgh/go-spew v1.1.1
github.com/google/go-cmp v0.6.0
github.com/hashicorp/aws-sdk-go-base v1.1.0
github.com/hashicorp/aws-sdk-go-base/v2 v2.0.0-beta.43
github.com/hashicorp/go-version v1.3.0
github.com/hashicorp/hcl/v2 v2.6.0
github.com/hashicorp/logutils v1.0.0
Expand All @@ -26,8 +31,25 @@ require (
github.com/apparentlymart/go-textseg v1.0.0 // indirect
github.com/apparentlymart/go-textseg/v12 v12.0.0 // indirect
github.com/armon/go-radix v0.0.0-20180808171621-7fddfc383310 // indirect
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.6.6 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.6.22 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.8.1 // indirect
github.com/aws/aws-sdk-go-v2/internal/v4a v1.3.22 // indirect
github.com/aws/aws-sdk-go-v2/service/dynamodb v1.25.5 // indirect
github.com/aws/aws-sdk-go-v2/service/iam v1.27.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.12.0 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/checksum v1.4.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/endpoint-discovery v1.8.5 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.12.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/s3shared v1.18.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sqs v1.28.4 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.24.3 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.28.3 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.32.3 // indirect
github.com/aws/smithy-go v1.22.0 // indirect
github.com/bgentry/speakeasy v0.1.0 // indirect
github.com/fatih/color v1.7.0 // indirect
github.com/fatih/color v1.15.0 // indirect
github.com/felixge/httpsnoop v1.0.4 // indirect
github.com/go-logr/logr v1.4.1 // indirect
github.com/go-logr/stdr v1.2.2 // indirect
Expand All @@ -37,22 +59,25 @@ require (
github.com/google/uuid v1.6.0 // indirect
github.com/googleapis/enterprise-certificate-proxy v0.3.2 // indirect
github.com/googleapis/gax-go/v2 v2.12.0 // indirect
github.com/hashicorp/errwrap v1.0.0 // indirect
github.com/hashicorp/go-cleanhttp v0.5.0 // indirect
github.com/hashicorp/go-multierror v1.0.0 // indirect
github.com/hashicorp/errwrap v1.1.0 // indirect
github.com/hashicorp/go-hclog v1.5.0 // indirect
github.com/hashicorp/go-multierror v1.1.1 // indirect
github.com/hashicorp/terraform-plugin-log v0.9.0 // indirect
github.com/jmespath/go-jmespath v0.4.0 // indirect
github.com/mattn/go-colorable v0.0.9 // indirect
github.com/mattn/go-isatty v0.0.3 // indirect
github.com/mattn/go-colorable v0.1.13 // indirect
github.com/mattn/go-isatty v0.0.17 // indirect
github.com/mitchellh/go-homedir v1.1.0 // indirect
github.com/mitchellh/go-wordwrap v0.0.0-20150314170334-ad45545899c7 // indirect
github.com/posener/complete v1.1.1 // indirect
go.opencensus.io v0.24.0 // indirect
go.opentelemetry.io/contrib/instrumentation/github.com/aws/aws-sdk-go-v2/otelaws v0.46.1 // indirect
go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc v0.47.0 // indirect
go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.47.0 // indirect
go.opentelemetry.io/otel v1.22.0 // indirect
go.opentelemetry.io/otel/metric v1.22.0 // indirect
go.opentelemetry.io/otel/trace v1.22.0 // indirect
golang.org/x/crypto v0.22.0 // indirect
golang.org/x/exp v0.0.0-20230905200255-921286631fa9 // indirect
golang.org/x/net v0.24.0 // indirect
golang.org/x/oauth2 v0.17.0 // indirect
golang.org/x/sync v0.6.0 // indirect
Expand Down
Loading

0 comments on commit 7798416

Please sign in to comment.