Full transit gateway connect support

[b-b3rn4rd]

Community Note

• Please vote on this pull request by adding a 👍 reaction to the original pull request comment to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for pull request followers and do not help prioritize the request

Closes #16844

Output from acceptance testing:

% make testacc TESTARGS='-run=TestAccAWSEc2TransitGateway_basic'        
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSEc2TransitGateway_basic -timeout 180m
=== RUN   TestAccAWSEc2TransitGateway_basic
=== PAUSE TestAccAWSEc2TransitGateway_basic
=== CONT  TestAccAWSEc2TransitGateway_basic
--- PASS: TestAccAWSEc2TransitGateway_basic (228.73s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	231.735s

% make testacc TESTARGS='-run=TestAccAWSEc2TransitGatewayConnect_basic'     
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSEc2TransitGatewayConnect_basic -timeout 180m
=== RUN   TestAccAWSEc2TransitGatewayConnect_basic
=== PAUSE TestAccAWSEc2TransitGatewayConnect_basic
=== CONT  TestAccAWSEc2TransitGatewayConnect_basic
--- PASS: TestAccAWSEc2TransitGatewayConnect_basic (499.44s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	504.580s

% make testacc TESTARGS='-run=TestAccAWSEc2TransitGatewayConnectPeer_basic'
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSEc2TransitGatewayConnectPeer_basic -timeout 180m
=== RUN   TestAccAWSEc2TransitGatewayConnectPeer_basic
=== PAUSE TestAccAWSEc2TransitGatewayConnectPeer_basic
=== CONT  TestAccAWSEc2TransitGatewayConnectPeer_basic
--- PASS: TestAccAWSEc2TransitGatewayConnectPeer_basic (771.23s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	776.280s

% make testacc TESTARGS='-run=TestAccAWSEc2TransitGatewayDataSource_Filter' 
==> Checking that code complies with gofmt requirements...
TF_ACC=1 go test ./aws -v -count 1 -parallel 20 -run=TestAccAWSEc2TransitGatewayDataSource_Filter -timeout 180m
=== RUN   TestAccAWSEc2TransitGatewayDataSource_Filter
=== PAUSE TestAccAWSEc2TransitGatewayDataSource_Filter
=== CONT  TestAccAWSEc2TransitGatewayDataSource_Filter
--- PASS: TestAccAWSEc2TransitGatewayDataSource_Filter (245.02s)
PASS
ok  	github.com/terraform-providers/terraform-provider-aws/aws	248.119s

[b-b3rn4rd]

Hi Team,
The following MR implements transit gateway connect support that we heavily rely on in our environment and I'm certain that this feature will become useful once people realise the benefit of GRE over VPN. I hope you will appreciate the time and effort it took me to get this ready.
Please let me know if this MR requires any further changes.

Thanks

[kenrusse]

I tested the code and ran into a minor issue with updating TransitGatewayCidrBlocks on an existing TGW. The API doesn't like when AddTransitGatewayCidrBlocks or RemoveTransitGatewayCidrBlocks are empty.

aws_ec2_transit_gateway.tgw will be updated in-place
~ resource "aws_ec2_transit_gateway" "tgw" {
id = "tgw-0e1af40a101db028a"
tags = {
"Name" = "Transit VPC TGW"
}
~ transit_gateway_cidr_blocks = [
+ "10.36.254.0/24",
]
# (11 unchanged attributes hidden)
}

│ Error: error updating EC2 Transit Gateway (tgw-0e1af40a101db028a) options: InvalidParameterValue: All Transit Gateway CIDR blocks must be a valid IPv4 or IPv6 CIDR and align with the first address in the block. Invalid block:
│ status code: 400, request id: 0eeef407-9082-48a8-bbd7-d9c9eebfe8af
│
│ with aws_ec2_transit_gateway.tgw,
│ on main.tf line 77, in resource "aws_ec2_transit_gateway" "tgw":
│ 77: resource "aws_ec2_transit_gateway" "tgw" {
│
╵

2021-09-14T14:22:20.300-0400 [INFO] provider.terraform-provider-aws: 2021/09/14 14:22:20 [DEBUG] [aws-sdk-go] DEBUG: Request ec2/ModifyTransitGateway Details:
---[ REQUEST POST-SIGN ]-----------------------------
POST / HTTP/1.1
Host: ec2.us-east-2.amazonaws.com
User-Agent: APN/1.0 HashiCorp/1.0 Terraform/0.15.5 (+https://www.terraform.io) terraform-provider-aws/dev (+https://registry.terraform.io/providers/hashicorp/aws) aws-sdk-go/1.40.33 (go1.16.6; darwin; amd64)
Content-Length: 180
Authorization: xxxx
Content-Type: application/x-www-form-urlencoded; charset=utf-8
X-Amz-Date: 20210914T182220Z
X-Amz-Security-Token: xxxx
Accept-Encoding: gzip

Action=ModifyTransitGateway&Options.AddTransitGatewayCidrBlocks.1=10.36.254.0%2F24&Options.RemoveTransitGatewayCidrBlocks=&TransitGatewayId=tgw-0e1af40a101db028a&Version=2016-11-15
-----------------------------------------------------: timestamp=2021-09-14T14:22:20.300-0400
2021-09-14T14:22:20.507-0400 [INFO] provider.terraform-provider-aws: 2021/09/14 14:22:20 [DEBUG] [aws-sdk-go] DEBUG: Response ec2/ModifyTransitGateway Details:
---[ RESPONSE ]--------------------------------------
HTTP/1.1 400 Bad Request
Connection: close
Transfer-Encoding: chunked
Cache-Control: no-cache, no-store
Content-Type: text/xml;charset=UTF-8
Date: Tue, 14 Sep 2021 18:22:20 GMT
Server: AmazonEC2
Strict-Transport-Security: max-age=31536000; includeSubDomains
Vary: accept-encoding
X-Amzn-Requestid: 0eeef407-9082-48a8-bbd7-d9c9eebfe8af

-----------------------------------------------------: timestamp=2021-09-14T14:22:20.507-0400
2021-09-14T14:22:20.507-0400 [INFO] provider.terraform-provider-aws: 2021/09/14 14:22:20 [DEBUG] [aws-sdk-go]
InvalidParameterValueAll Transit Gateway CIDR blocks must be a valid IPv4 or IPv6 CIDR and align with the first address in the block. Invalid block:0eeef407-9082-48a8-bbd7-d9c9eebfe8af: timestamp=2021-09-14T14:22:20.507-0400
2021-09-14T14:22:20.508-0400 [INFO] provider.terraform-provider-aws: 2021/09/14 14:22:20 [DEBUG] [aws-sdk-go] DEBUG: Validate Response ec2/ModifyTransitGateway failed, attempt 0/25, error InvalidParameterValue: All Transit Gateway CIDR blocks must be a valid IPv4 or IPv6 CIDR and align with the first address in the block. Invalid block:
status code: 400, request id: 0eeef407-9082-48a8-bbd7-d9c9eebfe8af: timestamp=2021-09-14T14:22:20.507-0400

Potential fix:
terraform-provider-aws/aws/resource_aws_ec2_transit_gateway.go

if d.HasChange("transit_gateway_cidr_blocks") {
		oldTransitGatewayCidrBlocks, newTransitGatewayCidrBlocks := d.GetChange("transit_gateway_cidr_blocks")

		transitGatewayModified = true
		AddTransitGatewayCidrBlocks := expandStringSet(newTransitGatewayCidrBlocks.(*schema.Set))
		if len(AddTransitGatewayCidrBlocks) > 0 {
			options.AddTransitGatewayCidrBlocks = AddTransitGatewayCidrBlocks
		}
		RemoveTransitGatewayCidrBlocks := expandStringSet(oldTransitGatewayCidrBlocks.(*schema.Set))
		if len(RemoveTransitGatewayCidrBlocks) > 0 {
			options.RemoveTransitGatewayCidrBlocks = RemoveTransitGatewayCidrBlocks
		}
}

[b-b3rn4rd]

Heya, thanks for helping testing with MR, I can confirm that the problem existed and was fixed with the suggested approach.

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_ec2_transit_gateway.example will be updated in-place
  ~ resource "aws_ec2_transit_gateway" "example" {
        id                                 = "tgw-078485988493db900"
        tags                               = {}
      ~ transit_gateway_cidr_blocks        = [
          - "10.0.0.0/24",
        ]
        # (12 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_ec2_transit_gateway.example: Modifying... [id=tgw-078485988493db900]
aws_ec2_transit_gateway.example: Modifications complete after 1s [id=tgw-078485988493db900]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
admzjs7@MacBook-Pro terraform-test % terraform plan 
aws_ec2_transit_gateway.example: Refreshing state... [id=tgw-078485988493db900]

No changes. Your infrastructure matches the configuration.Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following
symbols:
  ~ update in-place

Terraform will perform the following actions:

  # aws_ec2_transit_gateway.example will be updated in-place
  ~ resource "aws_ec2_transit_gateway" "example" {
        id                                 = "tgw-078485988493db900"
        tags                               = {}
      ~ transit_gateway_cidr_blocks        = [
          - "10.0.0.0/24",
        ]
        # (12 unchanged attributes hidden)
    }

Plan: 0 to add, 1 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value: yes

aws_ec2_transit_gateway.example: Modifying... [id=tgw-078485988493db900]
aws_ec2_transit_gateway.example: Modifications complete after 1s [id=tgw-078485988493db900]

Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
% terraform plan 
aws_ec2_transit_gateway.example: Refreshing state... [id=tgw-078485988493db900]

No changes. Your infrastructure matches the configuration.

[zhelding]

Pull request #21306 has significantly refactored the AWS Provider codebase. As a result, most PRs opened prior to the refactor now have merge conflicts that must be resolved before proceeding.

Specifically, PR #21306 relocated the code for all AWS resources and data sources from a single aws directory to a large number of separate directories in internal/service, each corresponding to a particular AWS service. This separation of code has also allowed for us to simplify the names of underlying functions -- while still avoiding namespace collisions.

We recognize that many pull requests have been open for some time without yet being addressed by our maintainers. Therefore, we want to make it clear that resolving these conflicts in no way affects the prioritization of a particular pull request. Once a pull request has been prioritized for review, the necessary changes will be made by a maintainer -- either directly or in collaboration with the pull request author.

For a more complete description of this refactor, including examples of how old filepaths and function names correspond to their new counterparts: please refer to issue #20000.

For a quick guide on how to amend your pull request to resolve the merge conflicts resulting from this refactor and bring it in line with our new code patterns: please refer to our Service Package Refactor Pull Request Guide.

[ewbankkit]

Superseded by #22181.

[ewbankkit]

@b-b3rn4rd Thanks for the contribution 🎉 👏.

[github-actions]

I'm going to lock this pull request because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems related to this change, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.