tests/provider: Removing Default VPC Default Subnets Breaks Certain Testing

[bflad]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

When the EC2 Default Subnet for an availability zone is deleted, certain acceptance testing is not able to pass. In general, resource acceptance testing should not rely on the Default VPC and Default Subnets except in certain explicit test cases (such as tests for EC2-Classic and Default VPC behavior) since they are a shared resource.

Workaround

A quick fix is recreating the EC2 Default Subnet in the availability zone and applying it to the various default subnet groups for other AWS services:

aws ec2 create-default-subnet --availability-zone us-west-2b

DEFAULT_EC2_SUBNET_A=$(aws ec2 describe-subnets --filters 'Name=default-for-az,Values=true' 'Name=availability-zone,Values=us-west-2a' --output text --query 'Subnets[].SubnetId')
DEFAULT_EC2_SUBNET_B=$(aws ec2 describe-subnets --filters 'Name=default-for-az,Values=true' 'Name=availability-zone,Values=us-west-2b' --output text --query 'Subnets[].SubnetId')
DEFAULT_EC2_SUBNET_C=$(aws ec2 describe-subnets --filters 'Name=default-for-az,Values=true' 'Name=availability-zone,Values=us-west-2c' --output text --query 'Subnets[].SubnetId')

aws dms modify-replication-subnet-group --replication-subnet-group-identifier default --subnet-ids $DEFAULT_EC2_SUBNET_A $DEFAULT_EC2_SUBNET_B $DEFAULT_EC2_SUBNET_C
aws docdb modify-db-subnet-group --db-subnet-group-name default --subnet-ids $DEFAULT_EC2_SUBNET_A $DEFAULT_EC2_SUBNET_B $DEFAULT_EC2_SUBNET_C
aws neptune modify-db-subnet-group --db-subnet-group-name default --subnet-ids $DEFAULT_EC2_SUBNET_A $DEFAULT_EC2_SUBNET_B $DEFAULT_EC2_SUBNET_C
aws rds modify-db-subnet-group --db-subnet-group-name default --subnet-ids $DEFAULT_EC2_SUBNET_A $DEFAULT_EC2_SUBNET_B $DEFAULT_EC2_SUBNET_C
aws redshift modify-cluster-subnet-group --cluster-subnet-group-name default --subnet-ids $DEFAULT_EC2_SUBNET_A $DEFAULT_EC2_SUBNET_B $DEFAULT_EC2_SUBNET_C

Tests Affected

--- FAIL: TestAccAWSASGNotification_update (780.37s)

--- FAIL: TestAccAWSAutoscalingAttachment_elb (42.64s)

--- FAIL: TestAccAWSDBInstance_CACertificateIdentifier (126.23s)
--- FAIL: TestAccAWSDBInstance_EnabledCloudwatchLogsExports_Oracle (237.54s)
--- FAIL: TestAccAWSDBInstance_FinalSnapshotIdentifier (106.75s)
--- FAIL: TestAccAWSDBInstance_FinalSnapshotIdentifier_SkipFinalSnapshot (136.67s)
--- FAIL: TestAccAWSDBInstance_MSSQL_DomainSnapshotRestore (1792.19s)
--- FAIL: TestAccAWSDBInstance_MinorVersion (207.15s)
--- FAIL: TestAccAWSDBInstance_MonitoringInterval (140.44s)
--- FAIL: TestAccAWSDBInstance_Password (117.85s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb (996.92s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_AllocatedStorage (136.36s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_AutoMinorVersionUpgrade (117.17s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_AvailabilityZone (158.28s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_CACertificateIdentifier (156.96s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_IamDatabaseAuthenticationEnabled (1026.81s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_MaxAllocatedStorage (997.18s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_Monitoring (128.89s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_MultiAZ (1321.34s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_ParameterGroupName (946.15s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_PerformanceInsightsEnabled (987.06s)
--- FAIL: TestAccAWSDBInstance_ReplicateSourceDb_VpcSecurityGroupIds (118.38s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_AllowMajorVersionUpgrade (763.19s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_BackupRetentionPeriod (136.06s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_Io1Storage (126.21s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_MaintenanceWindow (126.49s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_Monitoring (816.41s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_MultiAZ_SQLServer (146.66s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_ParameterGroupName (844.10s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_Port (126.29s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_VpcSecurityGroupIds (716.04s)
--- FAIL: TestAccAWSDBInstance_SnapshotIdentifier_VpcSecurityGroupIds_Tags (310.82s)
--- FAIL: TestAccAWSDBInstance_portUpdate (177.00s)
--- FAIL: TestAccAWSDBInstance_separate_iops_update (146.50s)

--- FAIL: TestAccAWSDBOptionGroup_basicDestroyWithInstance (146.62s)

--- FAIL: TestAccAWSDBSnapshot_tags (115.70s)

--- FAIL: TestAccAWSDmsReplicationInstance_AllocatedStorage (117.59s)
--- FAIL: TestAccAWSDmsReplicationInstance_Basic (116.38s)
--- FAIL: TestAccAWSDmsReplicationInstance_EngineVersion (434.33s)
--- FAIL: TestAccAWSDmsReplicationInstance_PreferredMaintenanceWindow (118.14s)
--- FAIL: TestAccAWSDmsReplicationInstance_PubliclyAccessible (117.47s)
--- FAIL: TestAccAWSDmsReplicationInstance_ReplicationInstanceClass (126.61s)
--- FAIL: TestAccAWSDmsReplicationInstance_Tags (247.93s)

--- FAIL: TestAccAWSDocDBClusterInstance_az (4.87s)
--- FAIL: TestAccAWSDocDBClusterInstance_basic (3.28s)
--- FAIL: TestAccAWSDocDBClusterInstance_disappears (4.13s)
--- FAIL: TestAccAWSDocDBClusterInstance_kmsKey (5.17s)
--- FAIL: TestAccAWSDocDBCluster_Port (6.29s)
--- FAIL: TestAccAWSDocDBCluster_backupsUpdate (3.91s)
--- FAIL: TestAccAWSDocDBCluster_basic (3.32s)
--- FAIL: TestAccAWSDocDBCluster_encrypted (3.02s)
--- FAIL: TestAccAWSDocDBCluster_kmsKey (5.18s)
--- FAIL: TestAccAWSDocDBCluster_takeFinalSnapshot (2.99s)
--- FAIL: TestAccAWSDocDBCluster_updateCloudwatchLogsExports (3.88s)
--- FAIL: TestAccAWSDocDBCluster_updateTags (3.83s)

--- FAIL: TestAccAWSELBAttachment_basic (175.68s)
--- FAIL: TestAccAWSELBAttachment_drift (175.82s)

--- FAIL: TestAccAWSELBUpdate_HealthCheck (8.03s)
--- FAIL: TestAccAWSELB_AccessLogs_disabled (6.00s)
--- FAIL: TestAccAWSELB_AccessLogs_enabled (5.77s)
--- FAIL: TestAccAWSELB_HealthCheck (8.07s)
--- FAIL: TestAccAWSELB_InstanceAttaching (7.47s)
--- FAIL: TestAccAWSELB_SecurityGroups (10.93s)
--- FAIL: TestAccAWSELB_availabilityZones (5.93s)
--- FAIL: TestAccAWSELB_basic (4.40s)
--- FAIL: TestAccAWSELB_disappears (5.84s)
--- FAIL: TestAccAWSELB_fullCharacterRange (6.12s)
--- FAIL: TestAccAWSELB_generatedName (5.87s)
--- FAIL: TestAccAWSELB_generatesNameForZeroValue (5.92s)
--- FAIL: TestAccAWSELB_listener (17.87s)
--- FAIL: TestAccAWSELB_namePrefix (5.88s)
--- FAIL: TestAccAWSELB_tags (5.84s)

--- FAIL: TestAccAWSNeptuneClusterInstance_basic (5.60s)
--- FAIL: TestAccAWSNeptuneClusterInstance_kmsKey (5.25s)
--- FAIL: TestAccAWSNeptuneClusterInstance_withaz (6.70s)
--- FAIL: TestAccAWSNeptuneCluster_backupsUpdate (4.68s)
--- FAIL: TestAccAWSNeptuneCluster_basic (4.78s)
--- FAIL: TestAccAWSNeptuneCluster_deleteProtection (6.16s)
--- FAIL: TestAccAWSNeptuneCluster_encrypted (4.73s)
--- FAIL: TestAccAWSNeptuneCluster_iamAuth (4.84s)
--- FAIL: TestAccAWSNeptuneCluster_kmsKey (6.25s)
--- FAIL: TestAccAWSNeptuneCluster_tags (6.52s)
--- FAIL: TestAccAWSNeptuneCluster_takeFinalSnapshot (4.90s)
--- FAIL: TestAccAWSNeptuneCluster_updateCloudwatchLogsExports (4.79s)
--- FAIL: TestAccAWSNeptuneCluster_updateIamRoles (8.13s)

--- FAIL: TestAccAWSOpsworksRdsDbInstance (157.71s)

--- FAIL: TestAccAWSRDSClusterEndpoint_basic (3.30s)
--- FAIL: TestAccAWSRDSClusterEndpoint_tags (4.38s)

--- FAIL: TestAccAWSRDSClusterInstance_CopyTagsToSnapshot (4.48s)
--- FAIL: TestAccAWSRDSClusterInstance_MonitoringInterval (230.84s)
--- FAIL: TestAccAWSRDSClusterInstance_MonitoringRoleArn_EnabledToRemoved (370.59s)
--- FAIL: TestAccAWSRDSClusterInstance_MonitoringRoleArn_RemovedToEnabled (231.93s)
--- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql1 (268.81s)
--- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql1_DefaultKeyToCustomKey (248.84s)
--- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraMysql2 (248.46s)
--- FAIL: TestAccAWSRDSClusterInstance_PerformanceInsightsKmsKeyId_AuroraPostgresql_DefaultKeyToCustomKey (248.41s)
--- FAIL: TestAccAWSRDSClusterInstance_az (6.31s)
--- FAIL: TestAccAWSRDSClusterInstance_basic (5.63s)
--- FAIL: TestAccAWSRDSClusterInstance_disappears (5.56s)
--- FAIL: TestAccAWSRDSClusterInstance_isAlreadyBeingDeleted (6.10s)
--- FAIL: TestAccAWSRDSClusterInstance_kmsKey (5.66s)

--- FAIL: TestAccAWSRDSCluster_AvailabilityZones (4.81s)

--- FAIL: TestAccAWSRedshiftCluster_changeAvailabilityZone (459.15s)

--- FAIL: TestAccAWSSpotFleetRequest_lowestPriceAzInGivenList (611.59s)

--- FAIL: TestAccDataSourceAwsElasticacheReplicationGroup_basic (4.85s)

[ewbankkit]

Related:

• r/aws_default_subnet: Full resource lifecycle #4957
• r/aws_default_vpc: Full resource lifecycle #4959

I hadn't thought of the case of a default VPC without default subnets when I added testAccPreCheckHasDefaultVpcOrEc2Classic()

https://github.com/terraform-providers/terraform-provider-aws/blob/e5392133e1d1950c1e129b8103fb1f5d377a0805/aws/provider_test.go#L1226-L1228

I should probably tighten that up for the cases where we are explicitly testing launching an instance without specifying a subnet ID.

[ewbankkit]

A good approach here would be to make those tests that aren't explicitly testing default VPC or EC2-Classic related functionality hermetic and create a VPC and associated resources as part of the test.
For those tests that do require a default VPC, tighten up the pre-flight tests to verify as many preconditions as possible.

[ewbankkit]

There are also cases where there are default subnets but no default Internet Gateway:

--- FAIL: TestAccAWSRoute53Record_weighted_alias (70.92s)
    testing.go:684: Step 0 error: errors during apply:
        
        Error: Error creating ELB: InvalidConfigurationRequest: Issue found with the default subnets: VPC vpc-b6fbb6cf has no internet gateway
        	status code: 409, request id: fcb5fe16-3fb6-4b9e-8724-a85b328cdd66
        
          on /tmp/tf-test003405979/main.tf line 44:
          (source code not available)
        
        
        
        Error: Error creating ELB: InvalidConfigurationRequest: Issue found with the default subnets: VPC vpc-b6fbb6cf has no internet gateway
        	status code: 409, request id: 109ad163-1974-424d-918e-c41ac99ff972
        
          on /tmp/tf-test003405979/main.tf line 15:
          (source code not available)

--- FAIL: TestAccAWSRoute53Record_weighted_alias
--- FAIL: TestAccAWSRoute53Record_Alias_Uppercase
--- FAIL: TestAccAWSRoute53Record_AliasChange
--- FAIL: TestAccAWSRoute53Record_Alias_Elb

[github-actions]

Marking this issue as stale due to inactivity. This helps our maintainers find and focus on the active issues. If this issue receives no comments in the next 30 days it will automatically be closed. Maintainers can also remove the stale label.

If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.