-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ecs: concurrent creation of http namespace fails if namespace is created by ECS cluster as default namespace #25554
Comments
Hi Unfortunately I can't reproduce this error. I use the code below to deploy in my default VPC. const vpc = getDefaultVpc(this);
const cluster = new ecs.Cluster(this, 'SomeCluster', { vpc });
cluster.addDefaultCloudMapNamespace({
name: 'TestNamespace',
type: servicediscovery.NamespaceType.HTTP,
useForServiceConnect: true,
}); When I run
In your provided sample, I didn't see the Did you specify |
This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled. |
No, I just specified the cluster as described in my reproduction steps above (I don't use the default VPC like you did). However, the method I don't know if that helps but here's the generated CloudFormation code. (The first two resources are the most relevant ones) Resources:
SomeClusterDefaultServiceDiscoveryNamespaceAE271633:
Type: AWS::ServiceDiscovery::HttpNamespace
Properties:
Name: TestNamespace
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/DefaultServiceDiscoveryNamespace/Resource
SomeCluster706C05D5:
Type: AWS::ECS::Cluster
Properties:
ServiceConnectDefaults:
Namespace: TestNamespace
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Resource
SomeClusterVpc5CA2B6C9:
Type: AWS::EC2::VPC
Properties:
CidrBlock: 10.0.0.0/16
EnableDnsHostnames: true
EnableDnsSupport: true
InstanceTenancy: default
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/Resource
SomeClusterVpcPublicSubnet1Subnet6904D9AF:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
CidrBlock: 10.0.0.0/18
MapPublicIpOnLaunch: true
Tags:
- Key: aws-cdk:subnet-name
Value: Public
- Key: aws-cdk:subnet-type
Value: Public
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1/Subnet
SomeClusterVpcPublicSubnet1RouteTableBF41FE7E:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1/RouteTable
SomeClusterVpcPublicSubnet1RouteTableAssociationD9DD8458:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId:
Ref: SomeClusterVpcPublicSubnet1RouteTableBF41FE7E
SubnetId:
Ref: SomeClusterVpcPublicSubnet1Subnet6904D9AF
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1/RouteTableAssociation
SomeClusterVpcPublicSubnet1DefaultRouteAB44F101:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: SomeClusterVpcPublicSubnet1RouteTableBF41FE7E
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: SomeClusterVpcIGW85473771
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
- SomeClusterVpcVPCGW6C1D677D
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1/DefaultRoute
SomeClusterVpcPublicSubnet1EIP9F872A3E:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1/EIP
SomeClusterVpcPublicSubnet1NATGatewayD532BA14:
Type: AWS::EC2::NatGateway
Properties:
SubnetId:
Ref: SomeClusterVpcPublicSubnet1Subnet6904D9AF
AllocationId:
Fn::GetAtt:
- SomeClusterVpcPublicSubnet1EIP9F872A3E
- AllocationId
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
- SomeClusterVpcPublicSubnet1DefaultRouteAB44F101
- SomeClusterVpcPublicSubnet1RouteTableAssociationD9DD8458
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet1/NATGateway
SomeClusterVpcPublicSubnet2Subnet800ABF10:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
CidrBlock: 10.0.64.0/18
MapPublicIpOnLaunch: true
Tags:
- Key: aws-cdk:subnet-name
Value: Public
- Key: aws-cdk:subnet-type
Value: Public
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2/Subnet
SomeClusterVpcPublicSubnet2RouteTableED16473D:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2/RouteTable
SomeClusterVpcPublicSubnet2RouteTableAssociationA5FF2897:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId:
Ref: SomeClusterVpcPublicSubnet2RouteTableED16473D
SubnetId:
Ref: SomeClusterVpcPublicSubnet2Subnet800ABF10
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2/RouteTableAssociation
SomeClusterVpcPublicSubnet2DefaultRoute26DB7E58:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: SomeClusterVpcPublicSubnet2RouteTableED16473D
DestinationCidrBlock: 0.0.0.0/0
GatewayId:
Ref: SomeClusterVpcIGW85473771
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
- SomeClusterVpcVPCGW6C1D677D
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2/DefaultRoute
SomeClusterVpcPublicSubnet2EIP23D54794:
Type: AWS::EC2::EIP
Properties:
Domain: vpc
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2/EIP
SomeClusterVpcPublicSubnet2NATGateway69D968A8:
Type: AWS::EC2::NatGateway
Properties:
SubnetId:
Ref: SomeClusterVpcPublicSubnet2Subnet800ABF10
AllocationId:
Fn::GetAtt:
- SomeClusterVpcPublicSubnet2EIP23D54794
- AllocationId
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
- SomeClusterVpcPublicSubnet2DefaultRoute26DB7E58
- SomeClusterVpcPublicSubnet2RouteTableAssociationA5FF2897
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PublicSubnet2/NATGateway
SomeClusterVpcPrivateSubnet1Subnet6FF65E75:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
AvailabilityZone:
Fn::Select:
- 0
- Fn::GetAZs: ""
CidrBlock: 10.0.128.0/18
MapPublicIpOnLaunch: false
Tags:
- Key: aws-cdk:subnet-name
Value: Private
- Key: aws-cdk:subnet-type
Value: Private
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet1
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet1/Subnet
SomeClusterVpcPrivateSubnet1RouteTable982B83C3:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet1
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet1/RouteTable
SomeClusterVpcPrivateSubnet1RouteTableAssociation47C277BF:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId:
Ref: SomeClusterVpcPrivateSubnet1RouteTable982B83C3
SubnetId:
Ref: SomeClusterVpcPrivateSubnet1Subnet6FF65E75
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet1/RouteTableAssociation
SomeClusterVpcPrivateSubnet1DefaultRoute5DA7FA68:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: SomeClusterVpcPrivateSubnet1RouteTable982B83C3
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId:
Ref: SomeClusterVpcPublicSubnet1NATGatewayD532BA14
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet1/DefaultRoute
SomeClusterVpcPrivateSubnet2Subnet9FFA7B2D:
Type: AWS::EC2::Subnet
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
AvailabilityZone:
Fn::Select:
- 1
- Fn::GetAZs: ""
CidrBlock: 10.0.192.0/18
MapPublicIpOnLaunch: false
Tags:
- Key: aws-cdk:subnet-name
Value: Private
- Key: aws-cdk:subnet-type
Value: Private
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet2
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet2/Subnet
SomeClusterVpcPrivateSubnet2RouteTable38317372:
Type: AWS::EC2::RouteTable
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet2
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet2/RouteTable
SomeClusterVpcPrivateSubnet2RouteTableAssociation6D40AFD5:
Type: AWS::EC2::SubnetRouteTableAssociation
Properties:
RouteTableId:
Ref: SomeClusterVpcPrivateSubnet2RouteTable38317372
SubnetId:
Ref: SomeClusterVpcPrivateSubnet2Subnet9FFA7B2D
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet2/RouteTableAssociation
SomeClusterVpcPrivateSubnet2DefaultRoute61296D70:
Type: AWS::EC2::Route
Properties:
RouteTableId:
Ref: SomeClusterVpcPrivateSubnet2RouteTable38317372
DestinationCidrBlock: 0.0.0.0/0
NatGatewayId:
Ref: SomeClusterVpcPublicSubnet2NATGateway69D968A8
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/PrivateSubnet2/DefaultRoute
SomeClusterVpcIGW85473771:
Type: AWS::EC2::InternetGateway
Properties:
Tags:
- Key: Name
Value: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/IGW
SomeClusterVpcVPCGW6C1D677D:
Type: AWS::EC2::VPCGatewayAttachment
Properties:
VpcId:
Ref: SomeClusterVpc5CA2B6C9
InternetGatewayId:
Ref: SomeClusterVpcIGW85473771
DependsOn:
- SomeClusterDefaultServiceDiscoveryNamespaceAE271633
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/SomeCluster/Vpc/VPCGW
CDKMetadata:
Type: AWS::CDK::Metadata
Properties:
Analytics: v2:deflate64:H4sIAAAAAAAA/1WQwU7DMAyGn4V7Glgl2HmqENtlqrppV5SmRnhrkyp2Ok1R3p2EDgonf/lsJX9cynUpnx7UlQrdXYoeWxkOrPRFJPUeQJMMVe+JwYnqw9wxCtClDKdRZ3mqK1H7tkd98K0Bzm6hxnqGo2p7WPziNkRWo2K05nc4w+uuzmWv+E0xXNVN1A6nhMvFO5OSJP4ZmJPcTxtOn/gcwHAUBG5CDR2SthO4mwxb5nGvBqBR6e/n/okYRQNkvZt7f7mypsMcNgpjO5BnepxWL3L1nHZ4JsTCecM4gGzm+gUze+rzXwEAAA==
Metadata:
aws:cdk:path: CdkNamespaceConcurrentOperationErrorStack/CDKMetadata/Default
Condition: CDKMetadataAvailable
Conditions:
CDKMetadataAvailable:
Fn::Or:
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- af-south-1
- Fn::Equals:
- Ref: AWS::Region
- ap-east-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-northeast-2
- Fn::Equals:
- Ref: AWS::Region
- ap-south-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-1
- Fn::Equals:
- Ref: AWS::Region
- ap-southeast-2
- Fn::Equals:
- Ref: AWS::Region
- ca-central-1
- Fn::Equals:
- Ref: AWS::Region
- cn-north-1
- Fn::Equals:
- Ref: AWS::Region
- cn-northwest-1
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- eu-central-1
- Fn::Equals:
- Ref: AWS::Region
- eu-north-1
- Fn::Equals:
- Ref: AWS::Region
- eu-south-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-1
- Fn::Equals:
- Ref: AWS::Region
- eu-west-2
- Fn::Equals:
- Ref: AWS::Region
- eu-west-3
- Fn::Equals:
- Ref: AWS::Region
- me-south-1
- Fn::Equals:
- Ref: AWS::Region
- sa-east-1
- Fn::Equals:
- Ref: AWS::Region
- us-east-1
- Fn::Equals:
- Ref: AWS::Region
- us-east-2
- Fn::Or:
- Fn::Equals:
- Ref: AWS::Region
- us-west-1
- Fn::Equals:
- Ref: AWS::Region
- us-west-2
Parameters:
BootstrapVersion:
Type: AWS::SSM::Parameter::Value<String>
Default: /cdk-bootstrap/hnb659fds/version
Description: Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]
Rules:
CheckBootstrapVersion:
Assertions:
- Assert:
Fn::Not:
- Fn::Contains:
- - "1"
- "2"
- "3"
- "4"
- "5"
- Ref: BootstrapVersion
AssertDescription: CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI. |
Looks to me like this has been fixed? In my cdk output it looks like ServiceConnectDefaults.Namespace uses the ARN of the sd namespace not the namespace name |
@lucasgadams Yeah it has been fixed with 4f60293 which references another issue #25616 describing the same problem. |
|
Describe the bug
If I create an ECS cluster and create a default CloudMap namespace using
.addDefaultCloudMapNamespace(...)
, CloudFormation tries to create the namespace twice and fails with the following error:See reproduction steps below. The problem is that calling the CDK method adds a
Namespace
property to theAWS::ECS::Cluster
resource if I setuseForServiceConnect: true
. The value of theNamespace
property is the namespace name. And according to the docs, settingNamespace
will try to create the namespace if it does not exist:AWS::ECS::Cluster Documentation
Expected Behavior
I expect that CloudFormation only tries to create one Http Namespace and the deployment does not fail.
Current Behavior
CloudFormation fails because there are two concurrent operations trying to create an Http namespace: one operation due to the
Namespace
property ofAWS::ECS::Cluster
and one operation due to theAWS::ServiceDiscovery::HttpNamespace
resource.Reproduction Steps
The following CDK stack can be used to reproduce the problem:
Possible Solution
For CDK users, a workaround is to add an explicit dependency between the cluster and namespace, e.g.:
However, I believe there are three better options:
Namespace
and having anAWS::ServiceDiscovery::HttpNamespace
resource in the same template.Additional Information/Context
One hint: While testing, somehow my namespaces were not deleted when deleting the CloudFormation stack. Hence, you might experience the error mentioned above even when setting the dependency from my suggested workaround if a namespace with the same name already exists. At least that happened to me and made the investigation a bit more complicated until I realized it 😀
CDK CLI Version
2.72.0
Framework Version
No response
Node.js Version
16.15.0
OS
MacOS
Language
Typescript
Language Version
4.9.5
Other information
No response
The text was updated successfully, but these errors were encountered: