Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(ecs): add BaseService.fromServiceArnWithCluster() for use in CodePipeline #18530

Merged
merged 13 commits into from
Jan 22, 2022
Merged
29 changes: 29 additions & 0 deletions packages/@aws-cdk/aws-codepipeline-actions/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -764,6 +764,35 @@ const deployStage = pipeline.addStage({

[image definition file]: https://docs.aws.amazon.com/codepipeline/latest/userguide/pipelines-create.html#pipelines-create-image-definitions

#### Deploying ECS applications with existing ECS service ARN
skinny85 marked this conversation as resolved.
Show resolved Hide resolved

CodePipeline can deploy an ECS service which [cluster formatted ECS service ARN](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-account-settings.html#ecs-resource-ids) this can be deploy across region and accounts as well using the ARN.
skinny85 marked this conversation as resolved.
Show resolved Hide resolved

```ts
import * as ecs from '@aws-cdk/aws-ecs';

const service = ecs.BaseService.fromServiceArnWithCluster(this, 'EcsService',
'arn:aws:ecs:us-east-1:123456789012:service/myClusterName/myServiceName'
);
const pipeline = new codepipeline.Pipeline(this, 'MyPipeline');
const buildOutput = new codepipeline.Artifact();
const deployStage = pipeline.addStage({
skinny85 marked this conversation as resolved.
Show resolved Hide resolved
stageName: 'Deploy',
actions: [
new codepipeline_actions.EcsDeployAction({
actionName: 'DeployAction',
service: service,
input: buildOutput,
}),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might want to add an note in here that for cross accounts you will want to provide the role, because if not a role name will be generated for you which doesn't actually get created, what do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, not sure I understand what you mean...?

If the Action is cross-account, a new Role, with a well-known name, will be generated for you in the correct account (if you don't specify the role property).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you don't provide role on the ECSDeployAction a roleArn will be provided in the code pipeline actions for you.
If you are deploying across accounts, that role will most likely not exist in the other account.

From my ecs deploy action test.

                RoleArn: {
                  'Fn::Join': [
                    '',
                    [
                      'arn:',
                      {
                        Ref: 'AWS::Partition',
                      },
                      ':iam::service-account:role/pipelinestack-support-serloyecsactionrole49867f847238c85af7c0',
                    ],
                  ],

In order for the deployment to work I would need to create an IAM Role with the name pipelinestack-support-serloyecsactionrole49867f847238c85af7c0 in the service account, in order for the pipeline to be able to deploy.

In a self-mutating CDK pipeline the self mutation will actually fail because the account doesn't exist so it can't update the Policy for the KMS key, bucket and Pipeline Role's Policy. I would guess an normal pipeline might have the same issue as well.

Providing an IAM role which is already created in the other account resolves that, typically we have been leveraging the CDK bootstrap deploy role and adding permissions to deploy ECS matching this is typically how we have been working around this. Write up from other PR on role.

Copy link
Contributor

@skinny85 skinny85 Jan 21, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I mean, that's not just some random Role name that we just make up 🙂. There is another Stack in the application that gets generated, in the target account, that contains that Role, and the Pipeline Stack depends on that Stack, so cdk deploy PipelineStack should correctly include it, deploy it first, and then everything should work.

If it doesn't, that's a bug we need to fix.

However, it's fine if you want to mention the Role in the ReadMe.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting, I guess I never noticed that other support stack being created for the service account, to create that role. I have see the other stack which is in the other region from the pipeline which gets generated in the pipeline account. I added a little blurb about the role, I know I have seen issues with the CDK pipeline when adding a new region having an issue if the role referenced hasn't been created.

],
});
```

When deploying across accounts especially in a CDK self mutating pipeline it is recommended to provide
the `role` on the `EcsDeployAction`. The role will need to have permissions assigned to it for
ECS deployment see [EcsDeployAction](https://github.com/aws/aws-cdk/blob/d735ce4bc9049b68b830641d68c97f4be565968d/packages/%40aws-cdk/aws-codepipeline-actions/lib/ecs/deploy-action.ts#L85-L110) for proper permissions to apply to the role.
skinny85 marked this conversation as resolved.
Show resolved Hide resolved

skinny85 marked this conversation as resolved.
Show resolved Hide resolved

#### Deploying ECS applications stored in a separate source code repository

The idiomatic CDK way of deploying an ECS application is to have your Dockerfiles and your CDK code in the same source code repository,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,84 @@ describe('ecs deploy action', () => {


});

test('can be created by existing service with cluster ARN format', () => {
const app = new cdk.App();
const stack = new cdk.Stack(app, 'PipelineStack', {
env: {
region: 'pipeline-region', account: 'pipeline-account',
},
});
const clusterName = 'cluster-name';
const serviceName = 'service-name';
const region = 'service-region';
const account = 'service-account';
const serviceArn = `arn:aws:ecs:${region}:${account}:service/${clusterName}/${serviceName}`;
const service = ecs.BaseService.fromServiceArnWithCluster(stack, 'FargateService', serviceArn);

const artifact = new codepipeline.Artifact('Artifact');
const bucket = new s3.Bucket(stack, 'PipelineBucket', {
versioned: true,
removalPolicy: cdk.RemovalPolicy.DESTROY,
});
const source = new cpactions.S3SourceAction({
actionName: 'Source',
output: artifact,
bucket,
bucketKey: 'key',
});
const action = new cpactions.EcsDeployAction({
actionName: 'ECS',
service: service,
input: artifact,
});
new codepipeline.Pipeline(stack, 'Pipeline', {
stages: [
{
stageName: 'Source',
actions: [source],
},
{
stageName: 'Deploy',
actions: [action],
},
],
});

Template.fromStack(stack).hasResourceProperties('AWS::CodePipeline::Pipeline', {
Stages: [
{},
{
Actions: [
{
Name: 'ECS',
ActionTypeId: {
Category: 'Deploy',
Provider: 'ECS',
},
Configuration: {
ClusterName: clusterName,
ServiceName: serviceName,
},
Region: region,
RoleArn: {
'Fn::Join': [
'',
[
'arn:',
{
Ref: 'AWS::Partition',
},
`:iam::${account}:role/pipelinestack-support-serloyecsactionrole49867f847238c85af7c0`,
],
],
},
},
],
},
],
});
});
});
});

Expand Down
9 changes: 9 additions & 0 deletions packages/@aws-cdk/aws-ecs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,6 +96,15 @@ const cluster = new ecs.Cluster(this, 'Cluster', {
});
```

The following code imports an existing cluster using the ARN which can be used to
import an Amazon ECS service either EC2 or Fargate.

```ts
const clusterArn = 'arn:aws:ecs:us-east-1:012345678910:cluster/clusterName';

const cluster = ecs.Cluster.fromClusterArn(this, 'Cluster', clusterArn);
```
skinny85 marked this conversation as resolved.
Show resolved Hide resolved

To use tasks with Amazon EC2 launch-type, you have to add capacity to
the cluster in order for tasks to be scheduled on your instances. Typically,
you add an AutoScalingGroup with instances running the latest
Expand Down
44 changes: 42 additions & 2 deletions packages/@aws-cdk/aws-ecs/lib/base/base-service.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,10 +5,10 @@ import * as elb from '@aws-cdk/aws-elasticloadbalancing';
import * as elbv2 from '@aws-cdk/aws-elasticloadbalancingv2';
import * as iam from '@aws-cdk/aws-iam';
import * as cloudmap from '@aws-cdk/aws-servicediscovery';
import { Annotations, Duration, IResolvable, IResource, Lazy, Resource, Stack } from '@aws-cdk/core';
import { Annotations, Duration, IResolvable, IResource, Lazy, Resource, Stack, ArnFormat } from '@aws-cdk/core';
import { Construct } from 'constructs';
import { LoadBalancerTargetOptions, NetworkMode, TaskDefinition } from '../base/task-definition';
import { ICluster, CapacityProviderStrategy, ExecuteCommandLogging } from '../cluster';
import { ICluster, CapacityProviderStrategy, ExecuteCommandLogging, Cluster } from '../cluster';
import { ContainerDefinition, Protocol } from '../container-definition';
import { CfnService } from '../ecs.generated';
import { ScalableTaskCount } from './scalable-task-count';
Expand Down Expand Up @@ -315,6 +315,46 @@ export interface IBaseService extends IService {
*/
export abstract class BaseService extends Resource
implements IBaseService, elbv2.IApplicationLoadBalancerTarget, elbv2.INetworkLoadBalancerTarget, elb.ILoadBalancerTarget {
/**
* Import an existing ECS/Fargate Service using the service cluster format.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a note in the docs that the service ARN must be in the "new" format, with maybe a link to the AWS docs about it.

* The format is the "new" format "arn:aws:ecs:region:aws_account_id:service/cluster-name/service-name"
* see https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-account-settings.html#ecs-resource-ids
skinny85 marked this conversation as resolved.
Show resolved Hide resolved
*/
public static fromServiceArnWithCluster(scope: Construct, id: string, serviceArn: string): IBaseService {
const stack = Stack.of(scope);
const arn = stack.splitArn(serviceArn, ArnFormat.SLASH_RESOURCE_NAME);
const resourceName = arn.resourceName;
if (!resourceName) {
throw new Error('Missing resource Name from service ARN: ${serviceArn}');
}
const resourceNameParts = resourceName.split('/');
if (resourceNameParts.length !== 2) {
throw new Error(`resource name ${resourceName} from service ARN: ${serviceArn} is not using the ARN cluster format`);
}
const clusterName = resourceNameParts[0];
const serviceName = resourceNameParts[1];

const clusterArn = Stack.of(scope).formatArn({
partition: arn.partition,
region: arn.region,
account: arn.account,
service: 'ecs',
resource: 'cluster',
resourceName: clusterName,
});

const cluster = Cluster.fromClusterArn(scope, `${id}Cluster`, clusterArn);

class Import extends Resource implements IBaseService {
public readonly serviceArn = serviceArn;
public readonly serviceName = serviceName;
public readonly cluster = cluster;
}

return new Import(scope, id, {
environmentFromArn: serviceArn,
});
}

/**
* The security groups which manage the allowed network traffic for the service.
Expand Down
37 changes: 36 additions & 1 deletion packages/@aws-cdk/aws-ecs/lib/cluster.ts
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ import * as kms from '@aws-cdk/aws-kms';
import * as logs from '@aws-cdk/aws-logs';
import * as s3 from '@aws-cdk/aws-s3';
import * as cloudmap from '@aws-cdk/aws-servicediscovery';
import { Duration, Lazy, IResource, Resource, Stack, Aspects, IAspect, IConstruct } from '@aws-cdk/core';
import { Duration, Lazy, IResource, Resource, Stack, Aspects, IAspect, IConstruct, ArnFormat } from '@aws-cdk/core';
import { Construct } from 'constructs';
import { BottleRocketImage, EcsOptimizedAmi } from './amis';
import { InstanceDrainHook } from './drain-hook/instance-drain-hook';
Expand Down Expand Up @@ -105,6 +105,41 @@ export class Cluster extends Resource implements ICluster {
return new ImportedCluster(scope, id, attrs);
}

/**
* Import an existing cluster to the stack from the cluster ARN.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a note in these docs that the Cluster returned does not allow accessing the vpc property.

* This does not provide access to the vpc, hasEc2Capacity, or connections use fromClusterAttributes to
* access those properties.
skinny85 marked this conversation as resolved.
Show resolved Hide resolved
*/
public static fromClusterArn(scope: Construct, id: string, clusterArn: string): ICluster {
const stack = Stack.of(scope);
const arn = stack.splitArn(clusterArn, ArnFormat.SLASH_RESOURCE_NAME);
const clusterName = arn.resourceName;

if (!clusterName) {
throw new Error(`Missing required Cluster Name from Cluster ARN: ${clusterArn}`);
}

const errorSuffix = 'is not available for a Cluster imported using fromClusterArn(), please use fromClusterAttributes() instead.';

class Import extends Resource implements ICluster {
public readonly clusterArn = clusterArn;
public readonly clusterName = clusterName!;
get hasEc2Capacity(): boolean {
throw new Error(`hasEc2Capacity ${errorSuffix}`);
}
get connections(): ec2.Connections {
throw new Error(`connections ${errorSuffix}`);
}
get vpc(): ec2.IVpc {
throw new Error(`vpc ${errorSuffix}`);
}
}
tobytipton marked this conversation as resolved.
Show resolved Hide resolved

return new Import(scope, id, {
environmentFromArn: clusterArn,
});
}

/**
* Manage the allowed network connections for the cluster with Security Groups.
*/
Expand Down
44 changes: 44 additions & 0 deletions packages/@aws-cdk/aws-ecs/test/base-service.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
import * as cdk from '@aws-cdk/core';
import * as ecs from '../lib';

let stack: cdk.Stack;

beforeEach(() => {
stack = new cdk.Stack();
});

describe('When import an ECS Service', () => {
test('with serviceArnWithCluster', () => {
// GIVEN
const clusterName = 'cluster-name';
const serviceName = 'my-http-service';
const region = 'service-region';
const account = 'service-account';
const serviceArn = `arn:aws:ecs:${region}:${account}:service/${clusterName}/${serviceName}`;

// WHEN
const service = ecs.BaseService.fromServiceArnWithCluster(stack, 'Service', serviceArn);

// THEN
expect(service.serviceArn).toEqual(serviceArn);
expect(service.serviceName).toEqual(serviceName);
expect(service.env.account).toEqual(account);
expect(service.env.region).toEqual(region);

expect(service.cluster.clusterName).toEqual(clusterName);
expect(service.cluster.env.account).toEqual(account);
expect(service.cluster.env.region).toEqual(region);
});

test('throws an expection if no resourceName provided on fromServiceArnWithCluster', () => {
expect(() => {
ecs.BaseService.fromServiceArnWithCluster(stack, 'Service', 'arn:aws:ecs:service-region:service-account:service');
}).toThrowError(/Missing resource Name from service ARN/);
});

test('throws an expection if not using cluster arn format on fromServiceArnWithCluster', () => {
expect(() => {
ecs.BaseService.fromServiceArnWithCluster(stack, 'Service', 'arn:aws:ecs:service-region:service-account:service/my-http-service');
}).toThrowError(/is not using the ARN cluster format/);
});
});
24 changes: 24 additions & 0 deletions packages/@aws-cdk/aws-ecs/test/cluster.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2140,6 +2140,30 @@ describe('cluster', () => {


});

test('When importing ECS Cluster via Arn', () => {
// GIVEN
const stack = new cdk.Stack();
const clusterName = 'my-cluster';
const region = 'service-region';
const account = 'service-account';
const cluster = ecs.Cluster.fromClusterArn(stack, 'Cluster', `arn:aws:ecs:${region}:${account}:cluster/${clusterName}`);

// THEN
expect(cluster.clusterName).toEqual(clusterName);
expect(cluster.env.region).toEqual(region);
expect(cluster.env.account).toEqual(account);
});

test('throws error when import ECS Cluster without resource name in arn', () => {
// GIVEN
const stack = new cdk.Stack();

// THEN
expect(() => {
ecs.Cluster.fromClusterArn(stack, 'Cluster', 'arn:aws:ecs:service-region:service-account:cluster');
}).toThrowError(/Missing required Cluster Name from Cluster ARN: /);
});
});

test('can add ASG capacity via Capacity Provider by not specifying machineImageType', () => {
Expand Down