Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(aws-cdk): Auto-delete stacks that failed creating before new attempt #917

Merged
merged 1 commit into from
Oct 12, 2018
Merged

fix(aws-cdk): Auto-delete stacks that failed creating before new attempt #917

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion packages/aws-cdk/lib/api/deploy-stack.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ import { prepareAssets } from '../assets';
import { debug, error } from '../logging';
import { Mode } from './aws-auth/credentials';
import { ToolkitInfo } from './toolkit-info';
import { describeStack, stackExists, waitForChangeSet, waitForStack } from './util/cloudformation';
import { describeStack, stackExists, stackFailedCreating, waitForChangeSet, waitForStack } from './util/cloudformation';
import { StackActivityMonitor } from './util/cloudformation/stack-activity-monitor';
import { StackStatus } from './util/cloudformation/stack-status';
import { SDK } from './util/sdk';
Expand Down Expand Up @@ -43,6 +43,15 @@ export async function deployStack(stack: cxapi.SynthesizedStack,
const cfn = await sdk.cloudFormation(stack.environment, Mode.ForWriting);
const bodyParameter = await makeBodyParameter(stack, toolkitInfo);

if (await stackFailedCreating(cfn, deployName)) {
debug(`Found existing stack ${deployName} that had previously failed creation. Deleting it before attempting to re-create it.`);
await cfn.deleteStack({ StackName: deployName }).promise();
const deletedStack = await waitForStack(cfn, deployName, false);
if (deletedStack && deletedStack.StackStatus !== 'DELETE_COMPLETE') {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see how we can ever get DELETE_COMPLETE state. To get the status of a deleted stack you need the full ARN (including unique ID), but deployName is amost certain to not include that? Or does it?

I assume this works by returning undefined as soon as the stack can't be fetched by just its name anymore?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's correct. The test here is "pedantically correct" (basically it won't break if waitForStack resolves to the status of the stack in a deleted-stack-friendly way.

throw new Error(`Failed deleting stack ${deployName} that had previously failed creation (current state: ${deletedStack.StackStatus})`);
}
}

const update = await stackExists(cfn, deployName);

const changeSetName = `CDK-${executionId}`;
Expand Down
14 changes: 14 additions & 0 deletions packages/aws-cdk/lib/api/util/cloudformation.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,20 @@ export async function stackExists(cfn: CloudFormation, stackName: string): Promi
return description !== undefined;
}

/**
* Checks whether a stack has failed creation in CloudFormation. This is identified by the current stack Status being
* ``ROLLBACK_COMPLETE``.
*
* @param cfn a CloudFormation client
* @param stackName the name of the stack to be checked for
*
* @returns +true+ if the stack exists and is in failed-creation state.
*/
export async function stackFailedCreating(cfn: CloudFormation, stackName: string): Promise<boolean> {
const description = await describeStack(cfn, stackName);
return description != null && description.StackStatus === 'ROLLBACK_COMPLETE';
}

/**
* Waits for a function to return non-+undefined+ before returning.
*
Expand Down