Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

(codepipeline) cross-region-stack-* causes cdk diff to fail for users who are only allowed to assume the lookup-role #25903

Closed
lestephane opened this issue Jun 8, 2023 · 3 comments
Assignees
Labels
@aws-cdk/aws-codepipeline Related to AWS CodePipeline bug This issue is a bug. closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. effort/medium Medium work item – several days of effort p1 response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days.

Comments

@lestephane
Copy link

lestephane commented Jun 8, 2023

Describe the bug

I've been unable to cdk diff my cdk app since I've added another region (to deploy certificates to us-east-1), because I follow AWS best practices of least privileged access, and only have the permission to assume the lookup-role in all the account / region pairs I need to work with. And BootstraplessSynthesizer does not use the lookup-role at all.

Expected Behavior

  • The lookup-role is assumed to determine the diff for the cross-support stack FIRST and the deploy-role is then assumed as fallback. Like it's done elsewhere.
  • cdk diff succeeds

Current Behavior

  • The deploy-role is assumed first
  • The cdk diff fails

Reproduction Steps

  • deploy a cdkpipeline (IacPipeline) in iac account A1 region R1 (R1 being geographically close, such as eu-central-1 for german customers), having first cdk bootstrap-ed A1R1 using privileged credentials.
  • add locally (on a dev machine) a new stack definition for another account A2 region R2 (R2 being us-east-1as is often required for cloudfront ceritifcates), having first cdk bootstrap-ed A2R2 using privileged credentials. The stack is not deployed yet.
  • the dev credentials are that of an A1 account role (iac-dev-role) whose only permissions are to allow AssumeRole of the cdk lookup-role for A1R1 and A2R2
  • a (least privileged) dev user who can only assume the lookup-role A1R1 and A2R2 runs cdk diff '**'
  • the command fails:
Stack cross-region-stack-A1:R2 (IacPipeline-support-R2)
current credentials could not be used to assume 'arn:aws:iam::A1:role/cdk-hnb659fds-deploy-role-A1-R1', but are for the right account. Proceeding anyway.

User: arn:aws:sts::A1:assumed-role/iac-dev-role/... is not authorized to perform: cloudformation:DescribeStacks on resource: arn:aws:cloudformation:R2:A1:stack/IacPipeline-support-us-east-1/* because no identity-based policy allows the cloudformation:DescribeStacks action

Possible Solution

cdk bootstrap could be extended with a --needed flag to output all the AxRy account+region pairs where the bootstrap still needs to run. That would improve the ergonomics a bit. I realize it's impossible to figure out which role to assume in each account to check for bootstrapping. But there is at least OrganizationAccountAccessRole you could try to assume in each target account if the current credentials allow.

Having this would have made it clear that I do NOT need to cdk bootstrap A1R2, because of the BootstraplessSynthesizer

Once I realize I do NOT need to cdk bootstrap A1R2, it becomes clear that it is the bootstrap synthesizer that is to blame, because it did not attempt to assume cdk-hnb659fds-lookup-role-A1-R1 first.

It took me a few hours to figure out all this. And the use of a BootstraplessSynthesizer looks like a code smell. Why is it not OK to just include A1Ry (for each foreign region y that the pipeline intends to deploy to) as needing to be bootstrapped? That would allow you to keep a consistent behaviour everywhere. And you would then be able to focus on how to enable bootstrapping all these AxRy pairs more efficiently. The current behaviour is confusing.

Additional Information/Context

No response

CDK CLI Version

2.83.0 (build 0fd7f2b)

Framework Version

No response

Node.js Version

v16.19.0

OS

Ubuntu 22.04.2 LTS

Language

Typescript

Language Version

[email protected]

Other information

No response

@lestephane lestephane added bug This issue is a bug. needs-triage This issue or PR still needs to be triaged. labels Jun 8, 2023
@github-actions github-actions bot added the @aws-cdk/aws-codepipeline Related to AWS CodePipeline label Jun 8, 2023
@peterwoodworth
Copy link
Contributor

Can you please expand on the steps you took, I'm a bit fuzzy on the details.

  • The error message you pasted is showing A1:R2, but you never describe trying to create an A1:R2 stack, what stack exactly is failing?
  • You mention BootstraplessSynthesizer a few times. Can you please show how you're using it, and clarify if you're using it when this behavior occurs?
  • Can you share the full cdk bootstrap commands you're running?

@peterwoodworth peterwoodworth added p1 response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days. effort/medium Medium work item – several days of effort needs-review and removed needs-triage This issue or PR still needs to be triaged. labels Jun 8, 2023
@github-actions
Copy link

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

@github-actions github-actions bot added the closing-soon This issue will automatically close in 4 days unless further comments are made. label Jun 11, 2023
@github-actions github-actions bot added closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. and removed closing-soon This issue will automatically close in 4 days unless further comments are made. labels Jun 16, 2023
@lestephane
Copy link
Author

I can try to boil it down, because the 2-3 h I spent coming up with the initial description of the problem (based on the code review I did on the CDK) seems to confuse rather than help.

Two facts:

  1. i have the permission to assume the lookup role in all account / region pairs involved in my app (including the account / region of the cross-region-stack- (which cdk creates automatically through some badly documented hack). So all regions and accounts are bootstrapped.

  2. I want to perform a cdk diff which is a read-only operation, and for this, it should be sufficient to be able to assume the lookup-role everywhere. I do have the permission to assume lookup-roles in all account / region pairs that were bootstrapped.

Based on 1. and 2., I should be able to do the cdk diff, but I'm not. The CDK cli insists on assuming deploy-role for the cross-region-* support stack.

As @rix0rrr pointed out, the cross-region part is a mess, so It's not like I'm coming out of the blue with this.

I'm showing you what happens when technical debt has accumulated: I can't do a CDK diff when I should be able to, in a cross-region scenario.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
@aws-cdk/aws-codepipeline Related to AWS CodePipeline bug This issue is a bug. closed-for-staleness This issue was automatically closed because it hadn't received any attention in a while. effort/medium Medium work item – several days of effort p1 response-requested Waiting on additional info and feedback. Will move to "closing-soon" in 7 days.
Projects
None yet
Development

No branches or pull requests

4 participants