NestedStack: more mening full diff log output

[ghost]

Describe the feature

Need a way to get diff logs out for NestedStack before i deploy CDK code to AWS

Use Case

Today when you are using multi-stacks in your CDK code its showing the diff before you will deploy, when you start using NestedStack this diff become very problematic becures its only referac til the NestedStach hash and not the full diff list over changes inside NestedStacks

The biggest issue here its if you don't can get this logs out you are working in a huge black-box when you are using NestedStack.

Proposed Solution

Solution num 1:
Use the diff log hash-key for NestedStack resoruces to read the changes out
cdk diff --nestedStackHash {diff-hash-key}

Solution num 2:
sending falg with to return all NestedStack logs back in the console
cdk diff --includeNestedStackLog

Solution num 3:
Support NestedStack with full output to json file so its can be manally parsed.
cdk diff --dumpFullJsonLog --includeNestedStack

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

CDK version used

2.24.1 (build 585f9ca)

Environment details (OS name and version, etc.)

macOS 12.3.1

[timothy-cloudopsguy]

Bump. This has been an issue for-e-ver. It's the reason we moved away from NestedStacks in CloudFormation years ago because it's similar there. You have to bend over backward to get the diffs to see the actual changes, it's like a black box waiting to explode your production environment with a resource replacement you weren't expecting.

[diranged]

Agreed... gotta bump this too, nested stack diff'ing is critical especially with a tool like CDK.

[rectalogic]

This was implemented in #18207 see #5722
Or was that all later reverted? I’m using 2.19.0 and nested stack diffs work

[ErrorsAndGlitches]

@rectalogic - I tried using version 2.38.1, which I had installed & it didn't work. I also tried installing 2.19.0, which also did not work. Here is my test code:

#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from 'aws-cdk-lib';
import { Construct } from 'constructs';
import { NestedStack, NestedStackProps } from 'aws-cdk-lib';
import { Bucket } from 'aws-cdk-lib/aws-s3';

class CdkTestStack extends cdk.Stack {
    constructor(scope: Construct, id: string, props?: cdk.StackProps) {
        super(scope, id, props);
        new TestNestedStack(this, {
            bucketName: 'bucket-test',
        });
    }
}

interface TestNestedStackProps extends NestedStackProps {
    bucketName: string
}

class TestNestedStack extends NestedStack {
    constructor(scope: Construct, props: TestNestedStackProps) {
        super(scope, 'test-nested-stack', props);

        new Bucket(this, 'test-bucket', {
            bucketName: props.bucketName,
        })
    }
}

const app = new cdk.App();
new CdkTestStack(app, 'CdkTestStack', {});

Running cdk diff produces this Resources section:

Resources
[~] AWS::CloudFormation::Stack test-nested-stack.NestedStack/test-nested-stack.NestedStackResource testnestedstackNestedStacktestnestedstackNestedStackResourceDE3D5562
 ├─ [~] NestedTemplate
 │   ├─ [+] Added: .Conditions
 │   └─ [+] Added: .Resources
 ├─ [+] TemplateURL
 │   └─ {"Fn::Join":["",["https://s3.",{"Ref":"AWS::Region"},".",{"Ref":"AWS::URLSuffix"},"/",{"Fn::Sub":"cdk-hnb659fds-assets-${AWS::AccountId}-${AWS::Region}"},"/14ee3ea6e71244758d383b10c26a6d0695e1bbe2c4ebc9a39a64bc867062eda2.json"]]}
 ├─ [+] DeletionPolicy
 │   └─ Delete
 ├─ [+] Metadata
 │   └─ {"aws:cdk:path":"CdkTestStack/test-nested-stack.NestedStack/test-nested-stack.NestedStackResource","aws:asset:path":"CdkTestStacktestnestedstack7CB2A8DF.nested.template.json","aws:asset:property":"TemplateURL"}
 └─ [+] UpdateReplacePolicy
     └─ Delete

I wanted to note that this is a new stack that I don't have deployed to my AWS account. Could that be an issue i.e. that nested stack diff doesn't work until a version of the nested stack is deployed?

[ErrorsAndGlitches]

Yup - it appears that the issue is that diff works when the nested stack already exists in the AWS account. If it's a new nested stack, diff doesn't show any differences.

We're going to try working around the issue by first deploying an empty nested stack prior to adding resources to it.

[comcalvi]

@ErrorsAndGlitches deploying the empty nested stack first produced the detailed diff?

[ErrorsAndGlitches]

@comcalvi yup. I created an empty stack, deployed, & then updated the nested stack code to have an S3 bucket. The diff then produced:

$ cdk diff
Stack CdkTestStack
Resources
[~] AWS::CloudFormation::Stack test-nested-stack.NestedStack/test-nested-stack.NestedStackResource testnestedstackNestedStacktestnestedstackNestedStackResourceDE3D5562
 ├─ [~] NestedTemplate
 │   └─ [~] .Resources:
 │       └─ [+] Added: .testbucket17BE0499D

We're trying the approach for our production stacks now. Will see if it works.

[github-actions]

This issue has not received a response in a while. If you want to keep this issue open, please leave a comment below and auto-close will be canceled.

[mauritz-lovgren]

I would also like the diff to be more similar to regular stack diffs. The diff for nested stacks are hiding important details that makes it harder to decide whether the change is a danger to the production environment or not.

[tommydongaws]

I'm trying to use aws_cdk.pipelines.ConfirmPermissionsBroadening, which shows the broadening permission changes in a stack. However, when I have nested stacks the permission changes do not show up in "cdk diff" called by "ConfirmPermissionsBroadening".

Making the "cdk diff" work for nested stacks should solve this issue. I have a minimal stack here for testing this:
https://github.com/tommydongaws/nestedstacktest

[gagipro]

+1

[maxim-shilov-deel]

Any plans to fix that?

[WhiteAutumn]

For anyone experiencing that nested stacks do not show up when running diff:

I did some digging and have come to the conclusion that assetMetadata must be set to true in cdk.json in order for diff to display changes in nested stacks, I had it set to false.

[timothy-cloudopsguy]

Looks like this still isn't fixed (at least for deploying a new stack).

Nested stack output:

$ ENV=mgmt cdk diff
Stack CiCdPipelinesMgmt
Parameters
[+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}

Resources
[~] AWS::CloudFormation::Stack BitbucketOpenIDConnect.NestedStack/BitbucketOpenIDConnect.NestedStackResource BitbucketOpenIDConnectNestedStackBitbucketOpenIDConnectNestedStackResource5AC3760A 
 ├─ [~] NestedTemplate
 │   ├─ [+] Added: .Outputs
 │   └─ [+] Added: .Resources
 ├─ [+] Tags
 │   └─ [{"Key":"env_name","Value":"mgmt"},{"Key":"stack_name","Value":"CiCdPipelinesMgmt"},{"Key":"stack_type","Value":"CDK"}]
 ├─ [+] TemplateURL
 │   └─ {"Fn::Join":["",["https://s3.us-east-2.",{"Ref":"AWS::URLSuffix"},"/cdk-hnb659fds-assets-***************-us-east-2/6755b8696f417f189879be725adeb29ef3b3c182453d8adb19ed1c478edf65f1.json"]]}
 ├─ [+] DeletionPolicy
 │   └─ Delete
 ├─ [+] Metadata
 │   └─ {"aws:cdk:path":"CiCdPipelinesMgmt/BitbucketOpenIDConnect.NestedStack/BitbucketOpenIDConnect.NestedStackResource","aws:asset:path":"CiCdPipelinesMgmtBitbucketOpenIDConnect82C0353E.nested.template.json","aws:asset:property":"TemplateURL"}
 └─ [+] UpdateReplacePolicy
     └─ Delete
[~] AWS::CloudFormation::Stack CircleCIOpenIDConnect.NestedStack/CircleCIOpenIDConnect.NestedStackResource CircleCIOpenIDConnectNestedStackCircleCIOpenIDConnectNestedStackResourceAF8B11F7 
 ├─ [~] NestedTemplate
 │   ├─ [+] Added: .Outputs
 │   └─ [+] Added: .Resources
 ├─ [+] Tags
 │   └─ [{"Key":"env_name","Value":"mgmt"},{"Key":"stack_name","Value":"CiCdPipelinesMgmt"},{"Key":"stack_type","Value":"CDK"}]
 ├─ [+] TemplateURL
 │   └─ {"Fn::Join":["",["https://s3.us-east-2.",{"Ref":"AWS::URLSuffix"},"/cdk-hnb659fds-assets-***************-us-east-2/996da0bd9a9ae8dcd10f36d04381d969c25244ee9bc1c86574de8d7da9ee507c.json"]]}
 ├─ [+] DeletionPolicy
 │   └─ Delete
 ├─ [+] Metadata
 │   └─ {"aws:cdk:path":"CiCdPipelinesMgmt/CircleCIOpenIDConnect.NestedStack/CircleCIOpenIDConnect.NestedStackResource","aws:asset:path":"CiCdPipelinesMgmtCircleCIOpenIDConnectFF68E532.nested.template.json","aws:asset:property":"TemplateURL"}
 └─ [+] UpdateReplacePolicy
     └─ Delete

Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}

vs Regular Stack output

 $ ENV=mgmt cdk diff
Stack CiCdPipelinesMgmt/BitbucketOpenIDConnect (CiCdPipelinesMgmt-Bitbucket-OpenIdConnect)
IAM Statement Changes
┌───┬─────────────────────────────┬────────┬─────────────────────────────┬─────────────────────────────┬──────────────────────────────┐
│   │ Resource                    │ Effect │ Action                      │ Principal                   │ Condition                    │
├───┼─────────────────────────────┼────────┼─────────────────────────────┼─────────────────────────────┼──────────────────────────────┤
│ + │ ${BitbucketOpenIDConnect/Cu │ Allow  │ sts:AssumeRole              │ Service:lambda.amazonaws.co │                              │
│   │ stom::AWSCDKOpenIdConnectPr │        │                             │ m                           │                              │
│   │ oviderCustomResourceProvide │        │                             │                             │                              │
│   │ r/Role.Arn}                 │        │                             │                             │                              │
├
...
...
too much to put here
...
└───┴─────────────────────────────┴────────┴─────────────────────────────┴─────────────────────────────┴──────────────────────────────┘
IAM Policy Changes
┌───┬────────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────┐
│   │ Resource                                                       │ Managed Policy ARN                                             │
├───┼────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┤
│ + │ ${BitbucketOpenIDConnect/Custom::AWSCDKOpenIdConnectProviderCu │ {"Fn::Sub":"arn:${AWS::Partition}:iam::aws:policy/service-role │
│   │ stomResourceProvider/Role}                                     │ /AWSLambdaBasicExecutionRole"}                                 │
├───┼────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┤
│ + │ ${BitbucketOpenIDConnect/cicd-svc-role}                        │ arn:${AWS::Partition}:iam::aws:policy/ReadOnlyAccess           │
│ + │ ${BitbucketOpenIDConnect/cicd-svc-role}                        │ arn:${AWS::Partition}:iam::aws:policy/AmazonInspector2ReadOnly │
│   │                                                                │ Access                                                         │
└───┴────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Parameters
[+] Parameter BitbucketOpenIDConnect/BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}

Resources
[+] Custom::AWSCDKOpenIdConnectProvider BitbucketOpenIDConnect/provider/Resource providerCDFD1FC2 
[+] AWS::IAM::Role BitbucketOpenIDConnect/Custom::AWSCDKOpenIdConnectProviderCustomResourceProvider/Role CustomAWSCDKOpenIdConnectProviderCustomResourceProviderRole517FED65 
[+] AWS::Lambda::Function BitbucketOpenIDConnect/Custom::AWSCDKOpenIdConnectProviderCustomResourceProvider/Handler CustomAWSCDKOpenIdConnectProviderCustomResourceProviderHandlerF2C543E0 
[+] AWS::IAM::Role BitbucketOpenIDConnect/cicd-svc-role cicdsvcroleA6A9F092 
[+] AWS::IAM::Policy BitbucketOpenIDConnect/cicd-svc-role/DefaultPolicy cicdsvcroleDefaultPolicy7D4A68B2 

Outputs
[+] Output BitbucketOpenIDConnect/OpenIdConnectProvider OpenIdConnectProvider: {"Description":"OpenID Connect Provider","Value":{"Ref":"providerCDFD1FC2"}}
[+] Output BitbucketOpenIDConnect/CiCdRoleArn CiCdRoleArn: {"Description":"cicd role arn","Value":{"Fn::Join":["",["arn:",{"Ref":"AWS::Partition"},":iam::***************:role/",{"Ref":"cicdsvcroleA6A9F092"}]]}}

Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}

Stack CiCdPipelinesMgmt/CircleCIOpenIDConnect (CiCdPipelinesMgmt-CircleCI-OpenIdConnect)
IAM Statement Changes
┌───┬─────────────────────────────┬────────┬─────────────────────────────┬─────────────────────────────┬──────────────────────────────┐
│   │ Resource                    │ Effect │ Action                      │ Principal                   │ Condition                    │
├───┼─────────────────────────────┼────────┼─────────────────────────────┼─────────────────────────────┼──────────────────────────────┤
│ + │ ${CircleCIOpenIDConnect/Cus │ Allow  │ sts:AssumeRole              │ Service:lambda.amazonaws.co │                              │
│   │ tom::AWSCDKOpenIdConnectPro │        │                             │ m                           │                              │
│   │ viderCustomResourceProvider │        │                             │                             │                              │
│   │ /Role.Arn}                  │        │                             │                             │                              │
├───┼────────────────────────
...
...
too much to put here
...
...
└───┴─────────────────────────────┴────────┴─────────────────────────────┴─────────────────────────────┴──────────────────────────────┘
IAM Policy Changes
┌───┬────────────────────────────────────────────────────────────────┬────────────────────────────────────────────────────────────────┐
│   │ Resource                                                       │ Managed Policy ARN                                             │
├───┼────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┤
│ + │ ${CircleCIOpenIDConnect/Custom::AWSCDKOpenIdConnectProviderCus │ {"Fn::Sub":"arn:${AWS::Partition}:iam::aws:policy/service-role │
│   │ tomResourceProvider/Role}                                      │ /AWSLambdaBasicExecutionRole"}                                 │
├───┼────────────────────────────────────────────────────────────────┼────────────────────────────────────────────────────────────────┤
│ + │ ${CircleCIOpenIDConnect/cicd-svc-role}                         │ arn:${AWS::Partition}:iam::aws:policy/ReadOnlyAccess           │
│ + │ ${CircleCIOpenIDConnect/cicd-svc-role}                         │ arn:${AWS::Partition}:iam::aws:policy/AmazonInspector2ReadOnly │
│   │                                                                │ Access                                                         │
└───┴────────────────────────────────────────────────────────────────┴────────────────────────────────────────────────────────────────┘
(NOTE: There may be security-related changes not in this list. See https://github.com/aws/aws-cdk/issues/1299)

Parameters
[+] Parameter CircleCIOpenIDConnect/BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}

Resources
[+] Custom::AWSCDKOpenIdConnectProvider CircleCIOpenIDConnect/provider/Resource providerCDFD1FC2 
[+] AWS::IAM::Role CircleCIOpenIDConnect/Custom::AWSCDKOpenIdConnectProviderCustomResourceProvider/Role CustomAWSCDKOpenIdConnectProviderCustomResourceProviderRole517FED65 
[+] AWS::Lambda::Function CircleCIOpenIDConnect/Custom::AWSCDKOpenIdConnectProviderCustomResourceProvider/Handler CustomAWSCDKOpenIdConnectProviderCustomResourceProviderHandlerF2C543E0 
[+] AWS::IAM::Role CircleCIOpenIDConnect/cicd-svc-role cicdsvcroleA6A9F092 
[+] AWS::IAM::Policy CircleCIOpenIDConnect/cicd-svc-role/DefaultPolicy cicdsvcroleDefaultPolicy7D4A68B2 

Outputs
[+] Output CircleCIOpenIDConnect/OpenIdConnectProvider OpenIdConnectProvider: {"Description":"OpenID Connect Provider","Value":{"Ref":"providerCDFD1FC2"}}
[+] Output CircleCIOpenIDConnect/CiCdRoleArn CiCdRoleArn: {"Description":"cicd role arn","Value":{"Fn::Join":["",["arn:",{"Ref":"AWS::Partition"},":iam::***************:role/",{"Ref":"cicdsvcroleA6A9F092"}]]}}

Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}

Stack CiCdPipelinesMgmt
Parameters
[+] Parameter BootstrapVersion BootstrapVersion: {"Type":"AWS::SSM::Parameter::Value<String>","Default":"/cdk-bootstrap/hnb659fds/version","Description":"Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"}

Other Changes
[+] Unknown Rules: {"CheckBootstrapVersion":{"Assertions":[{"Assert":{"Fn::Not":[{"Fn::Contains":[["1","2","3","4","5"],{"Ref":"BootstrapVersion"}]}]},"AssertDescription":"CDK bootstrap stack version 6 required. Please run 'cdk bootstrap' with a recent version of the CDK CLI."}]}}

So yeah, looks like the assetMetadata only works for stacks after initial deploy then? I need to see all resources being created in Prod before I hit the button to deploy it.

[WhiteAutumn]

So yeah, looks like the assetMetadata only works for stacks after initial deploy then?

Yes, this has been my experience as well.

[timothy-cloudopsguy]

So yeah, looks like the assetMetadata only works for stacks after initial deploy then?

Yes, this has been my experience as well.

Ok, sounds about right. I'll keep this thread pinned for a few more years while we wait for the real fix to be implemented. :-) And I'll continue to prefer Stack over NestedStack.