Long strings/refs incorrectly split over newlines causing broken user-data/etc

[0xdevalias]

When I am building strings to use in my user-data/etc, CDK seems to incorrectly insert newlines (at long lines?), causing scripts to break.

Relevant Snippets from code:

const cfnInitCmd = `/opt/aws/bin/cfn-init -v --stack=${new cdk.AwsStackName} --region=${new cdk.AwsRegion} --resource=${asgLaunchConfig.logicalId}`;

appAsg.addUserData(
            "yum install -y aws-cfn-bootstrap;",
            `${cfnInitCmd} --configsets install;`,
            `/opt/aws/bin/cfn-signal --stack=${new cdk.AwsStackName} --region=${new cdk.AwsRegion} --resource=${asgLaunchConfig.logicalId} --exit-code \$?;`,
        );

When synthesized, this ends up like:

AppAutoScalingGroupLaunchConfig48178128:
    Type: AWS::AutoScaling::LaunchConfiguration
    Properties:
..snip..
      UserData:
        Fn::Base64:
          Fn::Join:
            - ""
            - - |-
                #!/bin/bash
                yum install -y aws-cfn-bootstrap;
                /opt/aws/bin/cfn-init -v --stack=
              - Ref: AWS::StackName
              - " --region="
              - Ref: AWS::Region
              - >-2
                 --resource=AppAutoScalingGroupLaunchConfig48178128 --configsets
                install;
                /opt/aws/bin/cfn-signal --stack=
              - Ref: AWS::StackName
              - " --region="
              - Ref: AWS::Region
              - " --resource=AppAutoScalingGroupLaunchConfig48178128 --exit-code
                $?;"

Which ends up in the end user-data as:

#!/bin/bash
yum install -y aws-cfn-bootstrap;
/opt/aws/bin/cfn-init -v --stack=TokenizedEC2 --region=ap-southeast-2 --resource=AppAutoScalingGroupLaunchConfig48178128 --configsets
install; /opt/aws/bin/cfn-signal --stack=TokenizedEC2 --region=ap-southeast-2 --resource=AppAutoScalingGroupLaunchConfig48178128 --exit-code $?;

Note how the install parameter ends up on a new line, when it shouldn't, which then breaks things.

This is also happening in my AWS::CloudFormation::Init Metadata override, breaking my /etc/cfn/cfn-hup.conf config.

[eladb]

How are you deploying this? Through the toolkit or by synthesizing a YAML output and then deploying via some other tool? I am wondering if the issue is our YAML formatter?

[0xdevalias]

Through the toolkit itself. I was only synthesising to try and debug the problem. It seems it’s happening when I am using template strings (possibly also with string/variable concatenation), but I was able to work around it directly using cdk.FnJoin (see the UserData code on linked PR) Get Outlook for iOS<https://aka.ms/o0ukef>

…

________________________________ From: Elad Ben-Israel <[email protected]> Sent: Sunday, December 9, 2018 8:52 pm To: awslabs/aws-cdk Cc: 0xdevalias; Author Subject: Re: [awslabs/aws-cdk] Long strings/refs incorrectly split over newlines causing broken user-data/etc (#1309) How are you deploying this? Through the toolkit or by synthesizing a YAML output and then deploying via some other tool? I am wondering if the issue is our YAML formatter? — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#1309 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAuA4xzRtIvJM-91kdjNNLZ_Lc02zdC9ks5u3N1PgaJpZM4ZJ6xA>.

[eladb]

Okay, ran a few tests. It seems like the root cause is the way our YAML formatter renders the multiline value. I reproduced the issue with YAML output and successfully deployed with JSON. Also, when I shortened the lines (by changing the logical ID of the launch configuration), the YAML template also worked (which I suspect is what happens in your FnJoin case).

@RomainMuller what do you think? This YAML output is giving us too much headache. Feels like a cat and mouse. Should we just support JSON in the toolkit and let people generate their own YAML at their own risk? Is there a configuration where our YAML output is "stupid" and doesn't do crazy things like splitting multiline outputs? (couldn't find it).

[rix0rrr]

The maintainer of this YAML was very responsive last time around and had a patch out the next day. We can also take this to them.

[rix0rrr]

But in the mean time, yeah, no reason not to use JSON internally I suppose. It's not for human consumption anyway.

[eladb]

If we allow users to synthesize YAML, then it should work. It would be an awful experience if “cdk deploy” worked and “cdk synth” didn’t. Happy to continue to try make YAML work, but I am concerned...

[rix0rrr]

As far as I can tell the issue is because of the

              - >-2
                 --resource=AppAutoScalingGroupLaunchConfig48178128 --configsets
                install;
                /opt/aws/bin/cfn-signal --stack=

quoting, as opposed to

              - " --resource=AppAutoScalingGroupLaunchConfig48178128 --exit-code
                $?;"

[rix0rrr]

Reported here: eemeli/yaml#55

[eemeli]

This should be fixed by the just-released version 1.1.0 of yaml. It's also now clearer how to disable line folding altogether.

[eladb]

@eemeli thanks for the amazingly quick turnaround!