feat(s3-deployment): deploy data with deploy-time values

[eladb]

Allow deploying test-based content that can potentially include deploy-time values such as attributes of cloud resources.

Introduce a Source.data(objectKey, text) and Source.jsonData(objectKey, obj) where the data can naturally include deploy-time tokens such as references to resources (Ref) or to resource attributes (Fn::GetAtt).

For example:

const appConfig = {
  topic_arn: topic.topicArn,
  base_url: 'https://my-endpoint',
};

new s3deploy.BucketDeployment(this, 'BucketDeployment', {
  sources: [s3deploy.Source.jsonData('config.json', config)],
  destinationBucket: destinationBucket,
});

This is implemented by replacing the deploy-time tokens with markers that are replaced inside the s3-deployment custom resource.

Fixes #12903

---

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

[gitpod-io]

[jogold]

Hi @eladb,

FYI I'm currently addressing this like that:

new cr.AwsCustomResource(this, 'PutConfig', {
  policy: cr.AwsCustomResourcePolicy.fromStatements([
    new iam.PolicyStatement({
      actions: ['s3:PutObject'],
      resources: [props.bucket.arnForObjects('config.json')],
    }),
  ]),
  onUpdate: {
    service: 'S3',
    action: 'putObject',
    parameters: {
      Bucket: props.bucket.bucketName,
      Key: 'config.json',
      Body: Stack.of(this).toJsonString(props.config),
      ContentType: 'application/json',
      CacheControl: 'max-age=0, no-cache, no-store, must-revalidate',
    },
    physicalResourceId: cr.PhysicalResourceId.of('config'),
  },
});

[eladb]

@jogold this is very cool, but I since this means the contents of the file is inlined in the template, I think the 1MB limitation of the template size might be a big concern.

[jogold]

this means the contents of the file is inlined in the template, I think the 1MB limitation of the template size might be a big concern.

This is correct but one could argue that most of the use cases for files with deploy-time values are small files. Anyway, it's always nice (and fun 😄) to have a generalized solution.

[rfestag]

Definitely interested in this feature as it is developed!

A pattern I've run into before that was particularly frustrating is static websites that require deploy-time parameters (say, Cognito user pool information or bucket names). I generally do this through an S3 deployment bucket with a Dockerfile to build the assets. Is this something that could integrate easily with that? Or would the pattern be to render this content separately to the target bucket?

[ckifer]

+1 to this, thank you @eladb !

Looking for this solution right now and glad to see it in development, I use the exact pattern internally but built in is going to be fantastic!

[mergify]

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[aws-cdk-automation]

AWS CodeBuild CI Report

• CodeBuild project: AutoBuildProject89A8053A-LhjRyN9kxr8o
• Commit ID: f80dd01
• Result: SUCCEEDED
• Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

[mergify]

Thank you for contributing! Your pull request will be updated from master and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

[lafriakh]

@eladb I tried to use it with REST Api url and I get:

Error [ValidationError]: Template format error: Unresolved resource dependencies [...] in the Resources block of the template

I'm doing something wrong here, because your changes should make that possible.