Deploy Multiple Stacks at Once

[skorfmann]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Description

With #636 we introduced multiple stacks where we handle only one stack at the time via a given target-stack argument. We should enable a full, automated deployment.

• Resolve dependencies of stacks and build DAG
• Execute accordingly

References

• Blocked on Cross Stack References #651

[DanielMSchmidt]

This would be useful together with the Kubernetes Provider, a cluster normally needs some workloads on it to be useful, but the provider credentials can't be set to a not yet existing cluster.

Is this strictly blocked on #651? We can not infer the DAG based on references, but we could introduce an intermediate API that we can remove once the cross-stack references work. This would enable us to do some of this worked removed from the reference work.

I thought of sth to the effect of this as an API:

const app = new App();
const infraLayer = new Infrastructure(app, "infra");
const appLayer = new Applications(app, "app");

appLayer.dependsOn(infraLayer);

app.synth();

This would have non of the magic, you would need to use Terraform Data Elements in the application layer to get the resource context of the cluster, but you could express the relationship already.
This example would schedule a run of the infra layer before the run of the app layer, if the app layer is used as target-stack attribute

[DanielMSchmidt]

Talked with @ansgarm: We need to find a good UX around selecting the stacks to deploy.

• We could have a --with-dependencies flag and use one stack as an argument
• We could supply a list of stacks (possibly a glob like syntax) and deal with order for the user (would be a bit better I think as it could allow us to deal with parallel execution for the user at some point)

[skorfmann]

I really like that direction @DanielMSchmidt - The reason you have to select a stack right now is, that we can't resolve dependencies between stacks. If we would be able to do this, why not deploy all of them by default and allow the user to filter with the glob style you described?

[DanielMSchmidt]

No, if we introduce the API mentioned we would be able to describe dependencies between stacks. We can not automatically derive them before #651 is done though. The idea for the glob was to have the least surprising interface by default. Being a bit more verbose would help to avoid confusion if more than one stack is deployed / destroyed

[jsteinich]

May want to follow what AWS CDK does: https://docs.aws.amazon.com/cdk/latest/guide/cli.html#cli-stacks

[DanielMSchmidt]

yeah that's exactly what I meant, thank for the link!

[DanielMSchmidt]

Putting feedback from @ansgarm here:

Ideally the cdktf-cli could read the stack dependencies and check this in TFC and offer to setup the required permissions. But that's definitely something for the future 😁

[DanielMSchmidt]

Closed by

• feat(cli): serial multi-stack deploys #1609 (in band multi-stack deploy / destroy)
• feat(cli): handle dismissing a single stack vs stopping all stacks #1621 (add dismiss / stop semantic for multi-stack deployments)
• feat(cli): parallelize cdktf deploy/destroy #1625 (parallel multi-stack deploy / destroy)
• feat(cli): allow multiple stacks in the output command #1633 (multi-stack cdktf output)
• feat(cli): match multiple stacks with patterns #1634 (advanced pattern matching)

[github-actions]

I'm going to lock this issue because it has been closed for 30 days. This helps our maintainers find and focus on the active issues. If you've found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.