Support prerequisite apply of projects for apply_requirements

[Jno21]

Hi,

The title might not be very clear and can be reworked.
What I would like is the possibility to have a project apply in the apply_requirements: []
You can't run an apply if the project define in apply_requierements is not apply before.

Example:

version: 2
projects:
- name: staging
  dir: .
  workspace: staging
  terraform_version: v0.11.10
  apply_requirements: [approved]
- name: production
  dir: .
  workspace: production
  terraform_version: v0.11.10
  apply_requirements: [approved, staging]

In this example I can't apply production until staging is applied.
It could be very useful in some workflow.

[lkysow]

Hi Jonathan, your request makes sense. Basically you want to do promotions between environments and require one environment to be applied prior to the other.

[Jno21]

It is exacly what I would like 👍

[asaghri]

Hello guys,
Any news on this ? This feature would be great with workspaces !

[billyshambrook]

Hey, have been looking into how this could be implemented and from a quick browse of the code, seems like you could lookup whether the required projects have been applied here -

atlantis/server/events/project_command_runner.go

Line 213 in 708fa0c

for _, req := range ctx.ApplyRequirements {

.

Is it fair to assume that any other value in apply_requirements that is not approved or mergeable is a project that you require to of been applied?

Another thing to consider would be apply-all option. If a project is added to apply_requirements, this would break apply-all.

[vaibhav-sparebank1]

It's really good to have apply_requirements with other projects.
eg. If we have mentioned project name in apply_requirements, then that project should get apply first. So we can use outputs from that project. Now it fails with the new setup and works with already applied projects. But its really nice to have for new projects as well.

[vic614]

Hi, just want to follow up with this feature request. Is it being activity worked on?

[dupuy26]

In the event that somebody does pick up this feature request, I would suggest that there are also interesting use cases for prerequisites of plan operations, not just apply, so it might be better to avoid tying these to apply_requirements and instead create an entirely new (and better fit for purpose) configuration mechanism for these dependencies.

Our immediate use case is to factor out some common workflow taking place before the actual plan step in the plan stage workflow (in this case, checking for potential hazards in PRs, such as use of external data, which can be used to execute arbitrary code on the Atlantis instance during the plan stage, before any policy checking takes place).

Another, even more interesting, use case would be layered or cascaded projects, such as deploying AWS OpenSearch instances (and waiting for them to become live) before using the ElasticSearch provider to configure users and roles and indexes in the just created OpenSearch instances. This use case is # 9 on the list of most popular Terraform issues – Terraform itself cannot handle this in a single plan/apply cycle and requires deployment to be completed at each layer before proceeding to the next one. If Atlantis were able to represent these dependencies, I suspect it would be a very compelling feature.

[nitrocode]

I would suggest that there are also interesting use cases for prerequisites of plan operations

There is a key for plan_requirements as well

https://www.runatlantis.io/docs/command-requirements.html#mergeable

Another, even more interesting, use case would be layered or cascaded projects, such as deploying AWS OpenSearch instances (and waiting for them to become live) before using the ElasticSearch provider

See execution_order_group

https://www.runatlantis.io/docs/repo-level-atlantis-yaml.html#order-of-planning-applying

[dupuy26]

The execution_order_group seems almost what I'd need – the documentation doesn't mention whether any failure in an execution_order_group would prevent projects in any later execution_order_group from running (which I would require). It is also unclear what the ordering is for any projects where the execution_order_group is omitted. Having a default execution_order_group for any project where it is not specified – and the ability to specify that default in config – would be helpful.

For the layered Terraform use case, it also seems like this feature falls a bit short. The documentation is a bit ambiguous when it says "Atlantis runs planning/applying for project2 first, then for project1" – does that means it runs both plan and apply for project2 before it runs plan for project1?

Since atlantis apply generally requires a previous atlantis plan, I suppose this feature just sets the order within an atlantis plan or within an atlantis apply. Thinking more deeply about this, I realize that my idea of dependencies between layers doesn't mesh that well with an apply as a prerequisite for a plan.

In that circumstance, you would end up having to do multiple iterations of plan and apply (maybe an apply that is a prerequisite for another project with autoplan enabled could trigger a re-plan there?) but at this point it becomes tempting to look for some higher-level orchestration to manage the deployment. The Terraform plan/apply distinction becomes less useful when some things need to be applied before others can even be planned and a Terraform-specific tool like Atlantis may not be the best approach.

[nitrocode]

I believe terragrunt can apply directories in a specific order too and Atlantis supports terragrunt out of the box.

The documentation all come from community contributions. If you notice something in there that falls short, please feel free to experiment and propose changes to clarify anything ambiguous.

[oleg-glushak]

The execution_order_group seems almost what I'd need – the documentation doesn't mention whether any failure in an execution_order_group would prevent projects in any later execution_order_group from running (which I would require). It is also unclear what the ordering is for any projects where the execution_order_group is omitted. Having a default execution_order_group for any project where it is not specified – and the ability to specify that default in config – would be helpful.

For the layered Terraform use case, it also seems like this feature falls a bit short. The documentation is a bit ambiguous when it says "Atlantis runs planning/applying for project2 first, then for project1" – does that means it runs both plan and apply for project2 before it runs plan for project1?

Since atlantis apply generally requires a previous atlantis plan, I suppose this feature just sets the order within an atlantis plan or within an atlantis apply. Thinking more deeply about this, I realize that my idea of dependencies between layers doesn't mesh that well with an apply as a prerequisite for a plan.

In that circumstance, you would end up having to do multiple iterations of plan and apply (maybe an apply that is a prerequisite for another project with autoplan enabled could trigger a re-plan there?) but at this point it becomes tempting to look for some higher-level orchestration to manage the deployment. The Terraform plan/apply distinction becomes less useful when some things need to be applied before others can even be planned and a Terraform-specific tool like Atlantis may not be the best approach.

@dupuy26 I mentioned the same/similar issue here. It seems it's currently only achievable with Atlantis+Terragrunt+Terraform, but not Atlantis+Terraform, which is an issue for my current project, where Terragrunt isn't currently used.

[Luay-Sol]

I have followed a different approach to achieve the above, e.g promotions between environments. My approach does not use the apply_requirement key, instead, I have introduced a separate key depends_on into the project config.

 version: 3
 projects:
 - dir: .
   name: project-1
 - dir: .
   name: project-2
   depends_on: [project-1]

Found it less complicated than trying to mess up with the existing apply_requirements, as well as it is clear what the intention of it (Create dependencies between projects), which's a strategy that's used in many tools, e.g terraform.

This is the PR for the changes: #3292