feat(Add Pull Request Silence custom exit code for custom workflows)

[Mitsuwa]

What

Add a ATLANTIS_SQUELCH_CODE exit code to be able to be called from custom workflows during a run step, which will silence pull request comments and pull request status updates.

run: exit $ATLANTIS_SQUELCH_CODE

This can also be coupled with something like --enable-pr-squelching-code for extra concern of run commands exiting with a similar code.

This custom exit code should conform to unix standards

Please let me know if theres already a recommended way to do something like this

I believe I've combed the many docs and issues, tried many options but maybe I have missed something obvious 👍

Why

We are currently running two atlantis instances that both point at the same github repo. The repo contains multiple environments like (development and production) within folder

env/development/*
env/production/*

We have a single atlantis.yaml in the github repo root with something similar to

projects:
- dir: env/development/project1
   name: env/development/project1
...
- dir: env/development/projectN
   name: env/development/projectN
- dir: env/production/project1
   name: env/production/project1
...
- dir: env/production/projectN
   name: env/production/projectN
workflows:
...

Since github webhooks fire for all events to each atlantis instance, each receives requests to act on an atlantis cmd

Regarding the two instances of atlantis, one is meant to act ONLY on Development env and the other is to act ONLY on Production.

both atlantis have the same repo.yaml

repos:
- id: /.*/
  allowed_overrides: [workflow]
  allow_custom_workflows: true

So essentially we are leveraging the atlantis.yaml to define the workflows. The custom workflows themselves terminate with exit 0 on a condition I've set to say whether or not atlantis should allow a atlantis plan/apply or not. Which does work but the problem is that atlantis will comment back on the pr, from both instances. One with the actual plan, and the other with an empty comment, which is not helpful and is excessive

If we could allow a custom workflow to exit with a particular code, squelching the comments and pr status, it'd be very helpful to us and enable us to run multiple atlantis' easier.

Also if I exit 0 during apply runs, the non acting atlantis will actually merge the pr, before the other atlantis completes its run.

If you exit 1 the non acting atlantis will fail my pr on apply or plan, which can kinda work because the other atlantis will still be running and update it later but its not ideal, it also posts a failed message in the comment

I tried having the atlantis.yaml not specify its own workflows and then define (or not define them) in the server side repos.yaml but that only causes the non acting atlantis to comment with a not found error

I also tried adding empty workflows with just names or steps that were empty arrays but atlantis would either just do the default or have yaml parsing errors.

Implementation

I was playing around with implementing this but figured I should just reach out before I spend more time on something that will not be accepted, but the unit tests do seem to indicate something like this would work https://gist.github.com/Mitsuwa/6cbefc7631bba927baea62eef6c580c2

I'd be happy to continue working on this and implementing anything else around this, if you all are interested in this feature.

Alternate solution

I was also thinking if the serverside repos.yaml could specify its own projects list to act/not act on, that would also solve this.

[mveitas]

@Mitsuwa I think what you are looking for is related to both of the following issues: #310 and #473

[Mitsuwa]

Yeah you are correct, likely those two things could be solutions for us as well!