Adds support for specifying the atlantis.yaml filename on the server side

[darrylb-github]

This allows setting different configs for different instances of atlantis, which is useful when wanting to run different servers for production and staging.

Our use case is to have separate production and staging Atlantis servers. This enables that by using multiple atlantis.yaml config files in the same repo and setting the --repo-config flag on each server. This allows you to launch a staging Atlantis server pointing at a staging atlantis.yaml file (e.g. --repo-config atlantis-staging.yaml) and a production Atlantis server pointing at a production atlantis.yaml file in the same repo (e.g. --repo-config atlantis-production.yaml).

This way you can use different credentials for staging and production and maintain cleaner separation between environments.

Related to #249

[codecov]

Codecov Report

Merging #310 into master will increase coverage by 0.04%.
The diff coverage is 92.85%.

@@            Coverage Diff             @@
##           master     #310      +/-   ##
==========================================
+ Coverage   70.54%   70.58%   +0.04%     
==========================================
  Files          61       61              
  Lines        3639     3644       +5     
==========================================
+ Hits         2567     2572       +5     
  Misses        893      893              
  Partials      179      179

Impacted Files	Coverage Δ
cmd/server.go	79.59% <100%> (+0.57%)	⬆️
server/events/yaml/parser_validator.go	98.48% <100%> (ø)	⬆️
server/events/comment_parser.go	98.27% <100%> (ø)	⬆️
server/server.go	62.44% <100%> (+0.15%)	⬆️
server/events/project_command_builder.go	82.9% <84.61%> (ø)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0e4dc4f...0c551c3. Read the comment docs.

[osterman]

@darrylb-github was thinking about this some more and wonder if a --repo-pr-target-branch is necessary too.

So, for example a PR that merges into develop branch (--repo-pr-target-branch=develop) would get picked up but the by the atlantis server running with --repo-config=atlantis-develop.yaml (presumably in a staging account) and a PR that merges into the master branch (--repo-pr-target-brach=master) would get picked up by the atlantis server running with --repo-config=atlantis-master.yaml (presumably in a production account).

If not based on the target branch, how are you approaching it? project folders per account? dedicated repos per account?

[darrylb-github]

@darrylb-github was thinking about this some more and wonder if a --repo-pr-target-branch is necessary too.

So, for example a PR that merges into develop branch (--repo-pr-target-branch=develop) would get picked up but the by the atlantis server running with --repo-config=atlantis-develop.yaml (presumably in a staging account) and a PR that merges into the master branch (--repo-pr-target-brach=master) would get picked up by the atlantis server running with --repo-config=atlantis-master.yaml (presumably in a production account).

If not based on the target branch, how are you approaching it? project folders per account? dedicated repos per account?

@osterman We don't use gitflow approach so for us in that scenario we would keep those files in master and do separate PRs for dev/staging/prod changes. The staging atlantis server would monitor the dev and staging files and the production atlantis server would monitor prod.

We don't have long lived branches outside of master like develop / staging etc, so our PRs all go into master.

[johncblandii]

What about the scenario where you have this scenario?

• account 1: dev
• account 2: prod, uat, support

Each account would have an Atlantis, but one would run for 3 environments. Would this work in that scenario? It seems so, but I just wanted to verify.

[darrylb-github]

@johncblandii Yup, that should work no problem as you expect. One atlantis server would use creds from account 1 and the other from account 2. You'd then point the server using account 1 at an atlantis.yaml config looking at dev terraform files and the second server using account 2 at another atlantis.yaml config looking at prod, uat and support terraform files. You then setup two webhooks, one for each server. Both will fire every time but only one will come back with a plan depending on which terraform files change.

[johncblandii]

Excellent. Thx @darrylb-github.

[osterman]

@lkysow any feedback on this PR?

[sstarcher]

I'm looking forward to this anything I can assist with?

[darrylb-github]

Hey @lkysow ,

Any feedback on this PR? Anything else I need to do to get it considered for merging?

Thanks,
Darryl

[johncblandii]

Same here @sstarcher. I have a PR pending this release so I'm def' willing to help however I can.

[lkysow]

I'm writing up an RFC to cover multi-account deployments. I understand that this is one way to accomplish multi-account, however this specific method might not work for all use cases. I want to make sure everyone has a chance to speak to their use-case so we can find the best way to get this functionality into Atlantis.

If I merge this PR today then we have to support this functionality for many years, even if we find a better way to accomplish what you're trying to do. I want to avoid making that mistake.

[lkysow]

@johncblandii I'm curious what's your PR builds on top of this. Want to get that into the RFC.

[johncblandii]

@lkysow my PR is an internal deployment and configuration of Atlantis. It doesn't build upon, but I'm awaiting this so I can implement these features.

Sorry for the confusion.

[lkysow]

@darrylb-github do you have separate workflows for each account? That is, is the yaml in your staging config different from your production one? If so, in what way.

Or is the one file for staging, one for prod just to get Atlantis to ignore webhooks that pertain to one account.

[darrylb-github]

@lkysow They are very similar, but our staging one has 2 workflows (dev and staging) and production just one. These workflows just look at different directories, so for us we have 3 directories (dev, staging, prod) each with their own terraform configs and backends.

The main reason we want to run 2 atlantis servers is so that we can use different AWS accounts and permissions, and have the atlantis server for each environment/account live in that account (vs sharing one atlantis server with resources from other accounts/environments). An atlantis server applying changes to our staging environment shouldn't have access to apply changes to our production account.

[osterman]

We have it up and running now with our fork. Here's how we're organizing it. We're doing a demo tomorrow.

Each AWS account has a dedicated repo (e.g. for prod, staging, dev, root, testing, audit, etc) based on our geodesic base image. Each account runs it's own atlantis daemon on ECS Fargate, so we can gate upgrades of atlantis like all other software. Sometimes we'll want multiple atlantis repo configs per repo, but most of the time we'll be okay with just one.

https://github.com/cloudposse/root.cloudposse.co/blob/master/atlantis/root.yaml
https://github.com/cloudposse/testing.cloudposse.co/blob/add-atlantis/atlantis/testing.yaml (not yet merged, slightly out of date)

Here's an alternative approach where we can deploy one repo to multiple clusters by addressing atlantis differently (atlantis/prod vs atlantis/testing):
https://github.com/cloudposse/alpinist/tree/automation/atlantis

We also are close to getting something working with submodules, so we can make https://github.com/cloudposse/alpinist a submodule in one of our AWS account repos (for example).

We're assigning each atlantis server it's own wake word (e.g. atlantis/testing would awaken the testing server). We're also able to associate a given atlantis instance with different GitHub teams, granting them permissions. For example, atlantis/testing allows pretty much anything, while atlantis/root only allows a couple admins since this is how we can provision user accounts.

We implemented destroy as well. We can add this step to testing pipelines, but never include them in production pipelines.

We don't use terraform workspaces. State is isolated to each AWS account, which share nothing.

@lkysow might help if we give you a demo to see how we're using it, so you can incorporate some ideas.

[pratikmallya]

@lkysow is the draft RFC somewhere public where it can be viewed?

[sstarcher]

@lkysow I don't see the reason a server side config would be blocked by "multi-account RFC". My desire for a server side configuration is as a security measure to prevent users from running adhoc commands in a highly elevated environment such as Atlantis.

[lkysow]

Hey Shane, it's not blocked by multi-account. Server-side will come first. But this PR is about multi-account.

[sstarcher]

@lkysow ahh sorry after looking past the title that makes more sense. As the title currently is I thought it was the other ticket.

[lkysow]

This can be accomplished by setting a different server-side workflow on each of your servers.

[darrylb-github]

Hi @lkysow ,

Just to clarify, how would I use this to support staging and production environments looking at the same repo? I.e. production atlantis monitoring one directory in the repo and staging atlantis monitoring a different directory in the same repo.

Example directory structure:

├── Dockerfile
├── Jenkinsfile
├── README.md
├── atlantis.yaml
├── production
│   └── us-east-1
│       ├── main_backend.tf
└── staging
    └── us-east-1
        ├── main_backend.tf

In this scenario I want the atlantis server running in the staging AWS account to monitor the staging folder and the atlantis server running in the production AWS account to monitor the production folder. How do I use server-side-workflows to instruct staging atlantis to only monitor the staging folder? 🤔

With the code in this PR I did this by having atlantis-staging.yaml and atlantis-production.yaml, each defining their own project to only monitor the appropriate directory for that environment. I then ran the staging atlantis with --repo-config=atlantis-staging.yaml and the production one with --repo-config=atlantis-production.yaml.

Thanks,
Darryl

[lkysow]

Ahh okay sorry server-side workflows might not work then. You could define both workflows on each server with staging's server using a no-op prod workflow and the prod server using a no-op staging workflow.

But the status checks and empty comments might be annoying.

[sleevin]

@darrylb-github

I'm in the exact same situation as you. A single terraform repo that has folders for our different environments spread across two different aws accounts.

Did you happen to find a solution to your problem? I'd think that by using the server side repo config you'd run into the problem that @lkysow mentioned where atlantis would just comment back status checks and empty comments.