Github PR workflow support similar to Atlantis

[sstarcher]

A workflow similar to Atlantis for terraform would be a killer feature for helmfile - https://github.com/runatlantis/atlantis/

[wms]

If you'll permit me to shamelessly plug my own wares, I created a tool for exactly this purpose, which you may find here - https://github.com/fountainheadtechnologies/branch-officer

I'm using it successfully for internal projects at the moment, and would be delighted to get feedback from 'real' users.

[manics]

@warrenseymour Is https://github.com/fountainheadtechnologies/branch-officer a private repo? I get a 404.

[wms]

@manics Hey, thanks for pointing that out, it was indeed still set to 'private', which I've now rectified. I also forgot to mention that it's been published to NPM - https://www.npmjs.com/package/@fountainhead/branch-officer

Apologies to the Go-heads for writing it in JavaScript, but that's what I'm most productive in at this time.

[mumoshu]

I was looking for something like atlantis for integration with helmfile, with no luck.

@warrenseymour Thanks for sharing your awesome work! So, perhaps we could wrap branch-officer with a kind of github webhook handler written in e.g. AWS lambda or alike to DIY build atlantis?

[wms]

@mumoshu Indeed -- personally, I'm using branch-officer with CircleCI to build PRs, but there's no reason why you couldn't achieve a similar effect by invoking it via a Lambda-based webhook.

The only thing to be mindful of is that branch-officer is currently hard-coded to expect a GITHUB_TOKEN environment variable. CircleCI provides this automatically at build-time, so you'd need to define this variable in your Lambda configuration yourself (and take the relevant steps to make sure that token is kept 'secret'.)

[osterman]

For the record, you can use atlantis with helmfile =)

[mumoshu]

@osterman Hey! I was curious how atlantis could work in harmony with helmfile at first.

It turns out that atlantis allows the default behavior to be completely overriden.

So it would be a matter of just writing a atlantis.yaml looks like this, right?

version: 2
projects:
- dir: .
  workflow: myworkflow

workflows:
  myworkflow:
    plan:
      steps:
      - run: helmfile diff
    apply:
      steps:
      - run: helmfile apply

[osterman]

@mumoshu Exactly! ...provided it has a kube context.

[mumoshu]

@osterman Awesome. Really excited to see it ☺️

Just curious, but how do you deal with a more complex branching strategy with helmfile + atlantis?

For example. when PRs to the develop branch are deployable to the staging env and PRs to master branch are deployable to the prod env, how do you setup atlantis+helmfile?

I firstly came up with a single atlantis instance and a directory-per-environment setup like this:

.
├── atlantis.yaml
└── deploy
    ├── environments
    │   ├── prod
    │   │   └── values.yaml
    │   └── staging
    │       └── values.yaml
    └── helmfile.yaml

But this doesn't prevent atlantis from running prod deployments from a PR to the develop branch.

I also get to know about cloudposse-archives/atlantis-experiments#13 but even with it there seems no way to prevent the prod atlantis intance from responding to PRs to the develop branch.

[sstarcher]

I don't believe atlantis has any support for branches. Ya, so as is Atlantis could run vs helmfile you just lose things like autodiscovery and need to call out your directory structure.

[mumoshu]

I was just reading runatlantis/atlantis#310 along with the @osterman's comment there.

I think we would need:

• A custom wake word, so that commenting on a github PR helmfile apply instead of atlantis apply triggers the atlantis instance.
• Custom workflows, so that commenting on helmfile diff instead of helmfile plan works
• A branch restriction feature, so that we can bring up separate atlantis instances per environments for gitflow-flavored projects

Please add your ideas to the list so that we can better shape what we need!

[mumoshu]

Regarding the autodiscovery feat, I assume you're talking about helmfile's convention to use the helmfile.yaml at the current directory(which is usually the project root), right?

I do believe it is a great nature of helmfile, and hopefully helmfile could include an out-of-box support for atlantis-like features. Not sure if it is feasible to bring up our own, though.

@osterman Would there be any chance to maintain the fork of atlantis together, so that we can generalize atlantis to also support helmfile's use-cases, and add missing features?

[osterman]

Custom workflows, so that commenting on helmfile diff instead of helmfile plan works

This would be a "nice to have", but not a blocker for us.

A branch restriction feature, so that we can bring up separate atlantis instances per environments for gitflow-flavored projects

I think this would be cool. For now, we've been content with our fork's group based access controls on plan,apply and destroy actions using the --gh-team-whitelist arg we added. Anything that can continue to improve the security would be welcomed. Our strategy for the PoC we did was to use git submodules in our infrastructure repos. In our model, each AWS account (~stage) corresponds to a repo.

e.g. https://github.com/cloudposse/testing.cloudposse.co

I also get to know about cloudposse-archives/atlantis-experiments#13 but even with it there seems no way to prevent the prod atlantis intance from responding to PRs to the develop branch.

You're correct. Basically, atlantis in its current state (or our fork) doesn't do anything to prevent bad behavior. This would be a nice addition. I think maybe there should be a --gh-branch-whitelist that works like the other whitelist flags.

The other thing I'd like to have is automerge on successful apply.

@osterman Would there be any chance to maintain the fork of atlantis together, so that we can generalize atlantis to also support helmfile's use-cases, and add missing features?

100% - I'll be happy to add you as a contributor to the repo. Also, would like you to welcome you and @sstarcher to our slack community.

Also, I do want to point out that Luke (maintainer of Atlantis) is working on devising a more general spec factoring in our feedback. Our hope is to get this stuff mainstreamed eventually (in one shape or form), but we're diverging pretty quickly. We forked so we could satisfy our immediate needs to get the Atlantis experience. We're totally open to more "experimental" features in this fork.