Merge branch 'master' into bewie/apply_regexp

CHANGELOG.md

@@ -1,3 +1,31 @@
+# v0.17.0-beta
+Feature release. Due to a sizeable refactor and the number of configuration settings supported in Atlantis, this is a pre-release and should not be considered fully stable.
+
+## Features
+* Add `--enable-policy-checks` which adds a policy checking step to the Atlantis workflow and runs server-side conftest policies on the terraform plan output. ([#1317](https://github.com/runatlantis/atlantis/pull/1317) by @msarvar and @nishkrishnan)
+    - Supports `atlantis approve_policies` which allows a set of blessed github users to approve failing policies.
+* Support pre-workflow hooks on all comment/auto triggered commands ([#1418](https://github.com/runatlantis/atlantis/pull/1418) by @nishkrishnan)
+* Add `HEAD_COMMIT` to run steps
+* Update terraform version to 0.14.7
+
+## Backwards Incompatibilities/Notes
+* If you're using the Atlantis Docker image and aren't setting the `--default-tf-version` flag
+  then the default version of Terraform will now be 0.14.7. Simply set the above
+  flag to your desired default version to avoid any issues.
+
+## Downloads
+* [atlantis_darwin_amd64.zip](https://github.com/runatlantis/atlantis/releases/download/v0.17.0-beta/atlantis_darwin_amd64.zip)
+* [atlantis_linux_386.zip](https://github.com/runatlantis/atlantis/releases/download/v0.17.0-beta/atlantis_linux_386.zip)
+* [atlantis_linux_amd64.zip](https://github.com/runatlantis/atlantis/releases/download/v0.17.0-beta/atlantis_linux_amd64.zip)
+* [atlantis_linux_arm.zip](https://github.com/runatlantis/atlantis/releases/download/v0.17.0-beta/atlantis_linux_arm.zip)
+
+## Docker
+[`runatlantis/atlantis:v0.17.0-beta`](https://hub.docker.com/r/runatlantis/atlantis/tags/)
+
+## Diff v0.16.1..v0.17.0-beta
+https://github.com/runatlantis/atlantis/compare/v0.16.1...v0.17.0-beta
+
+
 # v0.16.1
 Few improvements and a number of bug fixes
 

CONTRIBUTING.md

@@ -71,6 +71,11 @@ docker-compose up --detach --build
 docker run --rm -v $(pwd):/go/src/github.com/runatlantis/atlantis -w /go/src/github.com/runatlantis/atlantis runatlantis/testing-env make test
 ```
 
+Or to run the integration tests
+```
+docker run --rm -v $(pwd):/go/src/github.com/runatlantis/atlantis -w /go/src/github.com/runatlantis/atlantis runatlantis/testing-env make test-all
+```
+
 ## Calling Your Local Atlantis From GitHub
 - Create a test terraform repository in your GitHub.
 - Create a personal access token for Atlantis. See [Create a GitHub token](https://github.com/runatlantis/atlantis/tree/master/runatlantis.io/docs/access-credentials.md#generating-an-access-token).

kustomize/bundle.yaml

@@ -22,7 +22,7 @@ spec:
         fsGroup: 1000 # Atlantis group (1000) read/write access to volumes.
       containers:
       - name: atlantis
-        image: runatlantis/atlantis:v0.16.1
+        image: runatlantis/atlantis:v0.17.0-beta
         env:
         - name: ATLANTIS_DATA_DIR
           value: /atlantis

main.go

@@ -20,7 +20,7 @@ import (
 	"github.com/spf13/viper"
 )
 
-const atlantisVersion = "0.16.1"
+const atlantisVersion = "0.17.0-beta"
 
 func main() {
 	v := viper.New()

runatlantis.io/docs/custom-workflows.md

@@ -349,6 +349,7 @@ Or a custom command
   * `HEAD_REPO_NAME` - Name of the repository that is getting merged into the base repository, ex. `atlantis`.
   * `HEAD_REPO_OWNER` - Owner of the repository that is getting merged into the base repository, ex. `acme-corp`.
   * `HEAD_BRANCH_NAME` - Name of the head branch of the pull request (the branch that is getting merged into the base)
+  * `HEAD_COMMIT` - The sha256 that points to the head of the branch that is being pull requested into the base. If the pull request is from Bitbucket Cloud the string will only be 12 characters long because Bitbucket Cloud truncates its commit IDs.
   * `BASE_BRANCH_NAME` - Name of the base branch of the pull request (the branch that the pull request is getting merged into)
   * `PROJECT_NAME` - Name of the project configured in `atlantis.yaml`. If no project name is configured this will be an empty string.
   * `PULL_NUM` - Pull request number or ID, ex. `2`.

runatlantis.io/docs/locking.md

@@ -64,5 +64,5 @@ Atlantis is doing is running `terraform plan` and `apply` and so all of the
 locking built in to those commands by Terraform isn't affected.
 
 In more detail, Terraform state locking locks the state while you run `terraform apply`
-so that multiple apply's can't run concurrently. Atlantis's locking is at a higher
+so that multiple applies can't run concurrently. Atlantis's locking is at a higher
 level because it prevents multiple pull requests from working on the same state.

runatlantis.io/docs/pre-workflow-hooks.md

@@ -50,6 +50,7 @@ command](custom-workflows.html#custom-run-command).
   * `HEAD_REPO_NAME` - Name of the repository that is getting merged into the base repository, ex. `atlantis`.
   * `HEAD_REPO_OWNER` - Owner of the repository that is getting merged into the base repository, ex. `acme-corp`.
   * `HEAD_BRANCH_NAME` - Name of the head branch of the pull request (the branch that is getting merged into the base)
+  * `HEAD_COMMIT` - The sha256 that points to the head of the branch that is being pull requested into the base. If the pull request is from Bitbucket Cloud the string will only be 12 characters long because Bitbucket Cloud truncates its commit IDs.
   * `BASE_BRANCH_NAME` - Name of the base branch of the pull request (the branch that the pull request is getting merged into)
   * `PULL_NUM` - Pull request number or ID, ex. `2`.
   * `PULL_AUTHOR` - Username of the pull request author, ex. `acme-user`.

runatlantis.io/docs/server-configuration.md

@@ -226,6 +226,13 @@ Values are chosen in this order:
   ```
   Stops atlantis locking projects and or workspaces when running terraform
 
+* ### `--enable-policy-checks`
+  <Badge text="beta" type="warn"/>
+  ```bash
+  atlantis server --enable-policy-checks
+  ```
+  Enables atlantis to run server side policies on the result of a terraform plan. Policies are defined in [server side repo config](https://www.runatlantis.io/docs/server-side-repo-config.html#reference).
+
 * ### `--enable-regexp-cmd`
   ```bash
   atlantis server --enable-regexp-cmd

runatlantis.io/docs/server-side-repo-config.md

@@ -28,6 +28,10 @@ repos:
   # Repo ID's are of the form {VCS hostname}/{org}/{repo name}, ex.
   # github.com/runatlantis/atlantis.
 - id: /.*/
+  # branch is an regex matching pull requests by base branch
+  # (the branch the pull request is getting merged into).
+  # By default, all branches are matched
+  branch: /.*/
 
   # apply_requirements sets the Apply Requirements for all repos that match.
   apply_requirements: [approved, mergeable]
@@ -331,6 +335,7 @@ custom workflows.
 |-----------|---------------------------------------------------------|-----------|----------|---------------------------------------------------------------------------------------|
 | repos     | array[[Repo](#repo)]                                    | see below | no       | List of repos to apply settings to.                                                   |
 | workflows | map[string: [Workflow](custom-workflows.html#workflow)] | see below | no       | Map from workflow name to workflow. Workflows override the default Atlantis commands. |
+| policies  | Policies.                                               | none      | no       | List of policy sets to run and associated metadata                                      |
 
 
 ::: tip A Note On Defaults
@@ -355,6 +360,7 @@ workflows:
     apply:
       steps: [apply]
 ```
+
 This gets merged with whatever config you write.
 If you set a workflow with the key `default`, it will override this.
 :::
@@ -400,3 +406,24 @@ If you set a workflow with the key `default`, it will override this.
   * `allow_custom_workflows` is set from the `id: /.*/` config and isn't unset
     by the `id: github.com/owner/repo` config because it didn't define that key.
 :::
+
+### Policies
+
+| Key                    | Type            | Default | Required  | Description                              |
+|------------------------|-----------------|---------|-----------|------------------------------------------|
+| conftest_version       | string          | none    | no        | conftest version to run all policy sets  |
+| owners                 | Owners(#Owners) | none    | yes       | owners that can approve failing policies |
+| policy_sets            | []PolicySet     | none    | yes       | set of policies to run on a plan output  |
+
+### Owners
+| Key         | Type              | Default | Required   | Description                                             |
+|-------------|-------------------|---------|------------|---------------------------------------------------------|
+| users       | []string          | none    | yes        | list of github users that can approve failing policies  |
+
+### PolicySet
+| Key                    | Type            | Default | Required  | Description                              |
+|------------------------|-----------------|---------|-----------|------------------------------------------|
+| name                   | string          | none    | yes       | unique name for the policy set           |
+| path                   | string          | none    | yes       | path to the rego policies                |
+| source                 | string          | none    | yes       | only `local` is supported at this time   |
+

server/events/command_runner.go

@@ -108,9 +108,10 @@ type DefaultCommandRunner struct {
 	// SilenceForkPRErrorsFlag is the name of the flag that controls fork PR's. We use
 	// this in our error message back to the user on a forked PR so they know
 	// how to disable error comment
-	SilenceForkPRErrorsFlag   string
-	CommentCommandRunnerByCmd map[models.CommandName]CommentCommandRunner
-	Drainer                   *Drainer
+	SilenceForkPRErrorsFlag       string
+	CommentCommandRunnerByCmd     map[models.CommandName]CommentCommandRunner
+	Drainer                       *Drainer
+	PreWorkflowHooksCommandRunner PreWorkflowHooksCommandRunner
 }
 
 // RunAutoplanCommand runs plan and policy_checks when a pull request is opened or updated.
@@ -139,12 +140,14 @@ func (c *DefaultCommandRunner) RunAutoplanCommand(baseRepo models.Repo, headRepo
 		return
 	}
 
-	autoPlanRunner := buildCommentCommandRunner(c, models.PlanCommand)
-	if autoPlanRunner == nil {
-		ctx.Log.Err("invalid autoplan command")
-		return
+	err := c.PreWorkflowHooksCommandRunner.RunPreHooks(ctx)
+
+	if err != nil {
+		ctx.Log.Err("Error running pre-workflow hooks %s. Proceeding with %s command.", err, models.PlanCommand)
 	}
 
+	autoPlanRunner := buildCommentCommandRunner(c, models.PlanCommand)
+
 	autoPlanRunner.Run(ctx, nil)
 }
 
@@ -182,12 +185,14 @@ func (c *DefaultCommandRunner) RunCommentCommand(baseRepo models.Repo, maybeHead
 		return
 	}
 
-	cmdRunner := buildCommentCommandRunner(c, cmd.CommandName())
-	if cmdRunner == nil {
-		ctx.Log.Err("command %s is not supported", cmd.Name.String())
-		return
+	err = c.PreWorkflowHooksCommandRunner.RunPreHooks(ctx)
+
+	if err != nil {
+		ctx.Log.Err("Error running pre-workflow hooks %s. Proceeding with %s command.", err, cmd.Name.String())
 	}
 
+	cmdRunner := buildCommentCommandRunner(c, cmd.CommandName())
+
 	cmdRunner.Run(ctx, cmd)
 }
 

server/events/command_runner_test.go

@@ -61,6 +61,7 @@ var approvePoliciesCommandRunner *events.ApprovePoliciesCommandRunner
 var planCommandRunner *events.PlanCommandRunner
 var applyCommandRunner *events.ApplyCommandRunner
 var unlockCommandRunner *events.UnlockCommandRunner
+var preWorkflowHooksCommandRunner events.PreWorkflowHooksCommandRunner
 
 func setup(t *testing.T) *vcsmocks.MockClient {
 	RegisterMockTestingT(t)
@@ -161,17 +162,22 @@ func setup(t *testing.T) *vcsmocks.MockClient {
 		models.UnlockCommand:          unlockCommandRunner,
 	}
 
+	preWorkflowHooksCommandRunner = mocks.NewMockPreWorkflowHooksCommandRunner()
+
+	When(preWorkflowHooksCommandRunner.RunPreHooks(matchers.AnyPtrToEventsCommandContext())).ThenReturn(nil)
+
 	ch = events.DefaultCommandRunner{
-		VCSClient:                 vcsClient,
-		CommentCommandRunnerByCmd: commentCommandRunnerByCmd,
-		EventParser:               eventParsing,
-		GithubPullGetter:          githubGetter,
-		GitlabMergeRequestGetter:  gitlabGetter,
-		AzureDevopsPullGetter:     azuredevopsGetter,
-		Logger:                    logger,
-		AllowForkPRs:              false,
-		AllowForkPRsFlag:          "allow-fork-prs-flag",
-		Drainer:                   drainer,
+		VCSClient:                     vcsClient,
+		CommentCommandRunnerByCmd:     commentCommandRunnerByCmd,
+		EventParser:                   eventParsing,
+		GithubPullGetter:              githubGetter,
+		GitlabMergeRequestGetter:      gitlabGetter,
+		AzureDevopsPullGetter:         azuredevopsGetter,
+		Logger:                        logger,
+		AllowForkPRs:                  false,
+		AllowForkPRsFlag:              "allow-fork-prs-flag",
+		Drainer:                       drainer,
+		PreWorkflowHooksCommandRunner: preWorkflowHooksCommandRunner,
 	}
 	return vcsClient
 }
@@ -645,7 +651,7 @@ func TestRunCommentCommand_DrainOngoing(t *testing.T) {
 }
 
 func TestRunCommentCommand_DrainNotOngoing(t *testing.T) {
-	t.Log("if drain is not ongoing then remove ongoing operation must be called even if panic occured")
+	t.Log("if drain is not ongoing then remove ongoing operation must be called even if panic occurred")
 	setup(t)
 	When(githubGetter.GetPullRequest(fixtures.GithubRepo, fixtures.Pull.Num)).ThenPanic("panic test - if you're seeing this in a test failure this isn't the failing test")
 	ch.RunCommentCommand(fixtures.GithubRepo, &fixtures.GithubRepo, nil, fixtures.User, fixtures.Pull.Num, nil)
@@ -662,7 +668,7 @@ func TestRunAutoplanCommand_DrainOngoing(t *testing.T) {
 }
 
 func TestRunAutoplanCommand_DrainNotOngoing(t *testing.T) {
-	t.Log("if drain is not ongoing then remove ongoing operation must be called even if panic occured")
+	t.Log("if drain is not ongoing then remove ongoing operation must be called even if panic occurred")
 	setup(t)
 	fixtures.Pull.BaseRepo = fixtures.GithubRepo
 	When(projectCommandBuilder.BuildAutoplanCommands(matchers.AnyPtrToEventsCommandContext())).ThenPanic("panic test - if you're seeing this in a test failure this isn't the failing test")

server/events/delete_lock_command.go

@@ -45,6 +45,7 @@ func (l *DefaultDeleteLockCommand) DeleteLocksByPull(repoFullName string, pullNu
 		return err
 	}
 	if len(locks) == 0 {
+		l.Logger.Debug("No locks found for pull")
 		return nil
 	}
 
@@ -61,6 +62,7 @@ func (l *DefaultDeleteLockCommand) deleteWorkingDir(lock models.ProjectLock) {
 	// installations of Atlantis will have locks in their DB that do not have
 	// this field on PullRequest. We skip deleting the working dir in this case.
 	if lock.Pull.BaseRepo == (models.Repo{}) {
+		l.Logger.Debug("Not deleting the working dir.")
 		return
 	}
 	unlock, err := l.WorkingDirLocker.TryLock(lock.Pull.BaseRepo.FullName, lock.Pull.Num, lock.Workspace)