Merge branch 'master' into fix-automerge-on-noapply

.circleci/config.yml

@@ -21,7 +21,7 @@ jobs:
         # We do this instead of setting --default-tf-version because setting
         # that flag starts the download asynchronously so we'd have a race
         # condition.
-        TERRAFORM_VERSION: 0.12.24
+        TERRAFORM_VERSION: 0.12.26
     steps:
     - checkout
     - run: make build-service

CHANGELOG.md

@@ -1,3 +1,119 @@
+# v0.13.0
+
+## Description
+This release enables support for running plans and applies in parallel **only when using Terraform workspaces**.
+It also enables graceful shutdown for Atlantis where it waits for in-progress plans and applies to complete.
+See below for the complete list.
+
+## Features
+* Upgrade default Terraform version in Docker image to 0.12.26.
+* Add support for parallel plans and applies ([#926](https://github.com/runatlantis/atlantis/pull/926) by @Fauzyy)
+
+  Running in parallel is only supported if you're using workspaces to separate your projects.
+  Projects in separate directories can **not** be run in parallel currently.
+  To use, set
+
+  ```yaml
+  parallel_plan: true
+  parallel_apply: true
+  ```
+  
+  In your repo-level `atlantis.yaml` file.
+* Add support for graceful shutdown ([#1051](https://github.com/runatlantis/atlantis/pull/1051) by @benoit74).
+  When Atlantis receive a SIGINT or SIGTERM it won't shut down immediately. It will wait for
+  in-progress plans and applies to complete. Any new actions, e.g. comments or autoplans
+  will be refused and an error comment will be posted to the PR indicating that Atlantis is shutting
+  down and the user should try again later.
+
+  In addition, a new `/status` endpoint has been added that currently only returns
+  the number of in-progress operations and whether the server is shutting down.
+
+* GitHub: A new flag `--allow-draft-prs` has been added that will re-enable the ability
+  for users to run plan and apply on GitHub draft PRs. This ability was removed in
+  v0.12.0. ([#1053](https://github.com/runatlantis/atlantis/pull/1053) by @cket)
+* GitHub: Preserve original commit message when automerging ([#1049](https://github.com/runatlantis/atlantis/pull/1049) by @pratikmallya).
+
+  This change removes the `[Atlantis] Automatically merging after successful apply` commit message
+  and instead has GitHub autogenerate the commit message similarly to how it would when
+  you click the "Merge" button in the UI.
+* Change log level for HTTP requests from INFO to DBUG, e.g.
+  ```
+  2020/05/26 12:16:20+0000 [INFO] server: GET /healthz – respond HTTP 200
+  2020/05/26 12:16:36+0000 [INFO] server: GET /healthz – from <IP>
+  ```
+  ([#1056](https://github.com/runatlantis/atlantis/pull/1056) by @tammert)
+* GitLab: Use correct link to merge requests (previously used `#<num>` instead of `!<num>`) ([#1059](https://github.com/runatlantis/atlantis/pull/1059) by @EppO)
+
+## Bugfixes
+* Azure DevOps: Project links link to pull requests now (Fixes [#957](https://github.com/runatlantis/atlantis/issues/957) by @mcdafydd)
+* GitHub: Release locks when GitHub draft PRs are closed ([#1038](https://github.com/runatlantis/atlantis/pull/1038) by @andrewring)
+* Ensure git-lfs is in our Docker image (Fixes [#1054](https://github.com/runatlantis/atlantis/pull/1054))
+
+## Backwards Incompatibilities / Notes:
+* If you're using the Atlantis Docker image and aren't setting the `--default-tf-version` flag
+  then the default version of Terraform will now be 0.12.26. Simply set the above
+  flag to your desired default version to avoid any issues.
+* HTTP requests are now logged as DBUG instead of INFO to reduce log spam. If you
+  still want to see these logs you must run with `--log-level=debug`.
+* Atlantis will no longer immediately shutdown when it receives a SIGINT or SIGTERM,
+  it will now wait for in-progress plans and applies to complete. To stop Atlantis
+  without waiting, send a SIGKILL.
+
+## Downloads
+* [atlantis_darwin_amd64.zip](https://github.com/runatlantis/atlantis/releases/download/v0.13.0/atlantis_darwin_amd64.zip)
+* [atlantis_linux_386.zip](https://github.com/runatlantis/atlantis/releases/download/v0.13.0/atlantis_linux_386.zip)
+* [atlantis_linux_amd64.zip](https://github.com/runatlantis/atlantis/releases/download/v0.13.0/atlantis_linux_amd64.zip)
+* [atlantis_linux_arm.zip](https://github.com/runatlantis/atlantis/releases/download/v0.13.0/atlantis_linux_arm.zip)
+
+## Docker
+[`runatlantis/atlantis:v0.13.0`](https://hub.docker.com/r/runatlantis/atlantis/tags/)
+
+## Diff v0.12.0..v0.13.0
+https://github.com/runatlantis/atlantis/compare/v0.12.0...v0.13.0
+
+# v0.12.0
+
+## Description
+This release contains one much-awaited GitHub-only feature: the ability to hide previous
+plan comments with the `--hide-prev-plan-comments` flag. It also contains
+a host of other small features and fags.
+
+## Features
+* GitHub: Add `--hide-prev-plan-comments` flag. When set, previous plan comments will be marked as outdated in GitHub's UI.
+  This collapses them making a PR with lots of plan comments easier to read. ([#994](https://github.com/runatlantis/atlantis/pull/994) by @goodspark)
+* GitHub: Ignore draft PRs until they're changed to "ready for review". ([#977](https://github.com/runatlantis/atlantis/pull/977) by @cket)
+* Upgrade default Terraform version in Docker image to 0.12.24.
+* Set `as_user` param when sending slack notifications so the message is decorated appropriately ([#907](https://github.com/runatlantis/atlantis/pull/907) by @tmcevoy14)
+* Add Git LFS support ([#872](https://github.com/runatlantis/atlantis/pull/872) by @remilapeyre)
+* Add `--silence-vcs-status-no-plans` flag that silences VCS commit status when autoplan finds no projects to plan.
+  When set, Atlantis won't create any VCS statuses if there no projects to plan. ([#959](https://github.com/runatlantis/atlantis/pull/959) by @cket)
+* Add `--disable-markdown-folding` flag that disables folding for long plan/apply outputs. ([#960](https://github.com/runatlantis/atlantis/pull/960) by @mhumeSF)
+* Ignore casing when setting log levels, e.g. `--log-level=INFO` now works. ([#976](https://github.com/runatlantis/atlantis/pull/976) by @jpreese)
+* Azure DevOps: Add policy checking. ([#984](https://github.com/runatlantis/atlantis/pull/984) by @jpreese)
+* Upgrade boltdb to latest maintained version. ([#992](https://github.com/runatlantis/atlantis/pull/992) by @amasover)
+
+## Bugfixes
+* Azure DevOps: Prevent pull request updated events from triggering autoplan when the event was caused by a change in approvals. (Fixes [#946](https://github.com/runatlantis/atlantis/issues/946) by @mcdafydd)
+
+## Backwards Incompatibilities / Notes:
+* GitHub draft PRs are now ignored until they're marked "ready for review" and opened as regular PRs.
+  **NOTE: ** This functionality was added back in Atlantis v0.13.0 via the `--allow-draft-prs` flag.
+* If you're using the Atlantis Docker image and aren't setting the `--default-tf-version` flag
+  then the default version of Terraform will now be 0.12.24. Simply set the above
+  flag to your desired default version to avoid any issues.
+
+## Downloads
+* [atlantis_darwin_amd64.zip](https://github.com/runatlantis/atlantis/releases/download/v0.12.0/atlantis_darwin_amd64.zip)
+* [atlantis_linux_386.zip](https://github.com/runatlantis/atlantis/releases/download/v0.12.0/atlantis_linux_386.zip)
+* [atlantis_linux_amd64.zip](https://github.com/runatlantis/atlantis/releases/download/v0.12.0/atlantis_linux_amd64.zip)
+* [atlantis_linux_arm.zip](https://github.com/runatlantis/atlantis/releases/download/v0.12.0/atlantis_linux_arm.zip)
+
+## Docker
+[`runatlantis/atlantis:v0.12.0`](https://hub.docker.com/r/runatlantis/atlantis/tags/)
+
+## Diff v0.11.1..v0.12.0
+https://github.com/runatlantis/atlantis/compare/v0.11.1...v0.12.0
+
 # v0.11.1
 
 ## Description

CONTRIBUTING.md

@@ -18,37 +18,25 @@ We take security issues seriously. Please email us directly at security [at] run
 open your browser to http://localhost:8080.
 * The website will be regenerated when your pull request is merged to master.
 
-
 # Developing
 
 ## Running Atlantis Locally
-Get the source code:
-```
-go get github.com/runatlantis/atlantis
-```
-This will clone Atlantis into `$GOPATH/src/github.com/runatlantis/atlantis` (where `$GOPATH` defaults to `~/go`).
-
-Go to that directory:
-```
-cd $GOPATH/src/github.com/runatlantis/atlantis
-```
-
-Compile Atlantis:
-```
-go install
-```
-
-Run Atlantis:
-```
-atlantis server --gh-user <your username> --gh-token <your token> --repo-whitelist <your repo> --gh-webhook-secret <your webhook secret> --log-level debug
-```
-If you get an error like `command not found: atlantis`, ensure that `$GOPATH/bin` is in your `$PATH`.
-
-Running Tests Locally:
+* Clone the repo from https://github.com/runatlantis/atlantis/
+* Compile Atlantis:
+    ```
+    go install
+    ```
+* Run Atlantis:
+    ```
+    atlantis server --gh-user <your username> --gh-token <your token> --repo-whitelist <your repo> --gh-webhook-secret <your webhook secret> --log-level debug
+    ```
+    If you get an error like `command not found: atlantis`, ensure that `$GOPATH/bin` is in your `$PATH`.
+
+## Running Tests Locally:
 
 `make test`. If you want to run the integration tests that actually run real `terraform` commands, run `make test-all`.
 
-Running Tests In Docker:
+## Running Tests In Docker:
 ```
 docker run --rm -v $(pwd):/go/src/github.com/runatlantis/atlantis -w /go/src/github.com/runatlantis/atlantis runatlantis/testing-env make test
 ```
@@ -102,6 +90,39 @@ This is easier to read and more consistent
 - if you need to test internally i.e. access non-exported stuff, call the file `{file under test}_internal_test.go`
 - use our testing utility for easier-to-read assertions: `import . "github.com/runatlantis/atlantis/testing"` and then use `Assert()`, `Equals()` and `Ok()`
 
+### Mocks
+We use [pegomock](https://github.com/petergtz/pegomock) for mocking. If you're
+modifying any interfaces that are mocked, you'll need to regen the mocks for that
+interface.
+
+If you see errors like:
+```
+# github.com/runatlantis/atlantis/server/events [github.com/runatlantis/atlantis/server/events.test]
+server/events/project_command_builder_internal_test.go:567:5: cannot use workingDir (type *MockWorkingDir) as type WorkingDir in field value:
+	*MockWorkingDir does not implement WorkingDir (missing ListAllFiles method)
+```
+
+Then you've likely modified an interface and now need to update the mocks.
+
+Each interface that is mocked has a `go:generate` command above it, e.g.
+```go
+//go:generate pegomock generate -m --use-experimental-model-gen --package mocks -o mocks/mock_project_command_builder.go ProjectCommandBuilder
+
+type ProjectCommandBuilder interface {
+	BuildAutoplanCommands(ctx *CommandContext) ([]models.ProjectCommandContext, error)
+}
+```
+
+To regen the mock, run `go generate` on that file, e.g.
+```sh
+go generate server/events/project_command_builder.go
+```
+
+If you get an error about `pegomock` not being available, install it:
+```sh
+go get github.com/petergtz/pegomock/...
+```
+
 # Creating a New Release
 1. Update version number in `main.go`.
 1. Update image tag version in the [kustomize/bundle.yaml](kustomize/bundle.yaml).

Dockerfile

@@ -1,15 +1,15 @@
 # The runatlantis/atlantis-base is created by docker-base/Dockerfile.
-FROM runatlantis/atlantis-base:v3.2
+FROM runatlantis/atlantis-base:v3.3
 LABEL authors="Anubhav Mishra, Luke Kysow"
 
 # install terraform binaries
-ENV DEFAULT_TERRAFORM_VERSION=0.12.24
+ENV DEFAULT_TERRAFORM_VERSION=0.12.26
 
 # In the official Atlantis image we only have the latest of each Terraform version.
 RUN AVAILABLE_TERRAFORM_VERSIONS="0.8.8 0.9.11 0.10.8 0.11.14 ${DEFAULT_TERRAFORM_VERSION}" && \
     for VERSION in ${AVAILABLE_TERRAFORM_VERSIONS}; do \
-        curl -LOks https://releases.hashicorp.com/terraform/${VERSION}/terraform_${VERSION}_linux_amd64.zip && \
-        curl -LOks https://releases.hashicorp.com/terraform/${VERSION}/terraform_${VERSION}_SHA256SUMS && \
+        curl -LOs https://releases.hashicorp.com/terraform/${VERSION}/terraform_${VERSION}_linux_amd64.zip && \
+        curl -LOs https://releases.hashicorp.com/terraform/${VERSION}/terraform_${VERSION}_SHA256SUMS && \
         sed -n "/terraform_${VERSION}_linux_amd64.zip/p" terraform_${VERSION}_SHA256SUMS | sha256sum -c && \
         mkdir -p /usr/local/bin/tf/versions/${VERSION} && \
         unzip terraform_${VERSION}_linux_amd64.zip -d /usr/local/bin/tf/versions/${VERSION} && \

Makefile

@@ -28,8 +28,8 @@ go-generate: ## Run go generate in all packages
 regen-mocks: ## Delete all mocks and matchers and then run go generate to regen them.
 	find . -type f | grep mocks/mock_ | grep -v vendor | xargs rm
 	find . -type f | grep mocks/matchers | grep -v vendor | xargs rm
-	@# not using $(PKG) here because that it includes directories that have now
-	@# been deleted, causing go generate to fail.
+	@# not using $(PKG) here because that includes directories that have now
+	@# been made empty, causing go generate to fail.
 	go list ./... | grep -v e2e | grep -v vendor | grep -v static | xargs go generate
 
 test: ## Run tests
@@ -47,6 +47,10 @@ test-coverage-html:
 	@go test -covermode atomic -coverpkg $(PKG_COMMAS) -coverprofile .cover/cover.out $(PKG)
 	go tool cover -html .cover/cover.out
 
+dev-docker:
+	GOOS=linux GOARCH=amd64 go build -mod=vendor -o atlantis .
+	docker build -f dev-Dockerfile -t atlantis-dev .
+
 dist: ## Package up everything in static/ using go-bindata-assetfs so it can be served by a single binary
 	rm -f server/static/bindata_assetfs.go && go-bindata-assetfs -pkg static -prefix server server/static/... && mv bindata_assetfs.go server/static
 

cmd/server.go

@@ -64,6 +64,7 @@ const (
 	GitlabWebhookSecretFlag    = "gitlab-webhook-secret" // nolint: gosec
 	HidePrevPlanComments       = "hide-prev-plan-comments"
 	LogLevelFlag               = "log-level"
+	AllowDraftPRs              = "allow-draft-prs"
 	PortFlag                   = "port"
 	RepoConfigFlag             = "repo-config"
 	RepoConfigJSONFlag         = "repo-config-json"
@@ -253,6 +254,10 @@ var boolFlags = map[string]boolFlag{
 		description:  "Disable \"atlantis apply\" command so a specific project/workspace/directory has to be specified for applies.",
 		defaultValue: false,
 	},
+	AllowDraftPRs: {
+		description:  "Enable autoplan for Github Draft Pull Requests",
+		defaultValue: false,
+	},
 	HidePrevPlanComments: {
 		description: "Hide previous plan comments to reduce clutter in the PR. " +
 			"VCS support is limited to: GitHub.",

cmd/server_test.go

@@ -75,6 +75,7 @@ var testFlags = map[string]interface{}{
 	GitlabUserFlag:             "gitlab-user",
 	GitlabWebhookSecretFlag:    "gitlab-secret",
 	LogLevelFlag:               "debug",
+	AllowDraftPRs:              true,
 	PortFlag:                   8181,
 	RepoWhitelistFlag:          "github.com/runatlantis/atlantis",
 	RequireApprovalFlag:        true,

dev-Dockerfile

@@ -0,0 +1,2 @@
+FROM runatlantis/atlantis:latest
+COPY atlantis /usr/local/bin/atlantis

go.mod

@@ -18,7 +18,7 @@ require (
 	github.com/go-playground/locales v0.12.1 // indirect
 	github.com/go-playground/universal-translator v0.16.0 // indirect
 	github.com/go-test/deep v1.0.3
-	github.com/google/go-github/v28 v28.0.0
+	github.com/google/go-github/v31 v31.0.0
 	github.com/google/uuid v0.0.0-20161128191214-064e2069ce9c // indirect
 	github.com/gorilla/context v0.0.0-20160226214623-1ea25387ff6f // indirect
 	github.com/gorilla/mux v1.6.2
@@ -42,6 +42,7 @@ require (
 	github.com/pelletier/go-toml v1.0.0 // indirect
 	github.com/petergtz/pegomock v2.7.0+incompatible
 	github.com/pkg/errors v0.8.0
+	github.com/remeh/sizedwaitgroup v1.0.0
 	github.com/shurcooL/githubv4 v0.0.0-20191127044304-8f68eb5628d0
 	github.com/shurcooL/graphql v0.0.0-20181231061246-d48a9a75455f // indirect
 	github.com/sirupsen/logrus v1.2.0 // indirect

go.sum

@@ -74,6 +74,7 @@ github.com/google/go-cmp v0.3.1 h1:Xye71clBPdm5HgqGwUkwhbynsUJZhDbS20FvLhQ2izg=
 github.com/google/go-cmp v0.3.1/go.mod h1:8QqcDgzrUqlUb/G2PQTWiueGozuR1884gddMywk6iLU=
 github.com/google/go-github/v28 v28.0.0 h1:+UjHI4+1W/vsXR4jJBWt0ZA74XHbvt5yBAvsf1M3bgM=
 github.com/google/go-github/v28 v28.0.0/go.mod h1:+5GboIspo7F0NG2qsvfYh7en6F3EK37uyqv+c35AR3s=
+github.com/google/go-github/v31 v31.0.0/go.mod h1:NQPZol8/1sMoWYGN2yaALIBytu17gAWfhbweiEed3pM=
 github.com/google/go-querystring v1.0.0 h1:Xkwi/a1rcvNg1PPYe5vI8GbeBY/jrVuDX5ASuANWTrk=
 github.com/google/go-querystring v1.0.0/go.mod h1:odCYkC5MyYFN7vkCjXpyrEuKhc/BUO6wN/zVPAxq5ck=
 github.com/google/martian v2.1.0+incompatible h1:/CP5g8u/VJHijgedC/Legn3BAbAaWPgecwXBIDzw5no=
@@ -177,6 +178,8 @@ github.com/pkg/errors v0.8.0 h1:WdK/asTD0HN+q6hsWO3/vpuAkAr+tw6aNJNDFFf0+qw=
 github.com/pkg/errors v0.8.0/go.mod h1:bwawxfHBFNV+L2hUp1rHADufV3IMtnDRdf1r5NINEl0=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
 github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
+github.com/remeh/sizedwaitgroup v1.0.0 h1:VNGGFwNo/R5+MJBf6yrsr110p0m4/OX4S3DCy7Kyl5E=
+github.com/remeh/sizedwaitgroup v1.0.0/go.mod h1:3j2R4OIe/SeS6YDhICBy22RWjJC5eNCJ1V+9+NVNYlo=
 github.com/sergi/go-diff v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
 github.com/sergi/go-diff v1.0.0/go.mod h1:0CfEIISq7TuYL3j771MWULgwwjU+GofnZX9QAmXWZgo=
 github.com/shurcooL/githubv4 v0.0.0-20191127044304-8f68eb5628d0 h1:T9uus1QvcPgeLShS30YOnnzk3r9Vvygp45muhlrufgY=
@@ -292,6 +295,7 @@ golang.org/x/tools v0.0.0-20190312170243-e65039ee4138/go.mod h1:LCzVGOaR6xXOjkQ3
 golang.org/x/tools v0.0.0-20190425150028-36563e24a262/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
 golang.org/x/tools v0.0.0-20190506145303-2d16b83fe98c/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
 golang.org/x/tools v0.0.0-20190606124116-d0a3d012864b/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
+golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0 h1:Dh6fw+p6FyRl5x/FvNswO1ji0lIGzm3KP8Y9VkS9PTE=
 golang.org/x/tools v0.0.0-20190628153133-6cdbf07be9d0/go.mod h1:/rFqwRUd4F7ZHNgwSSTFct+R/Kf4OFW1sUzUTQQTgfc=
 google.golang.org/api v0.4.0/go.mod h1:8k5glujaEP+g9n7WNsDg8QP6cUVNI86fCNMcbazEtwE=
 google.golang.org/api v0.7.0/go.mod h1:WtwebWUNSVBH/HAw79HIFXZNqEvBhG+Ra+ax0hx3E3M=

kustomize/bundle.yaml

@@ -22,7 +22,7 @@ spec:
         fsGroup: 1000 # Atlantis group (1000) read/write access to volumes.
       containers:
       - name: atlantis
-        image: runatlantis/atlantis:v0.11.1
+        image: runatlantis/atlantis:v0.13.0
         env:
         - name: ATLANTIS_DATA_DIR
           value: /atlantis

main.go

@@ -20,7 +20,7 @@ import (
 	"github.com/spf13/viper"
 )
 
-const atlantisVersion = "0.11.1"
+const atlantisVersion = "0.13.0"
 
 func main() {
 	v := viper.New()

runatlantis.io/docs/custom-workflows.md

@@ -349,6 +349,7 @@ Or a custom command
   * `PROJECT_NAME` - Name of the project configured in `atlantis.yaml`. If no project name is configured this will be an empty string.
   * `PULL_NUM` - Pull request number or ID, ex. `2`.
   * `PULL_AUTHOR` - Username of the pull request author, ex. `acme-user`.
+  * `REPO_REL_DIR` - The relative path of the project in the repository. For example if your project is in `dir1/dir2/` then this will be set to `"dir1/dir2"`. If your project is at the root this will be `"."`.
   * `USER_NAME` - Username of the VCS user running command, ex. `acme-user`. During an autoplan, the user will be the Atlantis API user, ex. `atlantis`.
   * `COMMENT_ARGS` - Any additional flags passed in the comment on the pull request. Flags are separated by commas and
   every character is escaped, ex. `atlantis plan -- arg1 arg2` will result in `COMMENT_ARGS=\a\r\g\1,\a\r\g\2`.

runatlantis.io/docs/server-configuration.md

@@ -15,6 +15,10 @@ All flags can be specified as environment variables.
 1. Uppercase all the letters => `GH_USER`
 1. Prefix with `ATLANTIS_` => `ATLANTIS_GH_USER`
 
+::: warning NOTE
+To set a boolean flag use `true` or `false` as the value.
+:::
+
 ::: warning NOTE
 The flag `--atlantis-url` is set by the environment variable `ATLANTIS_ATLANTIS_URL` **NOT** `ATLANTIS_URL`.
 :::