feat: state rm (runatlantis#2880)

* feat: state rm * review feedback * fix conflict for pegomock generation code * adopt state command into allow-commands * fix conflicts * fix: state rm works on workspace * notify import/state rm discard plan file * fix lint * use repeat instead warning for re-plan * perl -pi -e 's!\* 🔁 plan file was discarded. to!🚮 A plan file was discarded. Re-plan would be required before applying.\n\n\* 🔁 To!g' server/**/* * follow main branch
vincentgna · Jan 19, 2023 · cb0aadf · cb0aadf
1 parent 9e3d887
commit cb0aadf
Show file tree

Hide file tree

Showing 81 changed files with 1,793 additions and 217 deletions.
diff --git a/go.mod b/go.mod
@@ -43,6 +43,7 @@ require (
 	github.com/xanzy/go-gitlab v0.78.0
 	go.etcd.io/bbolt v1.3.6
 	go.uber.org/zap v1.24.0
+	golang.org/x/exp v0.0.0-20230113152452-c42ee1cf562e
 	golang.org/x/term v0.4.0
 	golang.org/x/text v0.6.0
 	gopkg.in/yaml.v2 v2.4.0

diff --git a/go.sum b/go.sum
@@ -508,6 +508,8 @@ golang.org/x/exp v0.0.0-20191227195350-da58074b4299/go.mod h1:2RIsYlXP63K8oxa1u0
 golang.org/x/exp v0.0.0-20200119233911-0405dc783f0a/go.mod h1:2RIsYlXP63K8oxa1u096TMicItID8zy7Y6sNkU49FU4=
 golang.org/x/exp v0.0.0-20200207192155-f17229e696bd/go.mod h1:J/WKrq2StrnmMY6+EHIKF9dgMWnmCNThgcyBT1FY9mM=
 golang.org/x/exp v0.0.0-20200224162631-6cc2880d07d6/go.mod h1:3jZMyOhIsHpP37uCMkUooju7aAi5cS1Q23tOzKc+0MU=
+golang.org/x/exp v0.0.0-20230113152452-c42ee1cf562e h1:uGuXqQsI2BAE8xNqSqNxhTdDdhlvpBvWFw/KBwtCtjI=
+golang.org/x/exp v0.0.0-20230113152452-c42ee1cf562e/go.mod h1:CxIveKay+FTh1D0yPZemJVgC/95VzuuOLq5Qi4xnoYc=
 golang.org/x/image v0.0.0-20190227222117-0694c2d4d067/go.mod h1:kZ7UVZpmo3dzQBMxlp+ypCbDeSB+sBbTgSJuh5dn5js=
 golang.org/x/image v0.0.0-20190802002840-cff245a6509b/go.mod h1:FeLwcggjj3mMvU+oOTbSwawSJRM1uh48EjtB4UJZlP0=
 golang.org/x/lint v0.0.0-20181026193005-c67002cb31c3/go.mod h1:UVdnD1Gm6xHRNCYTkRU2/jEulfH38KcIWyp/GAMgvoE=

diff --git a/runatlantis.io/docs/custom-workflows.md b/runatlantis.io/docs/custom-workflows.md
@@ -56,6 +56,11 @@ workflows:
         - init
         - import:
             extra_args: ["-var-file", "production.tfvars"]
+    state_rm:
+      steps:
+        - init
+        - state_rm:
+            extra_args: ["-lock=false"]
 ```
 Then in your repo-level `atlantis.yaml` file, you would reference the workflows:
 ```yaml
@@ -374,13 +379,15 @@ projects:
 plan:
 apply:
 import:
+state_rm:
 ```
 
-| Key    | Type            | Default                 | Required | Description                      |
-|--------|-----------------|-------------------------|----------|----------------------------------|
-| plan   | [Stage](#stage) | `steps: [init, plan]`   | no       | How to plan for this project.    |
-| apply  | [Stage](#stage) | `steps: [apply]`        | no       | How to apply for this project.   |
-| import | [Stage](#stage) | `steps: [init, import]` | no       | How to import for this project.  |
+| Key      | Type            | Default                   | Required | Description                           |
+|----------|-----------------|---------------------------|----------|---------------------------------------|
+| plan     | [Stage](#stage) | `steps: [init, plan]`     | no       | How to plan for this project.         |
+| apply    | [Stage](#stage) | `steps: [apply]`          | no       | How to apply for this project.        |
+| import   | [Stage](#stage) | `steps: [init, import]`   | no       | How to import for this project.       |
+| state_rm | [Stage](#stage) | `steps: [init, state_rm]` | no       | How to run state rm for this project. |
 
 ### Stage
 ```yaml
@@ -403,10 +410,11 @@ Steps can be a single string for a built-in command.
 - plan
 - apply
 - import
+- state_rm
 ```
-| Key                    | Type   | Default | Required | Description                                                                                                      |
-|------------------------|--------|---------|----------|------------------------------------------------------------------------------------------------------------------|
-| init/plan/apply/import | string | none    | no       | Use a built-in command without additional configuration. Only `init`, `plan`, `apply` and `import` are supported |
+| Key                             | Type   | Default | Required | Description                                                                                                                  |
+|---------------------------------|--------|---------|----------|------------------------------------------------------------------------------------------------------------------------------|
+| init/plan/apply/import/state_rm | string | none    | no       | Use a built-in command without additional configuration. Only `init`, `plan`, `apply`, `import` and `state_rm` are supported |
 
 #### Built-In Command With Extra Args
 A map from string to `extra_args` for a built-in command with extra arguments.
@@ -419,10 +427,12 @@ A map from string to `extra_args` for a built-in command with extra arguments.
     extra_args: [arg1, arg2]
 - import:
     extra_args: [arg1, arg2]
+- state_rm:
+    extra_args: [arg1, arg2]
 ```
-| Key                    | Type                               | Default | Required | Description                                                                                                                                                   |
-|------------------------|------------------------------------|---------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------|
-| init/plan/apply/import | map[`extra_args` -> array[string]] | none    | no       | Use a built-in command and append `extra_args`. Only `init`, `plan`, `apply` and `import` are supported as keys and only `extra_args` is supported as a value |
+| Key                             | Type                               | Default | Required | Description                                                                                                                                                               |
+|---------------------------------|------------------------------------|---------|----------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
+| init/plan/apply/import/state_rm | map[`extra_args` -> array[string]] | none    | no       | Use a built-in command and append `extra_args`. Only `init`, `plan`, `apply`, `import` and `state_rm` are supported as keys and only `extra_args` is supported as a value |
 
 #### Custom `run` Command
 Or a custom command

diff --git a/runatlantis.io/docs/server-configuration.md b/runatlantis.io/docs/server-configuration.md
@@ -57,7 +57,7 @@ Values are chosen in this order:
 
   Notes:
   * Accepts a comma separated list, ex. `command1,command2`.
-  * `version`, `plan`, `apply`, `unlock`, `approve_policies`, `import` and `all` are available.
+  * `version`, `plan`, `apply`, `unlock`, `approve_policies`, `import`, `state` and `all` are available.
   * `all` is a special keyword that allows all commands. If pass `all` then all other commands will be ignored.
 
 ### `--allow-draft-prs`

diff --git a/runatlantis.io/docs/using-atlantis.md b/runatlantis.io/docs/using-atlantis.md
@@ -169,6 +169,51 @@ atlantis import -d dir 'aws_instance.example["foo"]' i-1234567890abcdef0 -- -var
 ```
 If a flag is needed to be always appended, see [Custom Workflow Use Cases](custom-workflows.html#adding-extra-arguments-to-terraform-commands).
 
+---
+## atlantis state rm
+```bash
+atlantis state [options] rm ADDRESS... -- [terraform state rm flags]
+```
+### Explanation
+Runs `terraform state rm` that matches the directory/project/workspace.
+This command discards the terraform plan result. After run state rm and before an apply, another `atlantis plan` must be run again.
+
+To allow the `state` command requires [--allow-commands](/docs/server-configuration.html#allow-commands) configuration.
+
+### Examples
+```bash
+# Runs state rm
+atlantis state rm ADDRESS1 ADDRESS2
+
+# Runs state rm in the root directory of the repo with workspace `default`
+atlantis state -d . rm ADDRESS
+
+# Runs state rm in the `project1` directory of the repo with workspace `default`
+atlantis state -d project1 rm ADDRESS
+
+# Runs state rm in the root directory of the repo with workspace `staging`
+atlantis state -w staging rm ADDRESS
+```
+
+::: tip
+* If run state rm to for_each resources, it requires a single quoted address.
+  * ex. `atlantis state rm 'aws_instance.example["foo"]'`
+:::
+
+### Options
+* `-d directory` Run state rm a resource for this directory, relative to root of repo. Use `.` for root.
+* `-p project` Run state rm a resource for this project. Refers to the name of the project configured in the repo's [`atlantis.yaml`](repo-level-atlantis-yaml.html) repo configuration file. This cannot be used at the same time as `-d` or `-w`.
+* `-w workspace` Run state rm a resource for a specific [Terraform workspace](https://developer.hashicorp.com/terraform/language/state/workspaces). Ignore this if Terraform workspaces are unused.
+
+### Additional Terraform flags
+
+If `terraform state rm` requires additional arguments, like `-lock=false'`
+append them to the end of the comment after `--`, e.g.
+```
+atlantis state -d dir rm 'aws_instance.example["foo"]' -- -lock=false
+```
+If a flag is needed to be always appended, see [Custom Workflow Use Cases](custom-workflows.html#adding-extra-arguments-to-terraform-commands).
+
 ---
 ## atlantis unlock
 ```bash

diff --git a/server/controllers/events/events_controller_e2e_test.go b/server/controllers/events/events_controller_e2e_test.go
@@ -497,6 +497,71 @@ func TestGitHubWorkflow(t *testing.T) {
 				{"exp-output-merge.txt"},
 			},
 		},
+		{
+			Description:   "state rm single project",
+			RepoDir:       "state-rm-single-project",
+			ModifiedFiles: []string{"main.tf"},
+			ExpAutoplan:   true,
+			Comments: []string{
+				"atlantis import random_id.simple AA",
+				"atlantis import 'random_id.for_each[\"overridden\"]' BB -- -var var=overridden",
+				"atlantis import random_id.count[0] BB",
+				"atlantis plan -- -var var=overridden",
+				"atlantis state rm 'random_id.for_each[\"overridden\"]' -- -lock=false",
+				"atlantis state rm random_id.count[0] random_id.simple",
+				"atlantis plan -- -var var=overridden",
+			},
+			ExpReplies: [][]string{
+				{"exp-output-autoplan.txt"},
+				{"exp-output-import-simple.txt"},
+				{"exp-output-import-foreach.txt"},
+				{"exp-output-import-count.txt"},
+				{"exp-output-plan.txt"},
+				{"exp-output-state-rm-foreach.txt"},
+				{"exp-output-state-rm-multiple.txt"},
+				{"exp-output-plan-again.txt"},
+				{"exp-output-merged.txt"},
+			},
+		},
+		{
+			Description: "state rm workspace",
+			RepoDir:     "state-rm-workspace",
+			Comments: []string{
+				"atlantis import -p dir1-ops 'random_id.dummy1[0]' AA",
+				"atlantis plan -p dir1-ops",
+				"atlantis state rm -p dir1-ops 'random_id.dummy1[0]'",
+				"atlantis plan -p dir1-ops",
+			},
+			ExpReplies: [][]string{
+				{"exp-output-import-dummy1.txt"},
+				{"exp-output-plan.txt"},
+				{"exp-output-state-rm-dummy1.txt"},
+				{"exp-output-plan-again.txt"},
+				{"exp-output-merge.txt"},
+			},
+		},
+		{
+			Description:   "state rm multiple project",
+			RepoDir:       "state-rm-multiple-project",
+			ModifiedFiles: []string{"dir1/main.tf", "dir2/main.tf"},
+			ExpAutoplan:   true,
+			Comments: []string{
+				"atlantis import -d dir1 random_id.dummy AA",
+				"atlantis import -d dir2 random_id.dummy BB",
+				"atlantis plan",
+				"atlantis state rm random_id.dummy",
+				"atlantis plan",
+			},
+			ExpReplies: [][]string{
+				{"exp-output-autoplan.txt"},
+				{"exp-output-import-dummy1.txt"},
+				{"exp-output-import-dummy2.txt"},
+				{"exp-output-plan.txt"},
+				{"exp-output-state-rm-multiple-projects.txt"},
+				{"exp-output-plan-again.txt"},
+				{"exp-output-merged.txt"},
+			},
+		},
 	}
 	for _, c := range cases {
 		t.Run(c.Description, func(t *testing.T) {
@@ -1152,7 +1217,8 @@ func setupE2E(t *testing.T, repoDir string, opt setupOption) (events_controllers
 		ApplyStepRunner: &runtime.ApplyStepRunner{
 			TerraformExecutor: terraformClient,
 		},
-		ImportStepRunner: runtime.NewImportStepRunner(terraformClient, defaultTFVersion),
+		ImportStepRunner:  runtime.NewImportStepRunner(terraformClient, defaultTFVersion),
+		StateRmStepRunner: runtime.NewStateRmStepRunner(terraformClient, defaultTFVersion),
 		RunStepRunner: &runtime.RunStepRunner{
 			TerraformExecutor:       terraformClient,
 			DefaultTFVersion:        defaultTFVersion,
@@ -1262,13 +1328,20 @@ func setupE2E(t *testing.T, repoDir string, opt setupOption) (events_controllers
 		projectCommandRunner,
 	)
 
+	stateCommandRunner := events.NewStateCommandRunner(
+		pullUpdater,
+		projectCommandBuilder,
+		projectCommandRunner,
+	)
+
 	commentCommandRunnerByCmd := map[command.Name]events.CommentCommandRunner{
 		command.Plan:            planCommandRunner,
 		command.Apply:           applyCommandRunner,
 		command.ApprovePolicies: approvePoliciesCommandRunner,
 		command.Unlock:          unlockCommandRunner,
 		command.Version:         versionCommandRunner,
 		command.Import:          importCommandRunner,
+		command.State:           stateCommandRunner,
 	}
 
 	commandRunner := &events.DefaultCommandRunner{

diff --git a/...ntrollers/events/testdata/test-repos/import-multiple-project/exp-output-import-dummy1.txt b/...ntrollers/events/testdata/test-repos/import-multiple-project/exp-output-import-dummy1.txt
@@ -12,5 +12,7 @@ The resources that were imported are shown above. These resources are now in
 your Terraform state and will henceforth be managed by Terraform.
 ```
 
+:put_litter_in_its_place: A plan file was discarded. Re-plan would be required before applying.
+
 * :repeat: To **plan** this project again, comment:
   * `atlantis plan -d dir1`
diff --git a/...trollers/events/testdata/test-repos/import-single-project-var/exp-output-import-count.txt b/...trollers/events/testdata/test-repos/import-single-project-var/exp-output-import-count.txt
@@ -12,5 +12,7 @@ The resources that were imported are shown above. These resources are now in
 your Terraform state and will henceforth be managed by Terraform.
 ```
 
+:put_litter_in_its_place: A plan file was discarded. Re-plan would be required before applying.
+
 * :repeat: To **plan** this project again, comment:
   * `atlantis plan -d .`
diff --git a/...ollers/events/testdata/test-repos/import-single-project-var/exp-output-import-foreach.txt b/...ollers/events/testdata/test-repos/import-single-project-var/exp-output-import-foreach.txt
@@ -12,5 +12,7 @@ The resources that were imported are shown above. These resources are now in
 your Terraform state and will henceforth be managed by Terraform.
 ```
 
+:put_litter_in_its_place: A plan file was discarded. Re-plan would be required before applying.
+
 * :repeat: To **plan** this project again, comment:
   * `atlantis plan -d .`
diff --git a/...controllers/events/testdata/test-repos/import-single-project/exp-output-import-dummy1.txt b/...controllers/events/testdata/test-repos/import-single-project/exp-output-import-dummy1.txt
@@ -12,5 +12,7 @@ The resources that were imported are shown above. These resources are now in
 your Terraform state and will henceforth be managed by Terraform.
 ```
 
+:put_litter_in_its_place: A plan file was discarded. Re-plan would be required before applying.
+
 * :repeat: To **plan** this project again, comment:
   * `atlantis plan -d .`
diff --git a/...controllers/events/testdata/test-repos/import-single-project/exp-output-import-dummy2.txt b/...controllers/events/testdata/test-repos/import-single-project/exp-output-import-dummy2.txt
@@ -12,5 +12,7 @@ The resources that were imported are shown above. These resources are now in
 your Terraform state and will henceforth be managed by Terraform.
 ```
 
+:put_litter_in_its_place: A plan file was discarded. Re-plan would be required before applying.
+
 * :repeat: To **plan** this project again, comment:
   * `atlantis plan -d .`
diff --git a/...rollers/events/testdata/test-repos/import-workspace/exp-output-import-dir1-ops-dummy1.txt b/...rollers/events/testdata/test-repos/import-workspace/exp-output-import-dir1-ops-dummy1.txt
@@ -12,5 +12,7 @@ The resources that were imported are shown above. These resources are now in
 your Terraform state and will henceforth be managed by Terraform.
 ```
 
+:put_litter_in_its_place: A plan file was discarded. Re-plan would be required before applying.
+
 * :repeat: To **plan** this project again, comment:
   * `atlantis plan -p dir1-ops`
diff --git a/...rollers/events/testdata/test-repos/import-workspace/exp-output-import-dir1-ops-dummy2.txt b/...rollers/events/testdata/test-repos/import-workspace/exp-output-import-dir1-ops-dummy2.txt
@@ -12,5 +12,7 @@ The resources that were imported are shown above. These resources are now in
 your Terraform state and will henceforth be managed by Terraform.
 ```
 
+:put_litter_in_its_place: A plan file was discarded. Re-plan would be required before applying.
+
 * :repeat: To **plan** this project again, comment:
   * `atlantis plan -p dir1-ops`
diff --git a/server/controllers/events/testdata/test-repos/state-rm-multiple-project/atlantis.yaml b/server/controllers/events/testdata/test-repos/state-rm-multiple-project/atlantis.yaml
@@ -0,0 +1,4 @@
+version: 3
+projects:
+- dir: dir1
+- dir: dir2
diff --git a/server/controllers/events/testdata/test-repos/state-rm-multiple-project/dir1/main.tf b/server/controllers/events/testdata/test-repos/state-rm-multiple-project/dir1/main.tf
@@ -0,0 +1,4 @@
+resource "random_id" "dummy" {
+  keepers     = {}
+  byte_length = 1
+}
diff --git a/server/controllers/events/testdata/test-repos/state-rm-multiple-project/dir2/main.tf b/server/controllers/events/testdata/test-repos/state-rm-multiple-project/dir2/main.tf
@@ -0,0 +1,4 @@
+resource "random_id" "dummy" {
+  keepers     = {}
+  byte_length = 1
+}
diff --git a/.../controllers/events/testdata/test-repos/state-rm-multiple-project/exp-output-autoplan.txt b/.../controllers/events/testdata/test-repos/state-rm-multiple-project/exp-output-autoplan.txt
@@ -0,0 +1,75 @@
+Ran Plan for 2 projects:
+
+1. dir: `dir1` workspace: `default`
+1. dir: `dir2` workspace: `default`
+
+### 1. dir: `dir1` workspace: `default`
+<details><summary>Show Output</summary>
+
+```diff
+Terraform used the selected providers to generate the following execution
+plan. Resource actions are indicated with the following symbols:
++ create
+
+Terraform will perform the following actions:
+
+  # random_id.dummy will be created
++ resource "random_id" "dummy" {
+      + b64_std     = (known after apply)
+      + b64_url     = (known after apply)
+      + byte_length = 1
+      + dec         = (known after apply)
+      + hex         = (known after apply)
+      + id          = (known after apply)
+      + keepers     = {}
+    }
+
+Plan: 1 to add, 0 to change, 0 to destroy.
+```
+
+* :arrow_forward: To **apply** this plan, comment:
+    * `atlantis apply -d dir1`
+* :put_litter_in_its_place: To **delete** this plan click [here](lock-url)
+* :repeat: To **plan** this project again, comment:
+    * `atlantis plan -d dir1`
+</details>
+Plan: 1 to add, 0 to change, 0 to destroy.
+
+---
+### 2. dir: `dir2` workspace: `default`
+<details><summary>Show Output</summary>
+
+```diff
+Terraform used the selected providers to generate the following execution
+plan. Resource actions are indicated with the following symbols:
++ create
+
+Terraform will perform the following actions:
+
+  # random_id.dummy will be created
++ resource "random_id" "dummy" {
+      + b64_std     = (known after apply)
+      + b64_url     = (known after apply)
+      + byte_length = 1
+      + dec         = (known after apply)
+      + hex         = (known after apply)
+      + id          = (known after apply)
+      + keepers     = {}
+    }
+
+Plan: 1 to add, 0 to change, 0 to destroy.
+```
+
+* :arrow_forward: To **apply** this plan, comment:
+    * `atlantis apply -d dir2`
+* :put_litter_in_its_place: To **delete** this plan click [here](lock-url)
+* :repeat: To **plan** this project again, comment:
+    * `atlantis plan -d dir2`
+</details>
+Plan: 1 to add, 0 to change, 0 to destroy.
+
+---
+* :fast_forward: To **apply** all unapplied plans from this pull request, comment:
+    * `atlantis apply`
+* :put_litter_in_its_place: To delete all plans and locks for the PR, comment:
+    * `atlantis unlock`