Terragrunt support (#32)

* Add Terragrunt integration and general parser (#24) * - add kw parser - add tg command methods * - add plan_all parse output progress - replace terra* command func cli params with **kw * add tgbinary to ci * - add tg shared doc string decorator - separate some of the public methods tg_xxx() | tf_xxx() - fix parse_args() styling * - Rollback parse_args() - Add tg global args to parse_args() - Add tg global args decorator - Add all param to tg_xxx methods * - mv tg flags before tf flags for cli args - add common tg args to tg methods * - fix tg source module warning - add output() fixture to tg test * bump version * Terragrunt support (#29) * Don't fail when resource_changes key not present in output (#27) * Resolves #26 tftest failed when terraform plan is empty * add missing copyright boilerplate Co-authored-by: Ludovico Magnocavallo <[email protected]> * Update CHANGELOG.md * Update CHANGELOG.md * v1.5.6 for pypi * made minimal changes to fix tests (now all passed locally for me) * added a bit of code to parse run-all output and test to assert output * added terragrunt run in single dir test case + simplified run-all * added test cases for terragrunt parse args * refactored terragrunt cmd line parameters parsing * refactored as a separated class(TerragruntTest) but reused most of TerraformTest Co-authored-by: Mike Brown II <[email protected]> Co-authored-by: Ludovico Magnocavallo <[email protected]> Co-authored-by: Ludovico Magnocavallo <[email protected]> * speed up tests(session scope to reuse fixtures) + work around for strange tf run-all plan behaviour * added back change log * adding newline at end of file * Adding terragrunt section to readme * added some doc strings * remove binary param from _clean_up to fix tests * Update README.md * Update README.md * Update CHANGELOG.md * Update tftest.py Co-authored-by: Marshall Mamiya <[email protected]> Co-authored-by: Ludovico Magnocavallo <[email protected]> Co-authored-by: Mike Brown II <[email protected]> Co-authored-by: Ludovico Magnocavallo <[email protected]>
GoogleCloudPlatform · Jun 7, 2021 · 09e532b · 09e532b
1 parent 8106e54
commit 09e532b
Show file tree

Hide file tree

Showing 13 changed files with 354 additions and 35 deletions.
diff --git a/.ci/cloudbuild.test.yaml b/.ci/cloudbuild.test.yaml
@@ -23,7 +23,9 @@ steps:
         wget https://releases.hashicorp.com/terraform/${_TERRAFORM_VERSION}/terraform_${_TERRAFORM_VERSION}_linux_amd64.zip &&
         unzip terraform_${_TERRAFORM_VERSION}_linux_amd64.zip -d /builder/home/.local/bin &&
         rm terraform_${_TERRAFORM_VERSION}_linux_amd64.zip &&
-        chmod 755 /builder/home/.local/bin/terraform
+        wget -O /builder/home/.local/bin/terragrunt https://github.com/gruntwork-io/terragrunt/releases/download/v${_TERRAGRUNT_VERSION}/terragrunt_linux_amd64 &&
+        chmod 755 /builder/home/.local/bin/terraform &&
+        chmod 755 /builder/home/.local/bin/terragrunt
   # TODO(ludoo): split into two triggers with different filters
   - name: python:3.6-alpine
     id: test
@@ -36,6 +38,7 @@ steps:
 
 substitutions:
   _TERRAFORM_VERSION: 0.15.4
+  _TERRAGRUNT_VERSION: 0.29.8
 
 tags:
   - "ci"

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -7,6 +7,8 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+- Terragrunt support (@davidtam) [#32]
+
 ## [1.5.7]
 
 - improve Windows support [#28]

diff --git a/README.md b/README.md
@@ -6,7 +6,7 @@ It allows for different types of tests: lightweight tests that only use Terrafor
 
 As an additional convenience, the module also provides an easy way to request and access the plan output (via `terraform plan -out` and `terraform show`) and the outputs (via `terraform output -json`), and return them wrapped in simple classes that streamline accessing their attributes.
 
-This module is heavily inspired by two projects: [Terragrunt](https://github.com/gruntwork-io/terragrunt) for the lightweight approach to testing Terraform, and [python-terraform](https://github.com/beelit94/python-terraform) for wrapping the Terraform command in Python.
+This module is heavily inspired by two projects: [Terratest](https://github.com/gruntwork-io/terratest) for the lightweight approach to testing Terraform, and [python-terraform](https://github.com/beelit94/python-terraform) for wrapping the Terraform command in Python.
 
 ## Example Usage
 
@@ -46,6 +46,38 @@ def test_modules(plan):
   assert res['values']['location'] == plan.variables['gcs_location']
 ```
 
+## Terragrunt support
+
+Support for Terragrunt actually follows the same principle of the thin `TerraformTest` wrapper.
+
+Please see the following example for how to use it:
+
+```python
+import pytest
+import tftest
+
+
+@pytest.fixture
+def run_all_apply_out(fixtures_dir):
+  # notice for run-all, you need to specify when TerragruntTest is constructed
+  tg = tftest.TerragruntTest('tg_apply_all', fixtures_dir, tg_run_all=True)
+  # the rest is very similar to how you use TerraformTest
+  tg.setup()
+  # to use --terragrunt-<option>, pass in tg_<option in snake case>
+  tg.apply(output=False, tg_non_interactive=True)
+  yield tg.output()
+  tg.destroy(auto_approve=True, tg_non_interactive=True)
+
+
+def test_run_all_apply(run_all_apply_out):
+    triggers = [o["triggers"] for o in run_all_apply_out]
+    assert [{'name': 'foo', 'template': 'sample template foo'}] in triggers
+    assert [{'name': 'bar', 'template': 'sample template bar'}] in triggers
+    assert [{'name': 'one', 'template': 'sample template one'},
+            {'name': 'two', 'template': 'sample template two'}] in triggers
+    assert len(run_all_apply_out) == 3
+```
+
 ## Compatibility
 
 Starting from version `1.0.0` Terraform `0.12` is required, and tests written with previous versions of this module are incompatible. Check the [`CHANGELOG.md`](https://github.com/GoogleCloudPlatform/terraform-python-testing-helper/blob/master/CHANGELOG.md) file for details on what's changed.

diff --git a/test/fixtures/tg_apply_all/bar/terragrunt.hcl b/test/fixtures/tg_apply_all/bar/terragrunt.hcl
@@ -0,0 +1,11 @@
+include {
+  path = find_in_parent_folders()
+}
+
+terraform {
+  source = "../..//apply"
+}
+
+inputs = {
+    names = ["bar"]
+}
diff --git a/test/fixtures/tg_apply_all/foo/terragrunt.hcl b/test/fixtures/tg_apply_all/foo/terragrunt.hcl
@@ -0,0 +1,11 @@
+include {
+  path = find_in_parent_folders()
+}
+
+terraform {
+  source = "../..//apply"
+}
+
+inputs = {
+    names = ["foo"]
+}
diff --git a/test/fixtures/tg_apply_all/terragrunt.hcl b/test/fixtures/tg_apply_all/terragrunt.hcl
@@ -0,0 +1,11 @@
+terraform {
+  source = "..//apply"
+}
+
+generate "provider" {
+  path      = "provider.tf"
+  if_exists = "overwrite_terragrunt"
+  contents  = <<EOF
+provider "null" {}
+EOF
+}
diff --git a/test/test_args.py b/test/test_args.py
@@ -13,6 +13,7 @@
 # limitations under the License.
 
 "Test the function for mapping Terraform arguments."
+import pytest
 
 import tftest
 
@@ -45,10 +46,52 @@
 )
 
 
-def test_args():
+@pytest.mark.parametrize("kwargs, expected", ARGS_TESTS)
+def test_args(kwargs, expected):
   assert tftest.parse_args() == []
-  for kwargs, expected in ARGS_TESTS:
-    assert tftest.parse_args(**kwargs) == expected
+  assert tftest.parse_args(**kwargs) == expected
+
+
+TERRAGRUNT_ARGS_TESTCASES = [
+    ({"tg_config": "Obama"}, ['--terragrunt-config', 'Obama']),
+    ({"tg_tfpath": "Barrack"}, ['--terragrunt-tfpath', 'Barrack']),
+    ({"tg_no_auto_init": True}, ['--terragrunt-no-auto-init']),
+    ({"tg_no_auto_init": False}, []),
+    ({"tg_no_auto_retry": True}, ['--terragrunt-no-auto-retry']),
+    ({"tg_no_auto_retry": False}, []),
+    ({"tg_non_interactive": True}, ['--terragrunt-non-interactive']),
+    ({"tg_non_interactive": False}, []),
+    ({"tg_working_dir": "George"}, ['--terragrunt-working-dir', 'George']),
+    ({"tg_download_dir": "Bush"}, ['--terragrunt-download-dir', 'Bush']),
+    ({"tg_source": "Clinton"}, ['--terragrunt-source', 'Clinton']),
+    ({"tg_source_update": True}, ['--terragrunt-source-update']),
+    ({"tg_source_update": False}, []),
+    ({"tg_iam_role": "Bill"}, ['--terragrunt-iam-role', 'Bill']),
+    ({"tg_ignore_dependency_errors": True}, ['--terragrunt-ignore-dependency-errors']),
+    ({"tg_ignore_dependency_errors": False}, []),
+    ({"tg_ignore_dependency_order": True}, ['--terragrunt-ignore-dependency-order']),
+    ({"tg_ignore_dependency_order": False}, []),
+    ({"tg_ignore_external_dependencies": "dont care what is here"},
+     ['--terragrunt-ignore-external-dependencies']),
+    ({"tg_include_external_dependencies": True}, ['--terragrunt-include-external-dependencies']),
+    ({"tg_include_external_dependencies": False}, []),
+    ({"tg_parallelism": 20}, ['--terragrunt-parallelism 20']),
+    ({"tg_exclude_dir": "Ronald"}, ['--terragrunt-exclude-dir', 'Ronald']),
+    ({"tg_include_dir": "Reagan"}, ['--terragrunt-include-dir', 'Reagan']),
+    ({"tg_check": True}, ['--terragrunt-check']),
+    ({"tg_check": False}, []),
+    ({"tg_hclfmt_file": "Biden"}, ['--terragrunt-hclfmt-file', 'Biden']),
+    ({"tg_override_attr": {"Iron": "Man", "Captain": "America"}},
+     ['--terragrunt-override-attr=Iron=Man', '--terragrunt-override-attr=Captain=America']),
+    ({"tg_debug": True}, ['--terragrunt-debug']),
+    ({"tg_debug": False}, []),
+
+]
+
+
+@pytest.mark.parametrize("kwargs, expected", TERRAGRUNT_ARGS_TESTCASES)
+def test_terragrunt_args(kwargs, expected):
+  assert tftest.parse_args(**kwargs) == expected
 
 
 def test_var_args():

diff --git a/test/test_no_outputs.py b/test/test_no_outputs.py
@@ -18,7 +18,7 @@
 import tftest
 
 
-@pytest.fixture
+@pytest.fixture(scope="module")
 def plan(fixtures_dir):
   tf = tftest.TerraformTest('no_outputs', fixtures_dir)
   tf.setup(extra_files=['plan.auto.tfvars'])

diff --git a/test/test_plan.py b/test/test_plan.py
@@ -18,7 +18,7 @@
 import tftest
 
 
-@pytest.fixture
+@pytest.fixture(scope="module")
 def plan_out(fixtures_dir):
   import json
   with open('%s/plan_output.json' % fixtures_dir) as fp:

diff --git a/test/test_sample_apply.py b/test/test_sample_apply.py
@@ -24,7 +24,7 @@ def output(fixtures_dir):
   tf.setup()
   tf.apply()
   yield tf.output()
-  tf.destroy()
+  tf.destroy(**{"auto_approve": True})
 
 
 def test_apply(output):

diff --git a/test/test_sample_plan.py b/test/test_sample_plan.py
@@ -18,7 +18,7 @@
 import tftest
 
 
-@pytest.fixture
+@pytest.fixture(scope="module")
 def plan(fixtures_dir):
   tf = tftest.TerraformTest('plan', fixtures_dir)
   tf.setup(extra_files=['plan.auto.tfvars'])

diff --git a/test/test_tg_all.py b/test/test_tg_all.py
@@ -0,0 +1,77 @@
+# Copyright 2019 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     https://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+import pytest
+import tftest
+
+import os
+
+
+@pytest.fixture
+def run_all_apply_out(fixtures_dir):
+  tg = tftest.TerragruntTest('tg_apply_all', fixtures_dir, tg_run_all=True)
+  tg.setup()
+  tg.apply(output=False, tg_non_interactive=True)
+  yield tg.output()
+  tg.destroy(auto_approve=True, tg_non_interactive=True)
+
+
+@pytest.fixture
+def run_all_plan_output(fixtures_dir):
+  tg = tftest.TerragruntTest('tg_apply_all', fixtures_dir, tg_run_all=True)
+  tg.setup()
+  return tg.plan(output=True, tg_working_dir=os.path.join(fixtures_dir, 'tg_apply_all'))
+
+
+@pytest.fixture
+def plan_foo_output(fixtures_dir):
+  tg = tftest.TerragruntTest(os.path.join('tg_apply_all', 'foo'), fixtures_dir)
+  tg.setup()
+  return tg.plan(output=True)
+
+
+@pytest.fixture
+def bar_output(fixtures_dir):
+  tg = tftest.TerragruntTest(os.path.join('tg_apply_all', 'bar'), fixtures_dir)
+  tg.setup()
+  tg.apply(output=False, tg_non_interactive=True)
+  yield tg.output()
+  tg.destroy(auto_approve=True, tg_non_interactive=True)
+
+
+def test_run_all_apply(run_all_apply_out):
+  triggers = [o["triggers"] for o in run_all_apply_out]
+  assert [{'name': 'foo', 'template': 'sample template foo'}] in triggers
+  assert [{'name': 'bar', 'template': 'sample template bar'}] in triggers
+  assert [{'name': 'one', 'template': 'sample template one'},
+          {'name': 'two', 'template': 'sample template two'}] in triggers
+  assert len(run_all_apply_out) == 3
+
+
+def test_tg_single_directory_apply(bar_output):
+  assert bar_output["triggers"] == [{'name': 'bar', 'template': 'sample template bar'}]
+
+
+def test_run_all_plan(run_all_plan_output):
+  triggers = [o.outputs["triggers"] for o in run_all_plan_output]
+  assert [{'name': 'foo', 'template': 'sample template foo'}] in triggers
+  assert [{'name': 'bar', 'template': 'sample template bar'}] in triggers
+  assert [{'name': 'one', 'template': 'sample template one'},
+          {'name': 'two', 'template': 'sample template two'}] in triggers
+  assert len(run_all_plan_output) == 3
+
+
+def test_tg_single_directory_plan(plan_foo_output):
+  assert plan_foo_output.outputs['triggers'] == [{'name': 'foo', 'template': 'sample template foo'}]
+  assert plan_foo_output.variables['names'] == ['foo']