Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update aws-provider-patch to support nested attributes #1391

Merged
merged 1 commit into from
Oct 15, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
104 changes: 101 additions & 3 deletions cli/aws_provider_patch.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"github.com/zclconf/go-cty/cty"
"io/ioutil"
"path/filepath"
"strings"
)

// applyAwsProviderPatch finds all Terraform modules nested in the current code (i.e., in the .terraform/modules
Expand Down Expand Up @@ -173,16 +174,113 @@ func patchAwsProviderInTerraformCode(terraformCode string, terraformFilePath str
for _, block := range hclFile.Body().Blocks() {
if block.Type() == "provider" && len(block.Labels()) == 1 && block.Labels()[0] == "aws" {
for key, value := range attributesToOverride {
block.Body().SetAttributeValue(key, cty.StringVal(value))
attributeOverridden := overrideAttributeInBlock(block, key, value)
codeWasUpdated = codeWasUpdated || attributeOverridden
}

codeWasUpdated = true
}
}

return string(hclFile.Bytes()), codeWasUpdated, nil
}

// Override the attribute specified in the given key to the given value in a Terraform block: that is, if the attribute
// is already set, then update its value to the new value; if the attribute is not already set, do nothing. This method
// returns true if an attribute was overridden and false if nothing was changed.
//
// Note that you can set attributes within nested blocks by using a dot syntax similar to Terraform addresses: e.g.,
// "<NESTED_BLOCK>.<KEY>".
//
// Examples:
//
// Assume that block1 is:
//
// provider "aws" {
// region = var.aws_region
// assume_role {
// role_arn = var.role_arn
// }
// }
//
// If you call:
//
// overrideAttributeInBlock(block1, "region", "eu-west-1")
// overrideAttributeInBlock(block1, "assume_role.role_arn", "foo")
//
// The result would be:
//
// provider "aws" {
// region = "eu-west-1"
// assume_role {
// role_arn = "foo"
// }
// }
//
// Assume block2 is:
//
// provider "aws" {}
//
// If you call:
//
// overrideAttributeInBlock(block2, "region", "eu-west-1")
// overrideAttributeInBlock(block2, "assume_role.role_arn", "foo")
//
//
// The result would be:
//
// provider "aws" {}
func overrideAttributeInBlock(block *hclwrite.Block, key string, value string) bool {
body, attr := traverseBlock(block, strings.Split(key, "."))
if body == nil || body.GetAttribute(attr) == nil {
// We didn't find an existing block or attribute, so there's nothing to override
return false
}

body.SetAttributeValue(attr, cty.StringVal(value))
return true
}

// Given a Terraform block and slice of keys, return the body of the block that is indicated by the keys, and the
// attribute to set within that body. If the slice is of length one, this method returns the body of the current block
// and the one entry in the slice. However, if the slice contains multiple values, those indicate nested blocks, so
// this method will recursively descend into those blocks and return the body of the final one and the final entry in
// the slice to set on it. If a nested block is specified that doesn't actually exist, this method returns a nil body
// and empty string for the attribute.
//
// Examples:
//
// Assume block is:
//
// provider "aws" {
// region = var.aws_region
// assume_role {
// role_arn = var.role_arn
// }
// }
//
// traverseBlock(block, []string{"region"})
// => returns (<body of the current block>, "region")
//
// traverseBlock(block, []string{"assume_role", "role_arn"})
// => returns (<body of the nested assume_role block>, "role_arn")
//
// traverseBlock(block, []string{"foo"})
// => returns (nil, "")
//
// traverseBlock(block, []string{"assume_role", "foo"})
// => returns (nil, "")
func traverseBlock(block *hclwrite.Block, keyParts []string) (*hclwrite.Body, string) {
if block == nil {
return nil, ""
}

if len(keyParts) < 2 {
return block.Body(), strings.Join(keyParts, "")
}

blockName := keyParts[0]
return traverseBlock(block.Body().FirstMatchingBlock(blockName, nil), keyParts[1:])
}

// Custom error types

type MissingOverrides string
Expand Down
29 changes: 24 additions & 5 deletions cli/aws_provider_patch_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -173,9 +173,8 @@ provider "google" {
}

provider "aws" {
alias = "yet another"
region = "eu-west-1"
version = "0.3.0"
alias = "yet another"
region = "eu-west-1"
}

output "hello" {
Expand Down Expand Up @@ -273,6 +272,24 @@ provider "aws" {
}
`

const terraformCodeExampleAwsOneProviderNestedBlocks = `
provider "aws" {
region = var.aws_region
assume_role {
role_arn = var.role_arn
}
}
`

const terraformCodeExampleAwsOneProviderNestedBlocksRegionRoleArnExpected = `
provider "aws" {
region = "eu-west-1"
assume_role {
role_arn = "nested-override"
}
}
`

func TestPatchAwsProviderInTerraformCodeHappyPath(t *testing.T) {
t.Parallel()

Expand All @@ -288,8 +305,8 @@ func TestPatchAwsProviderInTerraformCodeHappyPath(t *testing.T) {
{"no provider", terraformCodeExampleOutputOnly, map[string]string{"region": "eu-west-1"}, false, []string{terraformCodeExampleOutputOnly}},
{"no aws provider", terraformCodeExampleGcpProvider, map[string]string{"region": "eu-west-1"}, false, []string{terraformCodeExampleGcpProvider}},
{"one empty aws provider, but no overrides", terraformCodeExampleAwsProviderEmptyOriginal, nil, false, []string{terraformCodeExampleAwsProviderEmptyOriginal}},
{"one empty aws provider, with region override", terraformCodeExampleAwsProviderEmptyOriginal, map[string]string{"region": "eu-west-1"}, true, []string{terraformCodeExampleAwsProviderRegionOverridenExpected}},
{"one empty aws provider, with region, version override", terraformCodeExampleAwsProviderEmptyOriginal, map[string]string{"region": "eu-west-1", "version": "0.3.0"}, true, []string{terraformCodeExampleAwsProviderRegionVersionOverridenExpected, terraformCodeExampleAwsProviderRegionVersionOverridenReverseOrderExpected}},
{"one empty aws provider, with region override", terraformCodeExampleAwsProviderEmptyOriginal, map[string]string{"region": "eu-west-1"}, false, []string{terraformCodeExampleAwsProviderEmptyOriginal}},
{"one empty aws provider, with region, version override", terraformCodeExampleAwsProviderEmptyOriginal, map[string]string{"region": "eu-west-1", "version": "0.3.0"}, false, []string{terraformCodeExampleAwsProviderEmptyOriginal}},
{"one non-empty aws provider, but no overrides", terraformCodeExampleAwsProviderNonEmptyOriginal, nil, false, []string{terraformCodeExampleAwsProviderNonEmptyOriginal}},
{"one non-empty aws provider, with region override", terraformCodeExampleAwsProviderNonEmptyOriginal, map[string]string{"region": "eu-west-1"}, true, []string{terraformCodeExampleAwsProviderRegionOverridenVersionNotOverriddenExpected}},
{"one non-empty aws provider, with region, version override", terraformCodeExampleAwsProviderNonEmptyOriginal, map[string]string{"region": "eu-west-1", "version": "0.3.0"}, true, []string{terraformCodeExampleAwsProviderRegionVersionOverridenExpected, terraformCodeExampleAwsProviderRegionVersionOverridenReverseOrderExpected}},
Expand All @@ -299,6 +316,8 @@ func TestPatchAwsProviderInTerraformCodeHappyPath(t *testing.T) {
{"multiple providers with comments, but no overrides", terraformCodeExampleAwsMultipleProvidersNonEmptyWithCommentsOriginal, nil, false, []string{terraformCodeExampleAwsMultipleProvidersNonEmptyWithCommentsOriginal}},
{"multiple providers with comments, with region override", terraformCodeExampleAwsMultipleProvidersNonEmptyWithCommentsOriginal, map[string]string{"region": "eu-west-1"}, true, []string{terraformCodeExampleAwsMultipleProvidersNonEmptyWithCommentsRegionOverriddenExpected}},
{"multiple providers with comments, with region, version override", terraformCodeExampleAwsMultipleProvidersNonEmptyWithCommentsOriginal, map[string]string{"region": "eu-west-1", "version": "0.3.0"}, true, []string{terraformCodeExampleAwsMultipleProvidersNonEmptyWithCommentsRegionVersionOverriddenExpected}},
{"one provider with nested blocks, with region and role_arn override", terraformCodeExampleAwsOneProviderNestedBlocks, map[string]string{"region": "eu-west-1", "assume_role.role_arn": "nested-override"}, true, []string{terraformCodeExampleAwsOneProviderNestedBlocksRegionRoleArnExpected}},
{"one provider with nested blocks, with region and role_arn override, plus non-matching overrides", terraformCodeExampleAwsOneProviderNestedBlocks, map[string]string{"region": "eu-west-1", "assume_role.role_arn": "nested-override", "should-be": "ignored", "assume_role.should-be": "ignored"}, true, []string{terraformCodeExampleAwsOneProviderNestedBlocksRegionRoleArnExpected}},
}

for _, testCase := range testCases {
Expand Down
57 changes: 44 additions & 13 deletions docs/_docs/04_reference/cli-options.md
Original file line number Diff line number Diff line change
Expand Up @@ -232,27 +232,56 @@ This will recursively search the current working directory for any folders that

### aws-provider-patch

Overwrite settings on nested AWS providers to work around a Terraform bug. Due to
[issue #13018](https://github.com/hashicorp/terraform/issues/13018), the `import` command may fail if your Terraform
Overwrite settings on nested AWS providers to work around several Terraform bugs. Due to
[issue #13018](https://github.com/hashicorp/terraform/issues/13018) and
[issue #26211](https://github.com/hashicorp/terraform/issues/26211), the `import` command may fail if your Terraform
code uses a module that has a `provider` block nested within it that sets any of its attributes to computed values.
This command is a hacky attempt at working around this problem by allowing you to temporarily hard-code those
attributes, based on the [terragrunt-override-attr](#terragrunt-override-attr) options you set, so that `import`
can work.
attributes so `import` can work.

You specify which attributes to hard-code using the [`--terragrunt-override-attr`](#terragrunt-override-attr) option,
passing it `ATTR=VALUE`, where `ATTR` is the attribute name and `VALUE` is the new value. Note that `ATTR` can specify
attributes within a nested block by specifying `<BLOCK>.<ATTR>`, where `<BLOCK>` is the block name. For example, let's
say you had a `provider` block in a module that looked like this:

```hcl
provider "aws" {
region = var.aws_region
assume_role {
role_arn = var.role_arn
}
}
```

Example:
Both the `region` and `role_arn` parameters are set to dynamic values, which will trigger those Terraform bugs. To work
around it, run the following command:

```bash
terragrunt aws-provider-patch --terragrunt-override-attr region=eu-west-1
terragrunt aws-provider-patch \
--terragrunt-override-attr region=eu-west-1 \
--terragrunt-override-attr assume_role.role_arn=""
```

When you run the command above, Terragrunt will:

1. Run `terraform init` to download the code for all your modules into `.terraform/modules`.
1. Scan all the Terraform code in `.terraform/modules`, find AWS `provider` blocks, and hard-code the `region` param
to `eu-west-1` for each one.

This should allow you to run `import` on the module and work around issue #13018.
1. Scan all the Terraform code in `.terraform/modules`, find AWS `provider` blocks, and for each one, hard-code:
1. The `region` param to `"eu-west-1"`.
1. The `role_arn` within the `assume_role` block to `""`.

The result will look like this:

```hcl
provider "aws" {
region = "eu-west-1"
assume_role {
role_arn = ""
}
}
```

This should allow you to run `import` on the module and work around those Terraform bugs. When you're done running
`import`, remember to delete your overridden code! E.g., Delete the `.terraform` or `.terragrunt-cache` folders.



Expand Down Expand Up @@ -498,7 +527,9 @@ When passed in, run `hclfmt` only on specified `*/terragrunt.hcl` file.
### terragrunt-override-attr

**CLI Arg**: `--terragrunt-override-attr`
**Requires an argument**: `--terragrunt-override-attr KEY=VALUE`
**Requires an argument**: `--terragrunt-override-attr ATTR=VALUE`

A `KEY=VALUE` attribute to override in a `provider` block as part of the [aws-provider-patch
command](#aws-provider-patch). May be specified multiple times.
Override the attribute named `ATTR` with the value `VALUE` in a `provider` block as part of the [aws-provider-patch
command](#aws-provider-patch). May be specified multiple times. Also, `ATTR` can specify attributes within a nested
block by specifying `<BLOCK>.<ATTR>`, where `<BLOCK>` is the block name: e.g., `assume_role.role` arn will override the
`role_arn` attribute of the `assume_role { ... }` block.