Panic when using complex types returned from modules

[mbrancato]

Terraform Version

$ terraform -v
Terraform v0.12.20
+ provider.vault v2.8.0

Terraform Configuration Files

I attempted to generate a minimal example, but was unable to reproduce. The code causing the problem looks like this.

edit: I've updated the examples to better reflect what the setup looked like.

module "app_test_one" {
  source = "./module-one/"
  app_name = "test_one"
}

module "app_test_two" {
  source = "./module-one/"
  app_name = "test_two"
}

module "policy_test" {
  source = "./module-two/"
  read_access = [module.app_test_one, module.app_test_two]
}

output "test" {
  value = module.policy_test
}

with module-one being:

variable "app_name" {
  type = string
}

locals {
  kv_v2_read_policy   = <<EOF
path "secret/data/${var.app_name}/*" {
  capabilities = ["read"]
}
EOF
  kv_v2_update_policy = <<EOF
path "secret/data/${var.app_name}/*" {
  capabilities = ["list","create","update","delete"]
}
EOF
}

resource "vault_policy" "read" {
  count = 1
  name  = "${var.app_name}_read_policy"
  policy = local.kv_v2_read_policy
}

resource "vault_policy" "update" {
  count = 1
  name  = "${var.app_name}_update_policy"
  policy = local.kv_v2_update_policy
}


output "policies" {
  value = [vault_policy.update.0.name,vault_policy.read.0.name]
}

and module-two being:

variable "read_access" {
  type = list(map(any))
  default = []
}

variable "update_access" {
  type = list(map(any))
  default = []
}

locals {
  read_policies = distinct(flatten([
    for app in var.read_access : [
      for key, val in app : [
        for grp in val : grp if substr(grp, -12, 12) == "_read_policy"
      ] if key == "policies"
    ]
  ]))
  update_policies = distinct(flatten([
    for app in var.update_access : [
      for key, val in app : [
        for grp in val : grp if substr(grp, -14, 14) == "_update_policy"
      ] if key == "policies"
    ]
  ]))
}

output "policies" {
  value = flatten([local.read_policies,local.update_policies])
}

Debug Output

Crash Output

I can't share the full crash.log since it contains internal code - if I hit this again, I will see if I can get a minimal example working:

console panic

Expected Behavior

Actual Behavior

Steps to Reproduce

1. terraform apply

Additional Context

I solved this by changing the type of the variable. Originally this was:

variable "read_access" {
  type = list(map(any))
  default = []
}

but I solved this by changing to:

variable "read_access" {
  type = list(map(list(string)))
  default = []
}

References

[mbrancato]

I've been able to generate a minimal example, and can share a crash log as well. I think this is the same bug - although the type is slightly different, it is flexible.

I can solve this crash by removing the variable's type statement entirely, it is not a fixed type as with the previous example. Also, I can make it map(any) as a workaround.

Link to crash log: crash.log

main code:

module "test" {
  source = "./module/"
  components = {
    kv = {}
    pki = {
      domains = ["fakeapp.domain.com"]
    }
  }
}

output "policies" {
  value = module.test
}

module code:

variable "components" {
  type    = map(map(any))
  default = { kv = {} }
}

locals {
  kv_enabled   = length([for i, j in var.components : 1 if i == "kv"]) > 0 ? true : false
  pki_enabled  = length([for i, j in var.components : 1 if i == "pki"]) > 0 ? true : false
  pki_settings = lookup(var.components, "pki", {})
}

data "vault_policy_document" "pki_read_policy" {
  count = local.pki_enabled ? 1 : 0

  dynamic "rule" {
    for_each = length(lookup(local.pki_settings, "domains", [])) > 0 ? toset(["enabled"]) : toset([])

    content {
      path         = "pki/certs/*"
      capabilities = ["list"]
      description  = "List certs"
    }
  }
}

output "policy" {
  value = data.vault_policy_document.pki_read_policy.0.hcl
}

[crabby345]

Also experienced this when trying to map vars into a complex variable type. Configuration too complex and private to share as another crash log. But +1 for the issue.

I think in my case it was map(objects) that contained map(objects). azs[A] contained 1 object in subnets. azs[B] contained 2 objects in subnets. I realise this isn't supported and worked out my error, but assume it should record an error not panic.

Originally - static

azs = map(object({
    subnets = map(object({
         name_as_prefix = bool  
         routing = string  
         cidr_block = string 
    }))  
}))

Attempting dynamic content in an instance of subnets

azs = map(object({ 
    subnets = any
/*     map(object({
            name_as_prefix = bool
            routing = string
            cidr_block = string
    })) */
}))

Working dynamic content

azs = map(object({   
    subnets = map(any  
/*     object({  
            name_as_prefix = bool  
            routing = string  
            cidr_block = string  
        }) */  
    )  
}))

panic: inconsistent map element types (cty.Object(map[string]cty.Type{"subnets":cty.Object(map[string]cty.Type{"betasubnet":cty.Object(map[string]cty.Type{"cidr_block":cty.String, "name_as_prefix":cty.Bool, "routing":cty.String})})}) then cty.Object(map[string]cty.Type{"subnets":cty.Object(map[string]cty.Type{"Alphasubnet":cty.Object(map[string]cty.Type{"cidr_block":cty.String, "name_as_prefix":cty.Bool, "routing":cty.String}), "ogle":cty.Object(map[string]cty.Type{"cidr_block":cty.String, "name_as_prefix":cty.Bool, "routing":cty.String})})}))

[smurfralf]

Using terraform v0.12.21 I got a similar crash. The variable inside a module was defined as
map(map(any))
with intent that inner map could provide any type of value. But when the module invocation provides a value where the inner map consists of string values, I saw the panic below. Workaround was to change module variable to map(map(string)) and forego the ability to support diverse types.

panic: inconsistent map element types (cty.Map(cty.String) then cty.Map(cty.DynamicPseudoType))

goroutine 258 [running]:
github.com/zclconf/go-cty/cty.MapVal(0xc000750d30, 0xc000750d00, 0xc000e7a2f0, 0xd, 0xc000750e88)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/value_init.go:207 +0x538
github.com/zclconf/go-cty/cty/convert.conversionObjectToMap.func2(0x22e4660, 0xc00000eec8, 0x1adf860, 0xc0001ff710, 0xc000ba6e00, 0x3, 0x4, 0xc000750f01, 0xc000750f78, 0x4136f5, ...)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/conversion_collection.go:338 +0x555
github.com/zclconf/go-cty/cty/convert.getConversion.func1(0x22e4660, 0xc00000eec8, 0x1adf860, 0xc0001ff710, 0xc0003cbfa0, 0x2, 0x2, 0x1adf860, 0xc0001ff710, 0x0, ...)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/conversion.go:46 +0x1ad
github.com/zclconf/go-cty/cty/convert.conversionObjectToObject.func1(0x22e4660, 0xc00000eed0, 0x1adf860, 0xc0001ff9b0, 0xc0003cbfa0, 0x2, 0x2, 0x7fc2fb9cd401, 0x40391a, 0xc00003e3f0, ...)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/conversion_object.go:65 +0x450
github.com/zclconf/go-cty/cty/convert.getConversion.func1(0x22e4660, 0xc00000eed0, 0x1adf860, 0xc0001ff9b0, 0xc000f09ad0, 0x1, 0x1, 0x1adf860, 0xc0001ff9b0, 0x4, ...)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/conversion.go:46 +0x1ad
github.com/zclconf/go-cty/cty/convert.conversionObjectToMap.func2(0x22e4660, 0xc00000eed8, 0x1adf860, 0xc0001ffc80, 0xc000f09ad0, 0x1, 0x1, 0xc000062380, 0x10, 0x10, ...)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/conversion_collection.go:330 +0x4a7
github.com/zclconf/go-cty/cty/convert.getConversion.func1(0x22e4660, 0xc00000eed8, 0x1adf860, 0xc0001ffc80, 0x0, 0x0, 0x0, 0xc000d07710, 0xc000f09ac0, 0x22e4620, ...)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/conversion.go:46 +0x1ad
github.com/zclconf/go-cty/cty/convert.retConversion.func1(0x22e4660, 0xc00000eed8, 0x1adf860, 0xc0001ffc80, 0xc000f09ac0, 0x0, 0x0, 0x0, 0x0, 0x0)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/conversion.go:179 +0x6b
github.com/zclconf/go-cty/cty/convert.Convert(0x22e4660, 0xc00000eed8, 0x1adf860, 0xc0001ffc80, 0x22e4620, 0xc000456990, 0x0, 0x22e4660, 0xc00000eed8, 0x1adf860, ...)
/opt/teamcity-agent/work/9e329aa031982669/pkg/mod/github.com/zclconf/[email protected]/cty/convert/public.go:51 +0x1af
github.com/hashicorp/terraform/terraform.(*EvalModuleCallArgument).Eval(0xc000ba6780, 0x2317d80, 0xc000968270, 0x2, 0x2, 0xc000a08000, 0x42)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval_variable.go:77 +0x160
github.com/hashicorp/terraform/terraform.EvalRaw(0x22a2180, 0xc000ba6780, 0x2317d80, 0xc000968270, 0x2d, 0x0, 0x0, 0x2d)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval.go:57 +0x131
github.com/hashicorp/terraform/terraform.(*EvalOpFilter).Eval(0xc0011bb590, 0x2317d80, 0xc000968270, 0x2, 0x2, 0xc0009beb38, 0x42b8a1)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval_filter_operation.go:37 +0x4c
github.com/hashicorp/terraform/terraform.EvalRaw(0x22a21c0, 0xc0011bb590, 0x2317d80, 0xc000968270, 0x14, 0xbf9e81f775137128, 0x1e393a7e5, 0x2d)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval.go:57 +0x131
github.com/hashicorp/terraform/terraform.(*EvalSequence).Eval(0xc00066d380, 0x2317d80, 0xc000968270, 0x2, 0x2, 0xecc5d5, 0x22a2860)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval_sequence.go:20 +0xfd
github.com/hashicorp/terraform/terraform.EvalRaw(0x22a2320, 0xc00066d380, 0x2317d80, 0xc000968270, 0x1ad02c0, 0x32f6465, 0x1a39aa0, 0xc000f08bb0)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval.go:57 +0x131
github.com/hashicorp/terraform/terraform.Eval(0x22a2320, 0xc00066d380, 0x2317d80, 0xc000968270, 0xc00066d380, 0x22a2320, 0xc00066d380, 0x0)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/eval.go:35 +0x4d
github.com/hashicorp/terraform/terraform.(*Graph).walk.func1(0x1c86280, 0xc0007ad6c0, 0x0, 0x0, 0x0)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/terraform/graph.go:90 +0xf40
github.com/hashicorp/terraform/dag.(*Walker).walkVertex(0xc0005dd300, 0x1c86280, 0xc0007ad6c0, 0xc000600600)
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/dag/walk.go:392 +0x353
created by github.com/hashicorp/terraform/dag.(*Walker).Update
/opt/teamcity-agent/work/9e329aa031982669/src/github.com/hashicorp/terraform/dag/walk.go:314 +0xa9b

[danieldreier]

I've reproduced this using @mbrancato's example on Terraform 0.12.25. For ease of reproduction, I've put a reproduction case in github that can be cloned and run.

[alisdair]

This bug has been fixed in the 0.13.0-beta1 prerelease, via this go-cty change.

The reproduction case still has a type error, which is caused by declaring the type of the module's components variable as map(map(any)), then passing in an object with heterogeneous object types as its values. A map(map(any)) still requires the second-level map's value types to be compatible, and object({}) is a different type from object({ domains = list(string) }).

For this use case, it might make more sense to have separate variables for kv and pki, and declare an object type for each. This would also allow you to lean on Terraform to validate the schema of your variables matches what the module expects.

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.