Terraform recreating all resources in a module due to data block issue

[mstavreski]

Hi all,

We are experiencing an issue with the Confluent provider where terraform tries to recreate all the resources if there is a dependency on a data block.

Here is an example:

The following code creates a service account and identity pool, and assigned them some schema-registry level permissions.

But due to the data blocks inside the rbac-schema-registry.tf file if I try to add another service account or identity pool in my .tfvars file, terraform will try to redeploy all the rbac permissions as it is re-running the data block and doesn't know what the output of it is at the plan step, as seen below.

Is it possible to resolve this issue so that we can run data blocks inside modules without having to redeploy everything in that module?

main.tf:

# Deploy Identity Provider

module "identity_provider" {
  source = "../modules/identity-provider"

  tenant_id    = var.identity_provider.tenant_id
  display_name = var.identity_provider.display_name
  jwks_uri     = var.identity_provider.jwks_uri
}

# Deploy Identity Pool

module "identity_pool" {
  depends_on = [module.identity_provider]
  source     = "../modules/identity-pool"
  for_each   = { for pool in var.identity_pool : pool.display_name => pool }
  identity_pool = {
    display_name   = each.value.display_name
    description    = each.value.description
    identity_claim = each.value.identity_claim
    filter         = each.value.filter
    provider_name  = each.value.provider_name
  }
}

# Deploy Service Account

module "service_account" {
  source   = "../modules/service-account"
  for_each = { for account in var.service_account : account.display_name => account }
  service_account = {
    display_name = each.value.display_name
    description  = each.value.description
  }
}

# Deploy Role Bindings

module "rbac_schema_registry" {
  depends_on = [module.identity_pool, module.service_account]
  source     = "../modules/rbac-schema-registry"
  for_each   = { for mapping in var.rbac_schema_registry : mapping.principal_display_name => mapping }
  principal = {
    display_name = each.value.principal_display_name
    type         = each.value.type
  }
  environment = {
    display_name = each.value.environment_display_name
  }
  roles                          = each.value.roles
  identity_provider_display_name = var.identity_provider.display_name
}

rbac-schema-registry.tf file:

terraform {
  required_providers {
    confluent = {
      source  = "confluentinc/confluent"
      version = ">=2.2.0"
    }
  }
#  required_version = ">= 1.9.5"
}

data "confluent_environment" "environment" {
  display_name = var.environment.display_name
}

data "confluent_identity_provider" "identity_provider" {
  count        = var.principal.type == "identity_pool" ? 1 : 0
  display_name = var.identity_provider_display_name
}

data "confluent_service_account" "service_account" {
  count        = var.principal.type == "service_account" ? 1 : 0
  display_name = var.principal.display_name
}

data "confluent_identity_pool" "identity_pool" {
  count        = var.principal.type == "identity_pool" ? 1 : 0
  display_name = var.principal.display_name
  identity_provider {
    id = data.confluent_identity_provider.identity_provider[0].id
  }
}

data "confluent_schema_registry_cluster" "schema_registry" {
  environment {
    id = data.confluent_environment.environment.id
  }
}

resource "confluent_role_binding" "role_binding" {
  for_each    = { for role in var.roles : role.role_name => role }
  principal   = "User:${var.principal.type == "service_account" ? data.confluent_service_account.service_account[0].id : data.confluent_identity_pool.identity_pool[0].id}"
  role_name   = each.value.role_name
  crn_pattern = "${data.confluent_schema_registry_cluster.schema_registry.resource_name}/${each.value.crn_pattern}"
}

Plan: 4 to add, 0 to change, 3 to destroy.

[linouk23]

@mstavreski thanks for creating this issue and sharing your TF config files with us! Could you share a minimal reproducible example?

[mstavreski]

@mstavreski thanks for creating this issue and sharing your TF config files with us! Could you share a minimal reproducible example?

Sure, you can replicate it using the code in the following zip file.
confluent-data-issue.zip

[mstavreski]

Hello @linouk23, just following up on this issue. Is there a resolution to it?