Unable to remove certain blocks from Terraform config

[dak1n1]

Description

When specifying a Terraform config, we can add certain blocks to the resource, such as adding this block to the cluster_role resource:

  aggregation_rule {
    cluster_role_selectors {
      match_labels = {
        "rbac.example.com/aggregate-to-monitoring" = "true"
      }
    }
  }

However, once we add the block, it cannot be removed in-place using Terraform. The user will either have to delete the block using kubectl or destroy and re-create the resource. If they attempt to apply an empty block (blockname{}), or remove the block entirely from their config, Terraform may either crash or return a Kubernetes API error.

Terraform Version and Provider Version

Terraform v0.12.28

• provider.kubernetes v1.12.0

Affected Resource(s)

• kubernetes_cluster_role
• kubernetes_deployment
• kubernetes_pod
• (probably others)

Terraform Configuration Files

resource "kubernetes_cluster_role" "testaggregated" {
  metadata {
    name = "testaggregated"
  }
#  aggregation_rule {
#    cluster_role_selectors {
#      match_labels = {
#        "rbac.example.com/aggregate-to-monitoring" = "true"
#      }
#
#      match_expressions {
#        key      = "environment"
#        operator = "In"
#        values   = ["non-exists-12345"]
#      }
#    }
#  }
}

Debug Output

https://gist.github.com/dak1n1/f7dc3231af106c432876a4139231da75

Expected Behavior

When a block is removed from the Terraform config, it should be removed from the remote resource.

Actual Behavior

Users have to manually delete objects (or parts of objects) when removing a config block. Otherwise, a Kubernetes API error (like the one below) is returned. Or, depending on the resource, Terraform may crash.

Error: Failed to update ClusterRole: ClusterRole.rbac.authorization.k8s.io "testaggregated" is invalid: aggregationRule.clusterRoleSelectors: Required value: at least one clusterRoleSelector required if aggregationRule is non-nil

Steps to Reproduce

1. Apply the above config.
2. Uncomment the aggregated_rules block and apply again. The update should be successful.
3. Comment out the aggregated_rules block. Updates will now return Kubernetes API errors.

Important Factoids

This is somewhat of a generic problem affecting multiple resources.

References

Config blocks (such as env_from{}) will crash Terraform when empty, unless we check for nil. Example check here.

• Pod spec with an empty env_from block causes crash #850
• Cannot remove volume_mount #870

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[dak1n1]

Another instance of this issue came up today with the limits{} block inside the Deployment resource. #1039

[dak1n1]

Related: unable to remove limits{} block. #754

[dak1n1]

I think we should add some required properties to some of these blocks so that users will get an error message that describes what they can do to fix the problem. (This will be in addition to checking for nil to avoid crashes). Currently some of our config blocks support this, while others do not, as noted in this screenshot.

My IDE highlights the blocks that are missing required properties, which is a useful guide for someone writing a Terraform config.

[dak1n1]

The empty PodSpec blocks now report an error instead of crashing:

        Error: Missing required argument
        The argument "name" is required, but no definition was found.

There is still the issue of being unable to remove blocks from a config once an object has been created. We would have to modify each resource's Update function to be able to handle removalsof attributes during in-place updates.

Offhand I'm not sure if any crashes remain.

[dak1n1]

If we had the ability to remove attributes on Update, we could also fix this issue #585

[github-actions]

Marking this issue as stale due to inactivity. If this issue receives no comments in the next 30 days it will automatically be closed. If this issue was automatically closed and you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. This helps our maintainers find and focus on the active issues. Maintainers may also remove the stale label at their discretion. Thank you!

[github-actions]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.
If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.