validate_schema=false not working as expected

[alexkuklin]

While trying to wrap kafka operatior installation into terraform, I've got

CustomResourceDefinition.apiextensions.k8s.io "kafkaclusters.kafka.banzaicloud.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes

tf code is

data "kubectl_path_documents" "crd" {
    pattern = "./manifests/*.yaml"
}

resource "kubectl_manifest" "kafka-operator-crds" {
    count     = length(data.kubectl_path_documents.crd.documents)
    yaml_body = element(data.kubectl_path_documents.crd.documents, count.index)

  validate_schema  = false
  wait_for_rollout = true
}

kubectl gives same error if used without --validate=false. With --validate=false it works fine.

[alekc]

Have you tried with server apply?

…

On Fri, 26 Jan 2024, 07:45 Alex Kuklin, ***@***.***> wrote: While trying to wrap kafka operatior installation into terraform, I've got CustomResourceDefinition.apiextensions.k8s.io "kafkaclusters.kafka.banzaicloud.io" is invalid: metadata.annotations: Too long: must have at most 262144 bytes tf code is data "kubectl_path_documents" "crd" { pattern = "./manifests/*.yaml" } resource "kubectl_manifest" "kafka-operator-crds" { count = length(data.kubectl_path_documents.crd.documents) yaml_body = element(data.kubectl_path_documents.crd.documents, count.index) validate_schema = false wait_for_rollout = true } kubectl gives same error if used without --validate=false. With --validate=false it works fine. — Reply to this email directly, view it on GitHub <#282>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACJ5V6TLBAUMS246XNFJUTYQNNINAVCNFSM6AAAAABCLXTNDOVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEYDCNZUGEYDIMY> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[alexkuklin]

It helped, thank you.

Not very clear though.

[alekc]

It's the same issue as with Prometheus. Basically, when you run a normal apply, it will try to construct the whole yaml document and add an annotation "last_applied_config" (or something similar). In case of CRD, it can be an issue because it will exceed the maximum length.

If you are doing server side apply, it will only try to apply the fields within your yaml document ignoring everything else (and the diff will be calculated on the server side).

Something along those lines.

P.s. and just FIY since you have opened an issue on a pretty much dead project, once you switch to the kubernetes 1.27 and higher you are going to be hit with #270 I would strongly suggest to switch to my maintained fork ;) https://github.com/alekc/terraform-provider-kubectl/

terraform {
  required_providers {
    kubectl = {
      source = "alekc/kubectl"
      version = "2.0.4"
    }
  }
}

[alexkuklin]

Oops.
I followed terraform registry link.
Thank you for heads up.