kubernetes.io annotations are not available as attributes on GCP

[haard]

Terraform Version and Provider Version

Terraform v0.12.25

• provider.google v3.22.0
• provider.google-beta v3.22.0
• provider.kubernetes v1.11.3

Affected Resource(s)

• kubernetes_ingress
• other kubernetes_ resources

Terraform Configuration Files

resource "kubernetes_ingress" "app" {
  provider = kubernetes.gke_app
  metadata {
    name = "app-ingress"
  }
  spec {
    backend {
      service_name = "app"
      service_port = 8080
    }
  }
}

Debug Output

Panic Output

Expected Behavior

What should have happened?
terraform state show kubernetes_ingress.app should show kubernetes.io/GCP annotations.

Actual Behavior

Output of terraform state show data.kubernetes_ingress.app

# data.kubernetes_ingress.app:
data "kubernetes_ingress" "app" {
    id                    = "default/app-ingress"
    load_balancer_ingress = [
        {
            hostname = ""
            ip       = "NN.NN.NN.NN"
        },
    ]
    spec                  = [
        {
            backend = [
                {
                    service_name = "app"
                    service_port = "8080"
                },
            ]
            rule    = []
            tls     = []
        },
    ]

    metadata {
        annotations      = {}
        generation       = 1
        labels           = {}
        name             = "app-ingress"
        namespace        = "default"
        resource_version = "90910"
        self_link        = "/apis/extensions/v1beta1/namespaces/default/ingresses/app-ingress"
        uid              = "aad8e98b-a654-11ea-b841-42010aa200d5"
    }
}

output of kubectl get ingress app-ingress -o yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    ingress.kubernetes.io/backends: '{"k8s-be-nnnnn--a2d6fsdfs4cf724d":"HEALTHY"}'
    ingress.kubernetes.io/forwarding-rule: k8s-fw-default-app-ingress--a2d6fsdfs4cf724d
    ingress.kubernetes.io/target-proxy: k8s-tp-default-app-ingress--a2d6fsdfs4cf724d
    ingress.kubernetes.io/url-map: k8s-um-default-app-ingress--a2d6fsdfs4cf724d
  creationTimestamp: "2020-06-04T11:15:16Z"
  generation: 1
  name: app-ingress
  namespace: default
  resourceVersion: "90901"
  selfLink: /apis/extensions/v1beta1/namespaces/default/ingresses/app-ingress
  uid: aad8e98b-a654-11ea-b841-42010aa200d5
spec:
  backend:
    serviceName: app
    servicePort: 8080
status:
  loadBalancer:
    ingress:
    - ip: nn.nn.nn.nn

Steps to Reproduce

Create a kubernetes_ingress on GCP (using the guide from the Terraform docs to layer the providers).
Show state of the ingress.

Important Factoids

References

May be related to #60

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

[stellirin]

Also facing this:

Terraform v0.12.28
+ provider.google v3.30.0

Specifically we're setting the "kubernetes.io/ingress.class" = "gce-multi-cluster" annotation on a kubernetes_ingress resource, which will trigger GCP to add the ingress.gcp.kubernetes.io/instance-groups annotation with data about the instance groups.

We then want to read back this annotation using a kubernetes_ingress data resource to configure a multi-cluster load balancer.

Our current work-around is to manually copy the JSON from the annotation into a local Terraform variable then continue processing from there (fortunately this annotation does not regularly change).

[aareet]

Relates to #722

[pdecat]

Hi, when using a kubernetes_ingress datasource to retrieve annotations, the work-around that I shared on #709 (comment) should also apply here, e.g.:

data "kubernetes_ingress" "myapp" {
  metadata {
    namespace = "mynamespace"
    name      = "myapp"
    annotations = {
      # Internal annotations are removed if not defined in configuration
      "ingress.gcp.kubernetes.io/instance-groups" = ""
    }
  }
}

[dak1n1]

We're looking into adding a provider-level annotation ignore/allow list, which should fix the issue described here. Let's track this over here: #746

[ghost]

I'm going to lock this issue because it has been closed for 30 days ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. If you feel I made an error 🤖 🙉 , please reach out to my human friends 👉 [email protected]. Thanks!