Infra IR was updated, but no listeners were found

[SerheoReutty]

Greetings. Help me please. What am I doing wrong.
I deployed envoy-gateway.
Deployed custom-envoy-proxy-config, which says to create services with NodePort type.
I deployed gateway-class
And deployed gateway using tlc terminating, specifying my cluster-issuer.
But in namespace envoy I have no service or feed for my deployment.
In the controller logs I see the following.

2024-05-27T09:30:22.088Z	INFO	xds-translator	runner/runner.go:55	received an update	{"runner": "xds-translator"}
2024-05-27T09:30:22.088Z	INFO	infrastructure	runner/runner.go:78	received an update	{"runner": "infrastructure"}
2024-05-27T09:30:22.088Z	INFO	xds-server	runner/runner.go:140	received an update	{"runner": "xds-server"}
2024-05-27T09:30:22.088Z	INFO	infrastructure	runner/runner.go:89	Infra IR was updated, but no listeners were found. Skipping infra creation.	{"runner": "infrastructure"}
2024-05-27T09:30:22.088Z	INFO	provider	kubernetes/status_updater.go:140	received a status update	{"runner": "provider", "namespace": "test-envoy", "name": "test-gateway"}

As far as I understand, it's complaining that it didn't find the necessary listeners that I specified in my gateway. But I don't understand how this should work. I think in my gateway I specify which port and host is being listened to. I must be wrong!?

Attached below is the manifest text of my custom config, gateway-class and gateway manifests

Custom-config

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: EnvoyProxy
metadata:
  name: custom-proxy-config
  namespace: envoy-gateway-system
spec:
  provider:
    type: Kubernetes
    kubernetes:
      envoyService:
        type: NodePort

gateway-class

apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
  name: envoy-gateway-class
spec:
  controllerName: gateway.envoyproxy.io/gatewayclass-controller
  parametersRef:
    group: gateway.envoyproxy.io
    kind: EnvoyProxy
    name: custom-proxy-config
    namespace: envoy-gateway-system

gateway

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: test-gateway
  annotations:
    cert-manager.io/cluster-issuer: "my-cert-manager"
spec:
  gatewayClassName: envoy-gateway-class
  listeners:
  - name: example-https
    protocol: HTTPS
    hostname: www.example.com
    port: 443
    tls:
      mode: Terminate
      certificateRefs:
      - kind: Secret
        name: test-gateway-https

[SerheoReutty]

I tried adding in gateway listeners on port 80. And the infrastructure is created for it. The required pod and service on NodePort appears, but the forwarding only happens on port 80. My example-http listener seems to be ignored.

apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
  name: test-gateway
  annotations:
    cert-manager.io/cluster-issuer: "my-cert-manager"
spec:
  gatewayClassName: envoy-gateway-class
  listeners:
    - name: http
      protocol: HTTP
      port: 80
    - name: example-https
      protocol: HTTPS
      hostname: www.example.com
      port: 443
      tls:
        mode: Terminate
        certificateRefs:
        - kind: Secret
          name: test-gateway-https

[arkodg]

can you share the o/p of kubectl get gateway <> -n <> -o yaml and kubectl get httroute <> -n <> -o yaml

[SerheoReutty]

kubectl get gateway test-gateway -n test-envoy -o yaml

kind: Gateway
metadata:
  annotations:
    cert-manager.io/cluster-issuer: letsencrypt-ours-dns
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"gateway.networking.k8s.io/v1beta1","kind":"Gateway","metadata":{"annotations":{"cert-manager.io/cluster-issuer":"letsencrypt-ours-dns"},"name":"test-gateway","namespace":"test-envoy"},"spec":{"gatewayClassName":"envoy-gateway-class","listeners":[{"name":"http","port":80,"protocol":"HTTP"},{"hostname":"example.com","name":"https","port":443,"protocol":"HTTPS","tls":{"certificateRefs":[{"kind":"Secret","name":"example-com"}],"mode":"Terminate"}}]}}
  creationTimestamp: "2024-05-29T08:28:00Z"
  generation: 1
  name: test-gateway
  namespace: test-envoy
  resourceVersion: "64896066"
  uid: 9cae5052-cb06-4ee1-8dbe-db48947265da
spec:
  gatewayClassName: envoy-gateway-class
  listeners:
  - allowedRoutes:
      namespaces:
        from: Same
    name: http
    port: 80
    protocol: HTTP
  - allowedRoutes:
      namespaces:
        from: Same
    hostname: example.com
    name: https
    port: 443
    protocol: HTTPS
    tls:
      certificateRefs:
      - group: ""
        kind: Secret
        name: example-com
      mode: Terminate
status:
  addresses:
  - type: IPAddress
    value: ip_node
  - type: IPAddress
    value: ip_node
  - type: IPAddress
    value: ip_node
  - type: IPAddress
    value: ip_node
  - type: IPAddress
    value: ip_node
  - type: IPAddress
    value: ip_node
  conditions:
  - lastTransitionTime: "2024-05-29T08:28:00Z"
    message: The Gateway has been scheduled by Envoy Gateway
    observedGeneration: 1
    reason: Accepted
    status: "True"
    type: Accepted
  - lastTransitionTime: "2024-05-29T08:28:03Z"
    message: Address assigned to the Gateway, 1/1 envoy Deployment replicas available
    observedGeneration: 1
    reason: Programmed
    status: "True"
    type: Programmed
  listeners:
  - attachedRoutes: 1
    conditions:
    - lastTransitionTime: "2024-05-29T08:28:00Z"
      message: Sending translated listener configuration to the data plane
      observedGeneration: 1
      reason: Programmed
      status: "True"
      type: Programmed
    - lastTransitionTime: "2024-05-29T08:28:00Z"
      message: Listener has been successfully translated
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-05-29T08:28:00Z"
      message: Listener references have been resolved
      observedGeneration: 1
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    name: http
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
    - group: gateway.networking.k8s.io
      kind: GRPCRoute
  - attachedRoutes: 1
    conditions:
    - lastTransitionTime: "2024-05-29T08:28:00Z"
      message: Secret test-envoy/example-com does not exist.
      observedGeneration: 1
      reason: InvalidCertificateRef
      status: "False"
      type: ResolvedRefs
    - lastTransitionTime: "2024-05-29T08:28:00Z"
      message: Listener is invalid, see other Conditions for details.
      observedGeneration: 1
      reason: Invalid
      status: "False"
      type: Programmed
    name: https
    supportedKinds:
    - group: gateway.networking.k8s.io
      kind: HTTPRoute
    - group: gateway.networking.k8s.io
      kind: GRPCRoute

kubectl get httproute example-route -n test-envoy -o yaml

kind: HTTPRoute
metadata:
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: |
      {"apiVersion":"gateway.networking.k8s.io/v1beta1","kind":"HTTPRoute","metadata":{"annotations":{},"name":"example-route","namespace":"test-envoy"},"spec":{"hostnames":["example.com"],"parentRefs":[{"name":"test-gateway"}],"rules":[{"backendRefs":[{"group":"","kind":"Service","name":"example-frontend","port":8080,"weight":1}],"matches":[{"path":{"type":"PathPrefix","value":"/"}}]}]}}
  creationTimestamp: "2024-05-27T13:15:56Z"
  generation: 1
  name: example-route
  namespace: test-envoy
  resourceVersion: "64894569"
  uid: aa125612-d98b-44f2-9802-1fccd08f5c45
spec:
  hostnames:
  - example.com
  parentRefs:
  - group: gateway.networking.k8s.io
    kind: Gateway
    name: test-gateway
  rules:
  - backendRefs:
    - group: ""
      kind: Service
      name: example-frontend
      port: 8080
      weight: 1
    matches:
    - path:
        type: PathPrefix
        value: /
status:
  parents:
  - conditions:
    - lastTransitionTime: "2024-05-29T08:23:00Z"
      message: Route is accepted
      observedGeneration: 1
      reason: Accepted
      status: "True"
      type: Accepted
    - lastTransitionTime: "2024-05-29T08:23:00Z"
      message: Resolved all the Object references for the Route
      observedGeneration: 1
      reason: ResolvedRefs
      status: "True"
      type: ResolvedRefs
    controllerName: gateway.envoyproxy.io/gatewayclass-controller
    parentRef:
      group: gateway.networking.k8s.io
      kind: Gateway
      name: test-gateway

Apparently the error is this - "message: Secret test-envoy/example-com does not exist." but I don't understand what exactly is wrong. I thought secret was created as part of the gateway manifest. Since in the manifest we specify who our issuer is.

[arkodg]

@SerheoReutty can you check if the secret exists ? if it does, you may be hitting #3496

[SerheoReutty]

I don't understand. As far as I understand, the use of issuer implies the creation of a necessary secret. That's how Ingress works, I truly believed that gateway would do the same.
How can I use sert-manager so that the secret is automatically created in the namepsace I need?

[arkodg]

https://cert-manager.io/docs/usage/gateway/ & https://gateway.envoyproxy.io/v1.0.1/tasks/security/tls-cert-manager/ outline this for now, closing this issue as this should be working as expected, please reopen if the integration steps on either of the docs dont work for you, we'll revisit and update the docs

[SerheoReutty]

Yes. I specified the anotation to my cert-manager.

metadata:
  name: test-gateway
  annotations:
    cert-manager.io/cluster-issuer: "my-cert-manager"  ###Real name issuer masked

I create a tls block, as stated in the documentation, to my listener

       tls:
       mode: Terminate
       certificateRefs:
       - kind: Secret
         name: test-gateway-https

I understood everything correctly, judging by the documentation, the anotation tells certmanager to create certificates. and creates a secret in my namespace.
But, secret not create. in gateway logs view this error

   conditions:
   - lastTransitionTime: "2024-05-29T08:28:00Z"
     message: Secret test-envoy/example-com does not exist.
     observedGeneration: 1
     reason: InvalidCertificateRef
     status: "False"
     type: ResolvedRefs
   - lastTransitionTime: "2024-05-29T08:28:00Z"
     message: Listener is invalid, see other Conditions for details.
     observedGeneration: 1
     reason: Invalid
     status: "False"
     type: Programmed

And I don't understand what the problem is. Is there a problem with the listener itself and that's why it doesn't get to creating the certificate? According to cert-manager logs, no request was received there.

[arkodg]

have you enabled the Gateway API feature in cert manager

args:
  - --feature-gates=ExperimentalGatewayAPISupport=true

?