k8s Gateway API support

[gemelen]

TL;DR

Add support for the Gateway API in the LoadBalancer managing part of the CCM, so its Gateway component would be provided with the LoadBalancer information on par with a LoadBalancer service type.

This would handle the cases when a k8s cluster is deployed and configured (exclusively or preferably) with the Gateway API approach of exposing services. That covers setups managed by latest versions of Cilium, Traefik, Nginx and others, see list of implementations and compatibility tables for v1.2, v1.1.

One related issue has already been raised here.

Expected behavior

In a cluster setup configured in a Gateway API fashion, a Gateway should be able to obtain an external address(es) from an (external) load balancer, created by or provided via Hetzner CCM.

[zfrgt]

that would be nice to have

[am-i-helpful]

Hi!
I was able to provision an external LB in Hetzner-Cloud using the Cilium Gateway-API following the public documentation of hccm I found here (https://github.com/vitobotta/hetzner-k3s/blob/main/docs/Load_balancers.md#load-balancers), and a bit of manual work annotating the service derived as part of 'Gateway' resource which I think is a bug at Cilium's end: cilium/cilium#36792.

In my opinion, the limitation is at the Cilium Gateway-API controller that they are not copying the metadata 'annotation' from the 'Gateway' resource to the provisioned service of type 'LoadBalancer', which would otherwise result in hccm to provision an external LoadBalancer automatically.

[kishoreinvits]

@gemelen , @am-i-helpful

If you are okay with experimental gateway CRDs, infrastructure annotations mentioned like below work with cilium. This is also mentioned in this cilium issue comment. The infrastructure field is GA and has been part of the Standard Channel since v1.2.0

That would be,

kubectl apply -f https://raw.githubusercontent.com/kubernetes-sigs/gateway-api/v1.1.0/config/crd/experimental/gateway.networking.k8s.io_gateways.yaml

kubectl apply -f - <<EOF
apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: my-gateway
  namespace: my-cilium
spec:
  gatewayClassName: cilium
  infrastructure:
    annotations:
      load-balancer.hetzner.cloud/location: fsn1
EOF

Note that cilium v1.16 docs (latest at the time of writing) asks to use v1.1.0 Gateway API CRDs, so I don't recommend using v1.2.0 experimntal as mentioned in that comment until cilium specifies it.

[lukasmetzner]

Hey,

the hcloud-cloud-controller-manager, which implements the cloud-provider interfaces, is not responsible for the Gateway API. This would require the implementation of a new Gateway Controller, which is currently not planned.

Best Regards,
Lukas