Support a custom Backend API for FQDN, IP and Unix Domain Sockets

[guydc]

Description:
Unix Domain Sockets are sometimes used instead of TCP sockets, since communication over UDS has lower latency. Envoy Proxy supports UDS addresses for upstream clusters. Currently, Gateway-API and Envoy Gateway do not support UDS, since UDS cannot be represented by a K8s Service resource.

In Envoy Gateway, support for UDS can improve latency to callout data-plane and control-plane services like Global Network/HTTP Rate Limit, Network/HTTP Ext Auth, HTTP Ext Proc and Secret Discovery Service (e.g. SPIRE) that run as sidecars or node-daemons. UDS is less likely to be used for communicating with backend applications, due to the high degree of coupling between the backend and the gateway.

Gateway-API supports a notion of implementation-specific Backend Object References. This requirement is driven by Envoy-specific architecture constraints, and it is not likely to be useful for the general Gateway-API community. For this reason, Envoy Gateway can introduce a new (implementation-specific) resource representing UDS backends.

[arkodg]

this was discussed in the call today

• @guydc @AliceProxy @kflynn are in favor of a single new CRD ( like UnixDomainService ) that acts as a custom backendRef only for unix domain sockets
• there are some security implications to supporting uds, it maybe possible to access the envoy admin server directly if its listening on a unix domain socket (its not by default today) . There's no version of ReferenceGrant that exists for things like this today
• if we are finally going down the path of supporting non k8s backend types, I'd suggest for us to also consider a generic way to represent a address (e.g. config in envoy https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/address.proto#config-core-v3-address) which can handle UnixDomainSocket , IPAddress and FQDN , which I've been trying to champion ever since the project started Support routing to non K8s upstream services #36, but the solution is a suboptimal one requiring multiple configs (Service and EndpointSlice https://gateway.envoyproxy.io/latest/user/traffic/routing-outside-kubernetes/)
• any approach here will probably need to be opt in, and not enabled by default, because this is allowing access to the same host as the proxy

[guydc]

if we are finally going down the path of supporting non k8s backend types, I'd suggest for us to also consider a generic way to represent a address (e.g. config in envoy https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/address.proto#config-core-v3-address) which can handle UnixDomainSocket , IPAddress and FQDN , which I've been trying to champion ever since the project started #36, but the solution is a suboptimal one requiring multiple configs (Service and EndpointSlice https://gateway.envoyproxy.io/latest/user/traffic/routing-outside-kubernetes/)

Big +1 on tackling common and generic non-k8s-service backends as part of this Issue.

any approach here will probably need to be opt in, and not enabled by default, because this is allowing access to the same host as the proxy

Good point. We don't have to treat UDS backends the same way that we treat listener ports. I think that UDS backends can be translated only into XDS clusters. Any changes required in the pod spec (mounts, sidecars, ...) can be handled with an EnvoyProxy deployment patch.

[github-actions]

This issue has been automatically marked as stale because it has not had activity in the last 30 days.