Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support custom backendRefs via extensions #4762

Open
muwaqar opened this issue Nov 22, 2024 · 10 comments
Open

Support custom backendRefs via extensions #4762

muwaqar opened this issue Nov 22, 2024 · 10 comments
Labels
area/extensions help wanted Extra attention is needed kind/feature new feature

Comments

@muwaqar
Copy link

muwaqar commented Nov 22, 2024

Description:
Currently Envoy Gateway implements support for the following BackendReference.

However, vendors may need to support specific backends (S3, EC2, Lambda, ...) which EG does not support natively. In that case, the ask is to enhance the Extensions sub-system to invoke a custom extension which can program the filter chain, route and upstream cluster for the configured backend correctly.

[optional Relevant Links:]

Any extra documentation required to understand the issue.

I could find some previous discussion in #4423, specifically the comment at #4423 (comment)

Envoy Gateway extensions doc also mentions support for custom backendRefs is expected in future, but I could not find an issue tracking this, hence raising this.

Allow Extension Developers to introduce their own custom resources for extending the Gateway-API via ExtensionRefs, policyAttachments (future) and backendRefs (future).

@arkodg
Copy link
Contributor

arkodg commented Nov 22, 2024

+1 @muwaqar, @guydc and @zhaohuabing have already +1'd this idea in #4423
would you be interested in driving this work ?

@arkodg arkodg added the help wanted Extra attention is needed label Nov 22, 2024
@arkodg
Copy link
Contributor

arkodg commented Nov 22, 2024

reg AWS backends, there's this issue #688 tracking the work, although aws is a vendor specific backend, its fine to add a aws setting in the Backend resource since a Envoy filter exists for this in Envoy Proxy (we established this precedence when adding the datadog tracing field, which also is a native option in envoy proxy)

@zhaohuabing
Copy link
Member

+1 native support for aws backend as they're widely used, starting with lambda could be a great first step.

@muwaqar
Copy link
Author

muwaqar commented Nov 22, 2024

great to hear there is broader consensus on this, is there a extension hook/api design sketched out or discussed for this already?

@zhaohuabing
Copy link
Member

zhaohuabing commented Nov 22, 2024

Native support for aws lambda can be added to https://github.com/envoyproxy/gateway/blob/main/api/v1alpha1/backend_types.go and implemented in the Gateway API translator and xDS translator.

For custom Backend, there's no extension hook/api design in plance, and this will require furtuer discussion.

@arkodg
Copy link
Contributor

arkodg commented Nov 22, 2024

Here are some links related to the extension manager route

@muwaqar
Copy link
Author

muwaqar commented Dec 4, 2024

I was reading through the Envoy AI Gateway proposal document and seems like they have a similar use-case to set custom resources as HTTPRoute backendRefs (pasting snippet from the doc below).

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
   name:  mistral-routing
spec:
   rules:
   - matches:
     - path:
        type: prefix
        value: /mistral
     backendRefs:
     - group: serving.kserve.io/v1alpha1
       kind: LLMBackend
       name: kserve-mistral

is this enhancement to envoy-gateway already scoped and tracked by that effort?

@arkodg
Copy link
Contributor

arkodg commented Dec 5, 2024

imo its unrelated, afaik the updated API is envoyproxy/ai-gateway#20 and the custom backend links to a custom route type and the ai-gateway controller converts those custom types into Envoy Gateway resources (like HTTPRoute and Backend , (@mathetake please keep me honest here)

@mathetake
Copy link
Member

yeah we need the custom HTTPRoute-ish stuff as well not only Backend, so this won't help us much (by the way the design doc you shared is obsolete and not necessarily reflect the actual stuff Bloomberg and Tetrate worked on)

@liorokman
Copy link
Contributor

Custom backends can already be added with the extension server, but it's a bit complicated.

The idea is to add a regular route with one of the standard backends, and use the VirtualHost modification hook to modify the standard backend's translation into whatever you need.

We could make this easier to do with the extension server by providing a "PlaceholderBackend" that Envoy Gateway doesn't actually know how to translate to XDS, but can send to an extension server for further handling.

For example:

apiVersion: gateway.envoyproxy.io/v1alpha1
kind: PlaceholderBackend
metadata:
  name: go-to-s3
spec: {}
---
apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: backend
spec:
  parentRefs:
    - name: eg
  hostnames:
    - "www.example.com"
  rules:
    - backendRefs:
        - group: gateway.envoyproxy.io
          kind: PlaceholderBackend
          name: go-to-s3
      matches:
        - path:
            type: PathPrefix
            value: /

When the Extension Server receives the call to the VirtualHost hook for the XDS generated for the go-to-s3 backend, it can fill in the details in XDS as needed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/extensions help wanted Extra attention is needed kind/feature new feature
Projects
None yet
Development

No branches or pull requests

5 participants