Support Max Requests Per Connection in Backend Traffic Policy

[guydc]

Description:

Envoy's Common HTTP Protocol Options can be used to set a limit on the total number of requests that can be handled by a connection. Once the limit is reached, the connection is closed. Several envoy-based projects expose this setting for upstream connections.

This parameter can be used for two purposes:

• effectively disabling upstream connection reuse
• coordinating persistent connection lifespan with the upstream backend. For example, nginx resets persistent connection after 100/1k requests by default. If the proxy does not preemptively close the connection, a TCP race condition may occur, leading to request failures.

Relevant Links:
Contour: https://projectcontour.io/docs/main/config/api/#projectcontour.io/v1alpha1.ClusterParameters
Istio: https://istio.io/latest/docs/reference/config/networking/destination-rule/#ConnectionPoolSettings-HTTPSettings
Gloo Edge: https://docs.solo.io/gloo-edge/1.7.23/reference/api/github.com/solo-io/gloo/projects/gloo/api/v1/connection.proto.sk/#connectionconfig

[arkodg]

should we split up circuitBreaker into http and tcp and move this there ?

[guydc]

should we split up circuitBreaker into http and tcp and move this there ?

I think that envoy circuit breakers currently apply cluster-level limits on concurrent resource use (concurrent connections, requests in flight, requests in queue). This setting will apply a connection-level limit on total resources (count of requests). So, this will slightly generalize the meaning of circuit breaker in EG.

I think that the istio ConnectionPoolSettings is an interesting interface covering both types of limits (and other attributes).

For the time being, I think that we can add this setting to circuit breaker as you suggest. Or, we can consider a new top-level container that will also include circuit breaker, timeout, keep alive and possibly other BTP attributes. WDYT?

[arkodg]

yeah from a user perspective, this field is about limiting a request (returning early) when the circuit trips, so Id vote to put it in circuitBreaker

connectionPool as another top level field doesnt feel very useful to me - I like retry , circuitBreaker at the top level, but others might find that too overwhelming