diff --git a/docs/proposals/proxy-template-redesign.md b/docs/proposals/proxy-template-redesign.md new file mode 100644 index 000000000000..4cd4f6d308e2 --- /dev/null +++ b/docs/proposals/proxy-template-redesign.md @@ -0,0 +1,240 @@ +# Proxy Template redesign + +## Context + +The main goal of Kuma is to provide simple to use policies that abstract away complexity of the Envoy configuration. +Unfortunately, not every field in Envoy can be abstracted by Kuma, sometimes a user needs to fine-tune properties of Envoy. + +Proxy Template is the policy that was suppose to fulfill this need, but it only allows omitting generating Envoy config by Kuma, add new Envoy resources or replace existing one. +The problem is that what user really wants is to **modify** configuration that is generated by Kuma (ex. add a timeout on a cluster, add a filter to the filter chain) + +## Requirements + +* Add new resources on top of Kuma resources (new Cluster/Listener etc.). +* Remove all resources generated by Kuma. +* Remove some resources generated by Kuma. +* Modify resources generated by Kuma (replace field in the config) +* Add new network filter to listener. (take order into account) +* Add new HTTP filter to HTTP Connection Manager. (take order into account) +* Modify filters - although it's not a "top level" Envoy resource, it's very common need to modify it. + +## Configuration model + +Here is a full example of the configuration + +```yaml +type: ProxyTemplate +mesh: default +name: custom-template-1 +selectors: + - match: + service: backend +conf: + imports: + - default-proxy + modifications: + - cluster: + match: # optional + side: inbound # optional + name: backend # optional + operation: add # remove / patch + value: # Envoy YAML + - listener: + match: # optional + side: inbound # optional + name: xyz # optional + operation: add # remove / patch + value: # Envoy YAML + - route: + match: # optional + side: inbound # optional + name: xyz # optional + operation: add # remove / patch + value: # Envoy YAML + - virtualHost: + match: # optional + side: inbound # optional + name: xyz # optional + operation: add # remove / patch + value: # Envoy YAML + - httpFilter: + match: + side: inbound # optional + type: envoy.filters.http.buffer + listenerName: outbound:127.0.0.1:3000 # optional + operation: addBefore # addAfter / addFirst / addLast / remove / patch + value: # YAML of filter + - networkFilter: + operation: addAfter # addAfter / addFirst / addLast / remove / patch + match: + side: inbound # optional + type: envoy.filters.network.rbac + listenerName: inbound:127.0.0.1:1234 # optional + value: # YAML of filter +``` + +cluster/listener/route etc. is `oneof` - you can only use one of them in the object in the array. + +`match` can differ across resources. As we provide this feature for users, we may see more need for matching (for example - match only EDS clusters or match listener with metadata). +`side` can help you match resource that is generated on inbound or outbound side. + +In `httpFilter` and `networkFilter`: +`addAfter`, `addFirst`, `patch` require `type` in match. +`addFirst` and `addLast` you cannot use `name` in match. + +### Examples + +**1) Add a new Cluster on top of configuration generated by Kuma** +```yaml +conf: + imports: + - default-proxy + modifications: + - cluster: + operation: add + value: | + connectTimeout: 5s + name: localhost:9901 + loadAssignment: + clusterName: localhost:9901 + endpoints: + - lbEndpoints: + - endpoint: + address: + socketAddress: + address: 127.0.0.1 + portValue: 9901 + type: STATIC +``` + +There is no `match` section because it's not needed. + +**2) Remove route from the configuration generated by Kuma** +```yaml +conf: + imports: + - default-proxy + modifications: + - route: + match: + name: backend + operation: remove +``` + +**3) Patch cluster** + +Patch the cluster modifying connection timeout + +```yaml +conf: + imports: + - default-proxy + modifications: + - cluster: + match: + name: backend + operation: patch + value: + connectTimeout: 3s +``` + +**4) Add HTTP JWT filter on one listener picked by name** + +Use case: you want to add JWT but only on one inbound of your service + +```yaml +conf: + imports: + - default-proxy + modifications: + - httpFilter: + match: + listenerName: inbound:127.0.0.1:1234 + operation: addFirst + value: | + name: envoy.filters.http.jwt_authn + typedConfig: type.googleapis.com/envoy.config.filter.http.jwt_authn.v2alpha.JwtProvider + issuer: https://example.com + audiences: + - bookstore_android.apps.googleusercontent.com + - bookstore_web.apps.googleusercontent.com + remote_jwks: + http_uri: + uri: https://example.com/.well-known/jwks.json + cluster: example_jwks_cluster + cache_duration: + seconds: 300 +``` + +* Because of `listenerName: "inbound:127.0.0.1:1234"` it will be added only to listener with this name + +**5) Add network filters on every inbound listener** +```yaml +conf: + imports: + - default-proxy + modifications: + - networkFilter: + match: + side: inbound + name: envoy.filters.network.tcp_proxy + operation: addAfter + value: | + name: envoy.filters.network.local_ratelimit + typedConfig: type.googleapis.com/envoy.config.filter.network.local_rate_limit.v2alpha.LocalRateLimit + tokenBucket: + maxTokens: 3 + tokensPerFill: 1 + fillInterval: 3s +``` + +* Because there is no `listenerName` it matches all listeners. +* Because of `side: inbound` it matches on all inbound listeners. +* Because of `name: envoy.filters.network.tcp_proxy` it will be inserted just after `envoy.filters.network.tcp_proxy` + +**6) Remove network filter** + +```yaml +conf: + imports: + - default-proxy + modifications: + - networkFilter: + operation: remove + match: + name: envoy.filters.network.rbac + listenerName: inbound:127.0.0.1:1234 +``` + +**7) Patch HTTP Connection Manager** +```yaml +conf: + imports: + - default-proxy + modifications: + - networkFilter: + match: + type: envoy.filters.network.http_connection_manager + listenerName: inbound:127.0.0.1:1234 + operation: patch + value: + requestTimeout: 3s +``` + +**8) Remove all clusters** +```yaml +conf: + imports: + - default-proxy + modifications: + - cluster: + operation: remove +``` + +There is no `match` so it will match all + +## Notes + +* With new `modifications`, the old `resources` field should be deprecated and then removed. +* Modifications will be executed in order specified in the array +* When you do `operation: add` with resource of the same name that was already generated it should be replaced (just like with current behaviour)