Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added docs and fix for ref grant in service import case #1852

Merged
merged 4 commits into from
Sep 5, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
84 changes: 84 additions & 0 deletions docs/latest/user/multicluster-service.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
# Multicluster Service Routing

The Multicluster Service API ServiceImport object can be used as part of the GatewayAPI backendRef for configuring routes. For more information about multicluster service API follow [sig documentation](https://multicluster.sigs.k8s.io/concepts/multicluster-services-api/).

We will use [Submariner project](https://github.com/submariner-io/submariner) for setting up the multicluster environment for exporting the service to be routed from peer clusters.

# Setting KIND clusters and installing Submariner.

- We will be using KIND clusters to demonstrate this example.

```shell
git clone https://github.com/submariner-io/submariner-operator
cd submariner-operator
make clusters
```

Note: remain in submariner-operator directory for the rest of the steps in this section

- Install subctl:

```shell
curl -Ls https://get.submariner.io | VERSION=v0.14.6 bash
```

- Set up multicluster service API and submariner for cross cluster traffic using ServiceImport

```shell
subctl deploy-broker --kubeconfig output/kubeconfigs/kind-config-cluster1 --globalnet
subctl join --kubeconfig output/kubeconfigs/kind-config-cluster1 broker-info.subm --clusterid cluster1 --natt=false
subctl join --kubeconfig output/kubeconfigs/kind-config-cluster2 broker-info.subm --clusterid cluster2 --natt=false
```

Once the above steps are done and all the pods are up in both the clusters. We are ready for installing envoy gateway.

# Install EnvoyGateway

Install the Gateway API CRDs and Envoy Gateway in cluster1:

```shell
helm install eg oci://docker.io/envoyproxy/gateway-helm --version v0.0.0-latest -n envoy-gateway-system --create-namespace --kubeconfig output/kubeconfigs/kind-config-cluster1
```

Wait for Envoy Gateway to become available:

```shell
kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available --kubeconfig output/kubeconfigs/kind-config-cluster1
```

# Install Application

Install the backend application in cluster2 and export it through subctl command.

```shell
kubectl apply -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/kubernetes/application.yaml --kubeconfig output/kubeconfigs/kind-config-cluster2
subctl export service backend --namespace default --kubeconfig output/kubeconfigs/kind-config-cluster2
```

# Create GatewayAPI Objects

Create the GatewayAPI objects GatewayClass, Gateway and HTTPRoute in cluster1 to set up the routing.

```shell
kubectl apply -f https://raw.githubusercontent.com/envoyproxy/gateway/latest/examples/kubernetes/multicluster-service.yaml --kubeconfig output/kubeconfigs/kind-config-cluster1
```

## Testing the Configuration

Get the name of the Envoy service created the by the example Gateway:

```shell
export ENVOY_SERVICE=$(kubectl get svc -n envoy-gateway-system --selector=gateway.envoyproxy.io/owning-gateway-namespace=default,gateway.envoyproxy.io/owning-gateway-name=eg -o jsonpath='{.items[0].metadata.name}')
```

Port forward to the Envoy service:

```shell
kubectl -n envoy-gateway-system port-forward service/${ENVOY_SERVICE} 8888:80 &
```

Curl the example app through Envoy proxy:

```shell
curl --verbose --header "Host: www.example.com" http://localhost:8888/get
```
53 changes: 53 additions & 0 deletions examples/kubernetes/application.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
apiVersion: v1
kind: ServiceAccount
metadata:
name: backend
---
apiVersion: v1
kind: Service
metadata:
name: backend
labels:
app: backend
service: backend
spec:
ports:
- name: http
port: 3000
targetPort: 3000
selector:
app: backend
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: backend
spec:
replicas: 1
selector:
matchLabels:
app: backend
version: v1
template:
metadata:
labels:
app: backend
version: v1
spec:
serviceAccountName: backend
containers:
- image: gcr.io/k8s-staging-ingressconformance/echoserver:v20221109-7ee2f3e
imagePullPolicy: IfNotPresent
name: backend
ports:
- containerPort: 3000
env:
- name: POD_NAME
valueFrom:
fieldRef:
fieldPath: metadata.name
- name: NAMESPACE
valueFrom:
fieldRef:
fieldPath: metadata.namespace
54 changes: 54 additions & 0 deletions examples/kubernetes/multicluster-service.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
apiVersion: gateway.networking.k8s.io/v1beta1
kind: GatewayClass
metadata:
name: eg
spec:
controllerName: gateway.envoyproxy.io/gatewayclass-controller
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: Gateway
metadata:
name: eg
namespace: default
spec:
gatewayClassName: eg
listeners:
- name: http
protocol: HTTP
port: 80
---
apiVersion: gateway.networking.k8s.io/v1beta1
kind: HTTPRoute
metadata:
name: backend
namespace: default
spec:
parentRefs:
- name: eg
hostnames:
- "www.example.com"
rules:
- backendRefs:
- group: multicluster.x-k8s.io
kind: ServiceImport
name: backend-default-cluster2
namespace: submariner-operator
port: 3000
matches:
- path:
type: PathPrefix
value: /
---
apiVersion: gateway.networking.k8s.io/v1alpha2
kind: ReferenceGrant
metadata:
namespace: submariner-operator
name: referencegrant-1
spec:
from:
- group: gateway.networking.k8s.io
kind: HTTPRoute
namespace: default
to:
- group: multicluster.x-k8s.io
kind: ServiceImport
12 changes: 6 additions & 6 deletions internal/provider/kubernetes/routes.go
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ func (r *gatewayAPIReconciler) processTLSRoutes(ctx context.Context, gatewayName

if backendNamespace != tlsRoute.Namespace {
from := ObjectKindNamespacedName{kind: gatewayapi.KindTLSRoute, namespace: tlsRoute.Namespace, name: tlsRoute.Name}
to := ObjectKindNamespacedName{kind: gatewayapi.KindService, namespace: backendNamespace, name: string(backendRef.Name)}
to := ObjectKindNamespacedName{kind: gatewayapi.KindDerefOr(backendRef.Kind, gatewayapi.KindService), namespace: backendNamespace, name: string(backendRef.Name)}
refGrant, err := r.findReferenceGrant(ctx, from, to)
switch {
case err != nil:
Expand Down Expand Up @@ -142,7 +142,7 @@ func (r *gatewayAPIReconciler) processGRPCRoutes(ctx context.Context, gatewayNam
name: grpcRoute.Name,
}
to := ObjectKindNamespacedName{
kind: gatewayapi.KindService,
kind: gatewayapi.KindDerefOr(backendRef.Kind, gatewayapi.KindService),
namespace: backendNamespace,
name: string(backendRef.Name),
}
Expand Down Expand Up @@ -293,7 +293,7 @@ func (r *gatewayAPIReconciler) processHTTPRoutes(ctx context.Context, gatewayNam
name: httpRoute.Name,
}
to := ObjectKindNamespacedName{
kind: gatewayapi.KindService,
kind: gatewayapi.KindDerefOr(backendRef.Kind, gatewayapi.KindService),
namespace: backendNamespace,
name: string(backendRef.Name),
}
Expand Down Expand Up @@ -360,7 +360,7 @@ func (r *gatewayAPIReconciler) processHTTPRoutes(ctx context.Context, gatewayNam
name: httpRoute.Name,
}
to := ObjectKindNamespacedName{
kind: gatewayapi.KindService,
kind: gatewayapi.KindDerefOr(mirrorBackendRef.Kind, gatewayapi.KindService),
namespace: backendNamespace,
name: string(mirrorBackendRef.Name),
}
Expand Down Expand Up @@ -468,7 +468,7 @@ func (r *gatewayAPIReconciler) processTCPRoutes(ctx context.Context, gatewayName

if backendNamespace != tcpRoute.Namespace {
from := ObjectKindNamespacedName{kind: gatewayapi.KindTCPRoute, namespace: tcpRoute.Namespace, name: tcpRoute.Name}
to := ObjectKindNamespacedName{kind: gatewayapi.KindService, namespace: backendNamespace, name: string(backendRef.Name)}
to := ObjectKindNamespacedName{kind: gatewayapi.KindDerefOr(backendRef.Kind, gatewayapi.KindService), namespace: backendNamespace, name: string(backendRef.Name)}
refGrant, err := r.findReferenceGrant(ctx, from, to)
switch {
case err != nil:
Expand Down Expand Up @@ -530,7 +530,7 @@ func (r *gatewayAPIReconciler) processUDPRoutes(ctx context.Context, gatewayName

if backendNamespace != udpRoute.Namespace {
from := ObjectKindNamespacedName{kind: gatewayapi.KindUDPRoute, namespace: udpRoute.Namespace, name: udpRoute.Name}
to := ObjectKindNamespacedName{kind: gatewayapi.KindService, namespace: backendNamespace, name: string(backendRef.Name)}
to := ObjectKindNamespacedName{kind: gatewayapi.KindDerefOr(backendRef.Kind, gatewayapi.KindService), namespace: backendNamespace, name: string(backendRef.Name)}
refGrant, err := r.findReferenceGrant(ctx, from, to)
switch {
case err != nil:
Expand Down