Swimlane demo using Kubernetes, Istio, and OpenTelemetry, including request routing, fallback, and request header propagation.
Swimlane enables traffic routing between services. When a service has a swimlane deployed, traffic can be routed to that swimlane; otherwise, it falls back to the main service.
In this demo, we introduce swimlane for microservices architecture using Kubernetes, service mesh (Istio), and OpenTelemetry.
Swimlane Overview
The above figure illustrates the swimlane concept. Request without swimlane (R1) goes through the main swimlane. Request with swimlane (R2) routes to services in the same swimlane (service A and C); otherwise, falls back to the main swimlane (service B).
Swimlane can be applied to the following scenarios:
To develop multiple features at the same time, we might face a scenario that both featureA and featureB want to do an end-to-end test in dev environment. Usually, we have no choice but to do the testing in a serialized manner. With the help of swimlanes, we can deploy features on different swimlanes to isolate environments and do the testing in parallel.
Independent dev environments
Sometimes we need to route requests to different versioned services based on certain logic, such as canary deployment, or A/B testing. For example, requests from user1 always route to v1 services, and requests from user2 always route to v2 services.
To make fine-grained control over traffic, we can have an algorithm at Gateway to accurately control what swimlane a request should go. For example, when a request comes to Gateway with userId, we can have an algorithm like users with odd id go to swimlane v1, users with even id go to swimlane v2, and so on. We can even have a whitelist to filter traffic.
Fine-grained Traffic Control
We want to implement the swimlane feature transparent to microservices so that each service can focus on its application logic without coupling swimlane logic. Therefore, we need the following abilities in our microservices.
In a microservices architecture, services, especially stateless services, are disposable and go up and down easily. We hope upstream services know where to visit downstream services all the time. We cannot rely on IP addresses otherwise we need to configure our services frequently.
As a result, we introduce Kubernetes, a container orchestration product, to manage our services, which natively supports service discovery via Service resource and etcd, as well as its labels and selectors.
As we deploy services in tags (e.g. v1 or v2), we need to know how to route traffic to services with different tags. For example, requests with x-swimlane: v1 go to v1 services, and requests with x-swimlane: v2 go to v2 services, and so on. Moreover, one key feature in swimlane is fallback: if a service doesn't have a swimlane deployed, the request falls back to main service.
Kubernetes has its limitations in traffic routing. Service simply exposes the network to Pods. Ingress only works as a gateway and route traffic in the very beginning, not for communications between downstream services.
Therefore, we need a mechanism to control the inbound and outbound traffic of each service so that we can control routing in a more fine-grained manner. This is exactly what Service Mesh is doing. We use Istio to take over the traffic between services by injecting a sidecar (proxy) to each service, which is managed by a control plane.
Istio Architecture. source: Istio.io
Swimlane is a piece of info in a request, such as x-swimlane: v1, which needs to be propagated to all downstream services so that for each outbound traffic, service mesh (Istio) knows what services to look for, and it only falls back to main services if that a service doesn't have a deploy on that swimlane.
We can propagate swimlane through request headers. Unfortunately, Istio doesn't support header propagation: Why can't Istio propagate headers instead of the application.
The good news is, we don't need to maintain header-propagation logic in our applications either. OpenTelemetry is a project helps people enable effective observability. We can use its auto-instrumentation to propagate request headers without intruding application code.
Furthermore, we have OpenTelemetry Operator for Kubernetes to run Custom Resource Definitions (CRD) on K8s, which further provides a layer of transparency; otherwise, we need to inject auto-instrument manually in Dockerfile of each application image.
However, auto-instrument doesn't support all programming languages. Languages without auto-instrument do need to maintain some code in applications. Check out Injecting Auto-instrumentation for further information.
The following figure summarizes the tech architecture in this demo. We use
- Kubernetes to manage resources and do the service discovery;
- Istio for service mesh to take over traffic and do the traffic routing with fallback, which is powered by an
Envoy proxy(sidecar); - OpenTelemetry for request headers propagation by auto-instrument, which works as an
init-containerwhen starting a pod.
Demo Tech Architecture
| Product | Version | Desciption |
|---|---|---|
| Kubernetes | 1.27.4 | Resource management and service discovery |
| Istio | 1.20.1 | Service mesh taking over traffic in services |
| OpenTelemetry Operator | 0.90 | Telemetry for header propagation |
| cert-manager | 1.13.3 | Dependency of OpenTelemetry Operator for pod-to-pod communication certificates |
- Docker
- Kubernetes (we use minikube here)
- Istio
# Install istioctl for MacOS users using `brew` brew install istioctl istioctl install --set profile=demo -y kubectl label namespace default istio-injection=enabled
# 1. Start Docker: make sure Docker client is running
# 2. Start K8s environment
minikube start
# 3. Install dependencies (Istio/OpenTelemetry CRD)
make install
# 4. Deploy resources on K8s
# Need to wait for about 2 minutes for resources be ready
make deployThe resources are ready when you see the following message, and you can send requests to the endpoint.
Forwarding from 127.0.0.1:8080 -> 8080
Forwarding from [::1]:8080 -> 8080
In this demo, we have a chain of microservices: Gateway, GoodsService, and PricingService.
- Gateway: Go service, worked as a gateway to dispatch requests.
- GoodsService: Python service, deployed on
main,dev-feature1anddev-feature2swimlanes. - PricingService: Java service, deployed on
mainanddev-feature1swimlanes.
Demo Architecture
- visit all
mainservices
curl "http://127.0.0.1:8080/goods"
curl "http://127.0.0.1:8080/goods" -H 'x-swimlane: main'
- visit all
dev-feature1services
curl "http://127.0.0.1:8080/goods" -H 'x-swimlane: dev-feature1'
- visit
dev-feature2services, falls back tomainif no corresponding swimlane in that service
curl "http://127.0.0.1:8080/goods" -H 'x-swimlane: dev-feature2'
- falls back to
mainsince no service deployed in this swimlane
curl "http://127.0.0.1:8080/goods" -H 'x-swimlane: not_exist'
# 1.Delete resources on K8s
make undeploy
# 2.Uninstall CRD
make uninstall
# 3.Stop K8s
minikube stopKubernetes Gateway API will become the default API for traffic management in Istio in the future. However, at the time this demo is done, configuration for internal mesh (east-west) traffic is an experimental feature of the Gateway API.
This project implemented Istio Request Routing in both Gateway API and Istio API. The master branch uses Gateway API, while istio-api branch uses Istio API. You can go and check it out if you are interested in it.