From 8dba09217e76db2d3149b0d07e78566ff6d1483e Mon Sep 17 00:00:00 2001 From: Charles Uneze Date: Fri, 21 Jun 2024 08:03:11 +0100 Subject: [PATCH] Create simulation.md Co-authored-by: Shiming Zhang --- site/config.yaml | 6 ++ .../en/docs/technical-outcomes/simulation.md | 87 +++++++++++++++++++ 2 files changed, 93 insertions(+) create mode 100644 site/content/en/docs/technical-outcomes/simulation.md diff --git a/site/config.yaml b/site/config.yaml index c8b09b08c4..ecf8c506f6 100644 --- a/site/config.yaml +++ b/site/config.yaml @@ -117,6 +117,12 @@ menu: - identifier: contributing pageRef: "/docs/contributing" weight: 30 + # Technical Outcomes + - identifier: technical-outcomes + title: Technical Outcomes + - identifier: simulation + pageRef: "/docs/technical-outcomes/simulation" + parent: technical-outcomes # User Guide Children - identifier: all-in-one-image diff --git a/site/content/en/docs/technical-outcomes/simulation.md b/site/content/en/docs/technical-outcomes/simulation.md new file mode 100644 index 0000000000..753c248724 --- /dev/null +++ b/site/content/en/docs/technical-outcomes/simulation.md @@ -0,0 +1,87 @@ +--- +title: Simulation Testing +--- + +# Simulation Testing with KWOK + +{{< hint "info" >}} + +This document walks you through the technical outcome of using KWOK for simulation testing. + +{{< /hint >}} + +## Introduction to Simulation in Kubernetes with KWOK + +Simulation in Kubernetes environments involves replicating the behavior of real-world scenarios within a virtual +environment. It allows users to test and optimize various configurations without impacting production infrastructure. +Kubernetes presents unique challenges and complexities that simulation aims to address. By simulating Kubernetes +environments, users can assess performance, scalability, and resilience under safe and extreme conditions. It serves +as a learning experience. Examples of simulation tools are KWOK, Kind, Minikube, etc. + +Kubernetes WithOut Kubelet (KWOK) is a simulation tool under the [Kubernetes SIGs (Special Interest Groups)](https://github.com/kubernetes-sigs?q=&type=all&language=&sort=) umbrella +used to simulate resources in a Kubernetes cluster with fake resource attributes, all while consuming low physical +resources. On a 2 GB RAM machine, you can run 100 pods, each with 1 GB of RAM. Simulation in a KWOK cluster differs +from simulation in a Kind cluster; for example, a Kind cluster deploys real applications inside its cluster. If you +deploy a nginx pod to a Kind cluster, you could curl to its IP address and get an HTTP response containing its HTML +page. But in a KWOK cluster, you get nothing because the pod isn't real. + +## Key Simulation Features of KWOK + +1. **Lifecycle configuration:** The pod, node, and node lease can be simulated. + + - **Pod:** Status information like Pending, ContainerCreating, Running, and Terminating can be simulated. + - **Node:** Status information like NodeReady, and Ready states can be simulated. + +2. **Kubelet server** + + - **Metrics:** Using the [Kubernetes Metric Server](https://kwok.sigs.k8s.io/docs/examples/metrics-server/), or [Prometheus](https://kwok.sigs.k8s.io/docs/examples/prometheus/), metrics can be simulated in KWOK. + - **Exec:** Commands can be simulated to [execute](https://kwok.sigs.k8s.io/docs/user/exec-configuration/) inside a container running on a single fake pod or multiple pods. + - **Log:** Container [logs](https://kwok.sigs.k8s.io/docs/user/logs-configuration/) can be simulated in a single pod or multiple pods. + - **Attach:** The input/output streams from the main process running in a container can be [attached](https://kwok.sigs.k8s.io/docs/user/attach-configuration/#attach-configuration) to KWOK. + This can be simulated in a single pod or multiple pods. + - **PortForward:** [Port forwarding](https://kwok.sigs.k8s.io/docs/user/port-forward-configuration/#clusterportforward), which establishes a direct connection to a container port, can be simulated + for a single pod or multiple pods. + +## Applications and Use Cases + +1. **Scheduler Simulation:** Beyond the default Kubernetes scheduler, developers build custom schedulers +tailored to their business requirements. These requirements can include scheduling pods based on actual +usage vs requested usage, scheduling jobs based on queues, scheduling pods based on the location of their data, etc. +To test this custom scheduler, a cluster is required. Additionally, this cluster should be scalable to +test the customized scheduler at scale. To achieve this, KWOK can be used to create a simulated cluster +containing fake pods and jobs that can be scheduled to a fake node. +2. **Simulate Policy Enforcement:** Security teams and projects developing Kubernetes policy-based software +often need to simulate complex scenarios to test and validate policies in a cluster. KWOK can spin up fake +nodes and pods to test the enforcement of these policies. For example, [Kyverno](https://github.com/kyverno/kyverno) is a practice of this use case. +3. **Kubernetes Development and Testing:** Developers and administrators who are testing business requirements +can use KWOK to extend their real cluster. This means that a real cluster can be extended to a fake cluster +running KWOK. Objects like nodes, pods, jobs, init containers, etc. can be simulated. Using [stages](https://kwok.sigs.k8s.io/docs/user/stages-configuration/), the +lifecycle of a pod or node can be fine-tuned to simulate a real resource. Additionally, metrics from objects +can be captured using a Metrics server or Prometheus. Note that the real cluster doesn’t use a Kubernetes +Federation controller manager. +4. **Container Apps:** KWOK can be [deployed in an existing cluster](https://kwok.sigs.k8s.io/docs/user/kwok-in-cluster/) and used to test and develop container-based +applications by simulating thousands of containers, pods, and nodes. This allows developers to evaluate the +scalability of their containerized applications without needing large Kubernetes clusters. This is achieved by +tainting fake nodes to repel real running pods from being rescheduled onto them. Toleration is added to the +fake pods so they can be scheduled to the fake nodes despite the taint. Additionally, a node affinity matching a +label on the fake node provides an extra layer of reassurance, ensuring that the pods are only scheduled to the +fake nodes and never to the real nodes. +5. **Internal Cluster Control Plane:** KWOK uses a real kube-api server running on a Docker, Podman, or Containerd +runtime, which means that it can participate in a federated cluster, allowing a single non-KWOK management cluster +to centrally manage and deploy resources across its multiple clusters. +6. **GPUs:** Graphics processing units (GPUs) are used by developers who are building advanced AI workloads, video +workloads, etc. KWOK can be used to simulate GPU nodes, allowing developers to test and develop GPU-intensive +applications in a controlled environment. This is particularly useful for scenarios where actual GPU hardware +is unavailable or where the cost of running multiple GPUs is high. By simulating GPU nodes, developers can test +and optimize their applications without the need for expensive hardware. +7. **External Cluster Control Plane:** KWOK can be used to create a simulated cluster that interacts with external +control planes. Components of the external control plane can run outside the KWOK cluster, specifically a Docker +container, but manage and orchestrate resources within it through the Kubernetes API. This setup allows for +extensive testing and development of control plane features and interactions without requiring a +full-scale, production-grade Kubernetes cluster. For example, [NuoDB DBaaS](https://github.com/nuodb/terraform-provider-nuodbaas) is a practice of this use case. + +## Conclusion + +KWOK's ability to simulate realistic yet non-impactful diverse scenarios, +from GPU workloads to external control planes, etc. makes it an invaluable +tool for developers and administrators seeking to ensure robust and efficient Kubernetes deployments.