Networking Links

---

Kubernetes Networking recommended reading list.

Linux Networking

• Linux Kernel Networking Walkthrough: Getting packets from/to the NIC, Packet processing (TCP/IP Processing, Queuing from/to userspace (Socket Buffers, Flow Control) by tgraf
• Linux Networking Explained: Network devices, Namespaces, Routing, Veth, VLAN, IPVLAN, MACVLAN, MACVTAP, Bonding, Team, OVS, Bridge, BPF, IPSec by tgraf
• Monitoring and Tuning the Linux Networking Stack:
  • Receiving Data
  • Sending Data

Netfilter

• An In-Depth Guide to iptables, the Linux Firewall by supriyo-biswas
• Benchmarking nftables: Regarding scalability, ipset is a blessing to any iptables set up. Nftables follow the path with their native implementation of sets and take the concept to a higher level by extending the list of supported data types and allowing it to be used in further applications using (verdict) maps.
• Nftables ruleset debug/tracing: This is an equivalent of the old iptables method -J TRACE, but with some great improvements.
• • kube-proxy nftables and iptables vs a Service with 100k endpoints: Iptables performance is limited mainly by two reasons; Latency on the first packet of a connection caused by the linear search rule matching, Latency on the programming latency caused by the need to save and restore all the lines to the kernel in each transaction. The kernel community moved to nftables as replacement of iptables, with the goal of removing the existing performance bottlenecks by Antonio Ojea
• Iptables the end of an era: Kubernetes has decided to implement a new nftables proxy by Antonio Ojea
• Add an nftables-based kube-proxy backend: The default kube-proxy implementation on Linux is currently based on iptables. IPTables was the preferred packet filtering and processing system in the Linux kernel for many years (starting with the 2.4 kernel in 2001). However, problems with iptables led to the development of a successor, nftables, first made available in the 3.13 kernel in 2014, and growing increasingly featureful and usable as a replacement for iptables since then. Development on iptables has mostly stopped, with new features and performance improvements primarily going into nftables instead by Dan Winship.

Sockets

• High-Speed Packet Transmission in Go: From net.Dial to AF_XDP: Sending as many packets per second from a Linux machine by Andree Toonk

Linux Namespaces

• Introducing Linux Network Namespaces by scottslowe
• Making sense Of Linux namespaces by Matty9191
• A deep dive into Linux namespaces by iffyio
• Namespaces in operation by mkerrisk
• Containers from Scratch, recording by ericchiang
• Containers from scratch: The sequel, recording by lizrice
• Linux Namespaces by teddyking
• Namespaces, Threads, and Go by squeed
  • Linux Namespaces and Go Don't Mix by brb
• Network Namespaces Basics Explained in 15 Minutes by Mumshad Mannambeth

Kubernetes Networking

• The Almighty Pause Container by ianlewis
• There is No Such Thing as Container Networking, recording by kelseyhightower
• Kubernetes Networking: Behind the scenes by nleiva
• Kubernetes 101: Kubernetes Networking
• The Kubernetes Networking Guide by Michael Kashin
• Kubernetes Networking Demystified: A Brief Guide
• Tracing the path of network traffic in Kubernetes by Kristijan
• Understanding kubernetes networking: pods by Markbnj
• An illustrated guide to Kubernetes Networking [Part 1] by ApsOps
• A Hacker’s Guide to Kubernetes Networking by yaronha
• Kubernetes Cluster Networking
• Why Kubernetes doesn’t use libnetwork by thockin
• Understanding kubernetes networking: services by Markbnj
• Kubernetes Networking by cloudnativelabs
• Understand and Troubleshoot the “Magic” of k8s Networking, recording by rramkumar1 and freehan
• An Illustrated Guide to Kubernetes Networking by thockin
• The ins and outs of networking in Google Container Engine and Kubernetes, recording by thockin and matchstick
• Introduction to Kubernetes Networking, recording by bboreham
• Life of a Packet, recording by matchstick
• Google Kubernetes Engine networking
• Operating a Kubernetes network by jvns
• Understanding the Kubernetes Networking Model by soofaloofa
• Kubernetes Networking Master Class
• Kubernetes and Networks - why is this so dang hard? by thockin
• Bringing Traffic Into Your Kubernetes Cluster by thockin
• Deconstructing Kubernetes Networking
• Certified Calico Operator: Level 1
• Networking and Kubernetes: A Layered Approach by strongjz and Vallery Lancey
• Let's talk about Kubernetes on the Internet: Talks about exposed Kubernetes clusters on the Internet by Rory McCune
• Kubernetes Networking Explained – Guide for Beginners
• How Container Networking Works: a Docker Bridge Network From Scratch: Learn how to virtualize network environments, connect multiple Linux containers using veth pairs and Linux bridge devices, and even tried to configure IP routing and NAT to enable connectivity between the containers and the outside world by Ivan Velichko

IPVS

• IPVS-Based In-Cluster Load Balancing Deep Dive

eBPF

• eBPF, Microservices, Docker, and Cilium: From Novice to Seasoned by zanetworker
• Why is the kernel community replacing iptables with BPF? by cilium
• Using eBPF in Kubernetes
• BPF and XDP Reference Guide by cilium
• pwru (packet, where are you?)
• Dive into BPF: a list of reading material
• The eXpress Data Path: Review of the eXpress Data Path, the new kernel component for fast packet processing.

IPv6

• IPv4/IPv6 dual-stack
• Dual-stack Kubernetes with kubeadm-dind-cluster by pmichali
• kube-v6 by leblancd
• Kubernetes in IPv6 only by valentin2105
• Add IPv4/IPv6 dual stack KEP

• How to run an IPv6 conformance cluster in AWS using kubeadm by aojea

Multi-cluster

• Kubernetes multi-cluster networking made simple by nleiva

CNI

• Container Network Interface Specification
• Container Network Interface and Go
• Understanding CNI (Container Networking Interface)
• CNI, the Container Network Interface, recording by bboreham
• The Container Network Interface (CNI), recording by eyakubovich
• KubeCon "Container Network Interface: Network Plugins", recording by eyakubovich
• Kubernetes and the CNI Where We Are and What's Next, recording by squeed
• Comparative Kubernetes networks solutions
• Comparison of Networking Solutions for Kubernetes
• Comparative Kubernetes networks solutions
• Benchmark results of Kubernetes network plugins (CNI) over 10Gbit/s network
• Large-scale network simulations in Kubernetes, Part 1 - Building a CNI plugin
• Large-scale network simulations in Kubernetes, Part 2 - Network topology
• How to Write Your Own CNI Plug-in with Bash

Calico

• Calico for Kubernetes networking: the basics & examples

Ingress

• Kubernetes Ingress for Beginners
• Kubernetes Ingress with NGINX Ingress Controller Example: Explores the Ingress object in Kubernetes (K8s). Step-by-step walk through setting up an NGINX Ingress controller with Azure Kubernetes Service (AKS).

Network Solutions Details

DNS

• An Introduction to the Kubernetes DNS Service
• 5 – 15s DNS lookups on Kubernetes? by Quentin-M
• Debugging and Monitoring DNS issues in Kubernetes
• Kubernetes DNS at scale: Environments may operate at large scales (+10k nodes) and it seems that the DNS protocol has some limitations, mainly the number of records that fit into a DNS answer by Antonio Ojea
• Kubernetes pods /etc/resolv.conf ndots:5 option and why it may negatively affect your application performances

Routing

• Kubernetes Traffic Engineering with BGP by andrewsykim

Service Mesh

• The Service Mesh: Past, Present, and Future, recording by wmorgan
• Using Istio Multicluster to "Burst" Workloads Between Clusters

Istio

• Life of a Packet in Istio: Sidecar injection, init-container iptables configuration and traffic Interception in Sidecar proxy.
• Understanding the Sidecar Injection, Traffic Intercepting & Routing Process in Istio: What is the sidecar pattern and what advantages does it have? How are the sidecar injections done in Istio? How does the sidecar proxy do transparent traffic intercepting? How is the traffic routed to upstream?
• Tunneling egress traffic through the gateway: Make it possible to enable TCP tunneling in egress gateway. Then apps would perform standard requests that would be routed to the egress gateway, which would be responsible for tunneling received requests.
• Tunneling egress traffic - improvements: This document proposes improvements to recently implemented tunneling destination rule. Traffic cannot be tunneled when a service specifies its protocol as HTTP, GRPC or any other than TCP or TLS (in fact they could be tunneled when routed through a gateway, but with some other problems, so it’s not recommended). The reason for this limitation is that only TcpProxy can tunnel traffic, so in case of HTTP, where the underlying listener is HttpConnectionManager, it’s not possible without redirecting traffic to an intermediary TcpProxy.
• Dynamic setting value of TcpProxy.tunneling_config.hostname : The problem is that the tunneling_config.hostname can only be statically defined, so I can't tunnel traffic to multiple destinations having a single listener. One of the potential solutions would be to set hostname based on SNI. TLS Inspector filter provides SNI value in %REQUESTED_SERVER_NAME%, but it's not possible to overwrite Host header.
• Tunneling HTTP traffic transparently: Sidecars don't terminate TLS traffic, so when you initiate HTTPS request from your application, you can't perform header-based routing. But you can send plain text request, perform header-based routing and then apply a DestinationRule that originates TLS.
• Istio Creates Unnecessary Envoy Clusters: We use Istio's Sidecar CRD to limit to what external services a business service has the access. Despite that, thousands of unused Envoy clusters are created in the business service's Istio sidecar config.
• Troubleshooting "no healthy upstream" errors in Istio service mesh: A "no healthy upstream" error can be caused by Istio, a misconfigured network device, or actual network outages. Thus, it is difficult for the mesh operations team to pinpoint the cause or even predict its occurrence, because DevOps teams may unwittingly apply an incorrect configuration.
• How to direct outgoing traffics using HTTPS_PROXY through Egress Gateway in Service Mesh: All traffic of the client with HTTPS_PROXY would be forwarded to the external proxy through egress gateway of the Istio for accessing the external target service.
• Istio MTLS Smartness Explained: Discuss the underlying concepts behind the Istio’s automatic mTLS.
• Example policies to enforce mTLS between sidecar and egress gateway: The connection between sidecar and egress gateway could be: plaintext in (Istio) mTLS or TLS in (Istio) mTLS. An AuthorizationPolicy is applied on egress gateway to enforce egress access control.
• Aeraki: Manage any layer-7 traffic in a service mesh.