Skip to content

Commit

Permalink
Kubeamd updates for Windows and Containerd
Browse files Browse the repository at this point in the history
  • Loading branch information
@jsturtevant
jsturtevant committed Apr 11, 2022
1 parent 0b4f78f commit 135ebb4
Show file tree
Hide file tree
Showing 2 changed files with 117 additions and 62 deletions.
2 changes: 1 addition & 1 deletion content/en/docs/setup/production-environment/container-runtimes.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,7 @@ installing the `containerd.io` package can be found at
{{% /tab %}}
{{% tab name="Windows (PowerShell)" %}}

Start a Powershell session, set `$Version` to the desired version (ex: `$Version="1.4.3"`),
Start a Powershell session, set `$Version` to the desired version (ex: `$Version="1.6.2"`),
and then run the following commands:

1. Download containerd:
Expand Down
177 changes: 116 additions & 61 deletions content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -17,8 +17,6 @@ weight: 30
You can use Kubernetes to run a mixture of Linux and Windows nodes, so you can mix Pods that run on Linux on with Pods that run on Windows. This page shows how to register Windows nodes to your cluster.




## {{% heading "prerequisites" %}}
{{< version-check >}}

Expand All @@ -29,17 +27,11 @@ If you are using VXLAN/Overlay networking you must have also have [KB4489899](ht
* A Linux-based Kubernetes kubeadm cluster in which you have access to the control plane (see [Creating a single control-plane cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)).




## {{% heading "objectives" %}}


* Register a Windows node to the cluster
* Configure networking so Pods and Services on Linux and Windows can communicate with each other




<!-- lessoncontent -->

## Getting Started: Adding a Windows Node to Your Cluster
Expand Down Expand Up @@ -107,89 +99,123 @@ Once you have a Linux-based Kubernetes control-plane node you are ready to choos
kube-system kube-flannel-ds-54954 1/1 Running 0 1m
```
1. Add Windows Flannel and kube-proxy DaemonSets
Now you can add Windows-compatible versions of Flannel and kube-proxy. In order
to ensure that you get a compatible version of kube-proxy, you'll need to substitute
the tag of the image. The following example shows usage for Kubernetes {{< param "fullversion" >}},
but you should adjust the version for your own deployment.

```bash
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy.yml | sed 's/VERSION/{{< param "fullversion" >}}/g' | kubectl apply -f -
kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml
```
{{< note >}}
If you're using host-gateway use https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-host-gw.yml instead
{{< /note >}}
{{< note >}}
If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, you have to modify the line:
### Joining a Windows worker node
```powershell
wins cli process run --path /k/flannel/setup.exe --args "--mode=overlay --interface=Ethernet"
```
{{< note >}}
All code snippets in Windows sections are to be run in a PowerShell environment
with elevated permissions (Administrator) on the Windows worker node unless otherwise noted.
{{< /note >}}
in the `flannel-host-gw.yml` or `flannel-overlay.yml` file and specify your interface accordingly.
{{< tabs name="tab-windows-kubeadm-runtime-installation" >}}
```bash
# Example
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml | sed 's/Ethernet/Ethernet0 2/g' | kubectl apply -f -
```
{{< /note >}}
{{% tab name="CRI-containerD" %}}
### Intro
The following instructions are require HostProcess container support with Kubernetes 1.22+. If you do not support HostProcess containers in you cluster you can install Flannel and kube-proxy as windows services directly on the host.
### Joining a Windows worker node
The Before running any of these step the Windows node should have the following Windows Features installed: `Containers`,`Hyper-V`,`Hyper-V-PowerShell`. These can be installed using the Powershell `Install-WindowsFeature` command.
{{< note >}}
All code snippets in Windows sections are to be run in a PowerShell environment
with elevated permissions (Administrator) on the Windows worker node.
The following instructions are the manual steps to configure nodes. You can use the [image-builder](https://image-builder.sigs.k8s.io/capi/windows/windows.html) used as part of the Cluster Api project implements the following along with other optimizations.
{{< /note >}}
{{< tabs name="tab-windows-kubeadm-runtime-installation" >}}
#### Install containerD
{{% tab name="CRI-containerD" %}}
Follow the instructions for [installing the Containerd runtime on Windows using powershell](../../../setup/production-environment/container-runtimes.md#containerd)
#### Install containerD
#### Install kubelet
```powershell
curl.exe -LO https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/Install-Containerd.ps1
.\Install-Containerd.ps1
Kubeadm sets some flags via a environment file. To support this it is common to use [nssm](https://nssm.cc/) to configure kubelet. To install nssm do the following:
``` powershell
mkdir c:\k
$arch = "win64"
curl.exe -L https://k8stestinfrabinaries.blob.core.windows.net/nssm-mirror/nssm-2.24.zip -o nssm.zip
tar.exe C c:\k\ -xvf .\nssm.zip --strip-components 2 */$arch/*.exe
```
{{< note >}}
To install a specific version of containerD specify the version with -ContainerDVersion.
Next we will set `$KubernetesVersion` to the desired version (ex: `$KubernetesVersion="v1.24.0"`), and then run the following commands to install kubelet:
``` powerShell
curl.exe -L https://dl.k8s.io/$KubernetesVersion/bin/windows/amd64/kubelet.exe -o c:\k\kubelet.exe
@"
# Start file needed to support kubeadm extra args
`$FileContent = Get-Content -Path "/var/lib/kubelet/kubeadm-flags.env"
`$kubeAdmArgs = `$FileContent.TrimStart(`'KUBELET_KUBEADM_ARGS=`').Trim(`'"`')
`$args = "--cert-dir=`$env:SYSTEMDRIVE/var/lib/kubelet/pki",
"--config=`$env:SYSTEMDRIVE/var/lib/kubelet/config.yaml",
"--bootstrap-kubeconfig=`$env:SYSTEMDRIVE/etc/kubernetes/bootstrap-kubelet.conf",
"--kubeconfig=`$env:SYSTEMDRIVE/etc/kubernetes/kubelet.conf",
"--hostname-override=$(hostname)",
"--enable-debugging-handlers",
"--cgroups-per-qos=false",
"--enforce-node-allocatable=``"``"",
"--resolv-conf=``"``""
`$kubeletCommandLine = "c:\k\kubelet.exe " + (`$args -join " ") + " `$kubeAdmArgs"
Invoke-Expression `$kubeletCommandLine
"@ | Set-Content -Path c:\k\Start-kubelet.ps1
c:\k\nssm.exe install kubelet Powershell -ExecutionPolicy Bypass -NoProfile c:\k\Start-kubelet.ps1
c:\k\nssm.exe set Kubelet AppStdout C:\k\kubelet.log
c:\k\nssm.exe set Kubelet AppStderr C:\k\kubelet.err.log
```
```powershell
# Example
.\Install-Containerd.ps1 -ContainerDVersion 1.4.1
We need to open a firewall port:
```
New-NetFirewallRule -Name kubelet -DisplayName 'kubelet' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 10250
```
If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, specify the name with `-netAdapterName`.
Verify kubelet is installed with `Get-service kubelet`. Kubeadm will start kubelet during joining the node.
```powershell
# Example
.\Install-Containerd.ps1 -netAdapterName "Ethernet0 2"
```
Get-Service kubelet
Status Name DisplayName
------ ---- -----------
Stopped kubelet kubelet
```
{{< /note >}}
#### Install kubeadm
#### Install wins, kubelet, and kubeadm
Start a Powershell session, set `$KubernetesVersion` to the desired version (ex: `$KubernetesVersion="v1.24.0"`), and then run the following commands:
```PowerShell
curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/scripts/PrepareNode.ps1
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}} -ContainerRuntime containerD
```
[Install `crictl` from the cri-tools package](https://github.com/kubernetes-sigs/cri-tools)
which is required so that kubeadm can talk to the CRI endpoint.
curl.exe -L https://dl.k8s.io/$KubernetesVersion/bin/windows/amd64/kubeadm.exe -o c:\k\kubeadm.exe
```
#### Run `kubeadm` to join the node
Use the command that was given to you when you ran `kubeadm init` on a control plane host.
Use the command that was given to you when you ran `kubeadm init` on a control plane host. You will need to add `--cri-socket "npipe:////./pipe/containerd-containerd"` to tell kubeadm to use the correct containerd pipe.
The command will look like:
```
./kubeadm.exe join 10.240.0.10:6443 --token <your token> --discovery-token-ca-cert-hash sha256:<your hash> --cri-socket "npipe:////./pipe/containerd-containerd"
```
If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
(on a control plane host) to generate a new token and join command.
#### Install Flannel CNI DaemonSet
We will leverage host-process containers to run flannel as a DaemonSet:
```
kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay-hpc.yml
```
#### Install kube-proxy DaemonSet
We will leverage host-process containers to run kube-proxy as a DaemonSet. This can be run from any Linux machine that has kubectl installed with its context configured to your new cluster.
``` powershell
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy-flannel-hpc.yml | sed 's/KUBERNETES_VERSION/v1.23.5/g' | kubectl apply -f -
```
{{% /tab %}}
{{% tab name="Docker Engine" %}}
Expand Down Expand Up @@ -226,6 +252,35 @@ curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools
.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}}
```
#### Add Windows Flannel and kube-proxy DaemonSets
Now you can add Windows-compatible versions of Flannel and kube-proxy. In order
to ensure that you get a compatible version of kube-proxy, you'll need to substitute
the tag of the image. The following example shows usage for Kubernetes {{< param "fullversion" >}},
but you should adjust the version for your own deployment.
```bash
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy.yml | sed 's/VERSION/{{< param "fullversion" >}}/g' | kubectl apply -f -
kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml
```
{{< note >}}
If you're using host-gateway use https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-host-gw.yml instead
{{< /note >}}
{{< note >}}
If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, you have to modify the line:
```powershell
wins cli process run --path /k/flannel/setup.exe --args "--mode=overlay --interface=Ethernet"
```
in the `flannel-host-gw.yml` or `flannel-overlay.yml` file and specify your interface accordingly.
```bash
# Example
curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml | sed 's/Ethernet/Ethernet0 2/g' | kubectl apply -f -
```
#### Run `kubeadm` to join the node
Use the command that was given to you when you ran `kubeadm init` on a control plane host.
Expand Down

0 comments on commit 135ebb4

Please sign in to comment.