[wip] Kubeadm updates for Windows and Containerd

content/en/docs/setup/production-environment/container-runtimes.md

@@ -172,7 +172,7 @@ installing the `containerd.io` package can be found at
 {{% /tab %}}
 {{% tab name="Windows (PowerShell)" %}}
 
-Start a Powershell session, set `$Version` to the desired version (ex: `$Version="1.4.3"`), 
+Start a Powershell session, set `$Version` to the desired version (ex: `$Version="1.6.2"`), 
 and then run the following commands:
 
 1. Download containerd:

content/en/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes.md

@@ -17,8 +17,6 @@ weight: 30
 You can use Kubernetes to run a mixture of Linux and Windows nodes, so you can mix Pods that run on Linux on with Pods that run on Windows. This page shows how to register Windows nodes to your cluster.
 
 
-
-
 ## {{% heading "prerequisites" %}}
  {{< version-check >}}
 
@@ -29,17 +27,11 @@ If you are using VXLAN/Overlay networking you must have also have [KB4489899](ht
 * A Linux-based Kubernetes kubeadm cluster in which you have access to the control plane (see [Creating a single control-plane cluster with kubeadm](/docs/setup/production-environment/tools/kubeadm/create-cluster-kubeadm/)).
 
 
-
-
 ## {{% heading "objectives" %}}
 
-
 * Register a Windows node to the cluster
 * Configure networking so Pods and Services on Linux and Windows can communicate with each other
 
-
-
-
 <!-- lessoncontent -->
 
 ## Getting Started: Adding a Windows Node to Your Cluster
@@ -107,89 +99,123 @@ Once you have a Linux-based Kubernetes control-plane node you are ready to choos
     kube-system   kube-flannel-ds-54954                     1/1          Running   0          1m
     ```
 
-1. Add Windows Flannel and kube-proxy DaemonSets
-
-    Now you can add Windows-compatible versions of Flannel and kube-proxy. In order
-    to ensure that you get a compatible version of kube-proxy, you'll need to substitute
-    the tag of the image. The following example shows usage for Kubernetes {{< param "fullversion" >}},
-    but you should adjust the version for your own deployment.
-
-    ```bash
-    curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy.yml | sed 's/VERSION/{{< param "fullversion" >}}/g' | kubectl apply -f -
-    kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml
-    ```
-    {{< note >}}
-    If you're using host-gateway use https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-host-gw.yml instead
-    {{< /note >}}
-
-    {{< note >}}
-If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, you have to modify the line:
+### Joining a Windows worker node
 
-```powershell
-wins cli process run --path /k/flannel/setup.exe --args "--mode=overlay --interface=Ethernet"
-```
+{{< note >}}
+All code snippets in Windows sections are to be run in a PowerShell environment
+with elevated permissions (Administrator) on the Windows worker node unless otherwise noted.
+{{< /note >}}
 
-in the `flannel-host-gw.yml` or `flannel-overlay.yml` file and specify your interface accordingly.
+{{< tabs name="tab-windows-kubeadm-runtime-installation" >}}
 
-```bash
-# Example
-curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml | sed 's/Ethernet/Ethernet0 2/g' | kubectl apply -f -
-```
-    {{< /note >}}
+{{% tab name="CRI-containerD" %}}
 
+### Intro
 
+The following instructions are require HostProcess container support with Kubernetes 1.22+.  If you do not support HostProcess containers in you cluster you can install Flannel and kube-proxy as windows services directly on the host.  
 
-### Joining a Windows worker node
+The Before running any of these step the Windows node should have the following Windows Features installed: `Containers`,`Hyper-V`,`Hyper-V-PowerShell`.  These can be installed using the Powershell `Install-WindowsFeature` command.
 
 {{< note >}}
-All code snippets in Windows sections are to be run in a PowerShell environment
-with elevated permissions (Administrator) on the Windows worker node.
+The following instructions are the manual steps to configure nodes. You can use the [image-builder](https://image-builder.sigs.k8s.io/capi/windows/windows.html) used as part of the Cluster Api project implements the following along with other optimizations.
 {{< /note >}}
 
-{{< tabs name="tab-windows-kubeadm-runtime-installation" >}}
+#### Install containerD
 
-{{% tab name="CRI-containerD" %}}
+Follow the instructions for [installing the Containerd runtime on Windows using powershell](../../../setup/production-environment/container-runtimes.md#containerd)
 
-#### Install containerD
+#### Install kubelet
 
-```powershell
-curl.exe -LO https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/Install-Containerd.ps1
-.\Install-Containerd.ps1
+Kubeadm sets some flags via a environment file.  To support this it is common to use [nssm](https://nssm.cc/) to configure kubelet.  To install nssm do the following:
+
+``` powershell
+mkdir c:\k
+$arch = "win64"
+curl.exe -L https://k8stestinfrabinaries.blob.core.windows.net/nssm-mirror/nssm-2.24.zip -o nssm.zip
+tar.exe C c:\k\ -xvf .\nssm.zip --strip-components 2 */$arch/*.exe
 ```
 
-{{< note >}}
-To install a specific version of containerD specify the version with -ContainerDVersion.
+Next we will set `$KubernetesVersion` to the desired version (ex: `$KubernetesVersion="v1.24.0"`), and then run the following commands to install kubelet:
+
+``` powerShell
+curl.exe -L https://dl.k8s.io/$KubernetesVersion/bin/windows/amd64/kubelet.exe -o c:\k\kubelet.exe
+
+@"
+# Start file needed to support kubeadm extra args
+`$FileContent = Get-Content -Path "/var/lib/kubelet/kubeadm-flags.env"
+`$kubeAdmArgs = `$FileContent.TrimStart(`'KUBELET_KUBEADM_ARGS=`').Trim(`'"`')
+
+`$args = "--cert-dir=`$env:SYSTEMDRIVE/var/lib/kubelet/pki",
+        "--config=`$env:SYSTEMDRIVE/var/lib/kubelet/config.yaml",
+        "--bootstrap-kubeconfig=`$env:SYSTEMDRIVE/etc/kubernetes/bootstrap-kubelet.conf",
+        "--kubeconfig=`$env:SYSTEMDRIVE/etc/kubernetes/kubelet.conf",
+        "--hostname-override=$(hostname)",
+        "--enable-debugging-handlers",
+        "--cgroups-per-qos=false",
+        "--enforce-node-allocatable=``"``"",
+        "--resolv-conf=``"``""
+
+`$kubeletCommandLine = "c:\k\kubelet.exe " + (`$args -join " ") + " `$kubeAdmArgs"
+Invoke-Expression `$kubeletCommandLine
+"@ | Set-Content -Path c:\k\Start-kubelet.ps1
+
+c:\k\nssm.exe install kubelet Powershell -ExecutionPolicy Bypass -NoProfile c:\k\Start-kubelet.ps1
+c:\k\nssm.exe set Kubelet AppStdout C:\k\kubelet.log
+c:\k\nssm.exe set Kubelet AppStderr C:\k\kubelet.err.log 
+```
 
-```powershell
-# Example
-.\Install-Containerd.ps1 -ContainerDVersion 1.4.1
+We need to open a firewall port:
+
+```
+New-NetFirewallRule -Name kubelet -DisplayName 'kubelet' -Enabled True -Direction Inbound -Protocol TCP -Action Allow -LocalPort 10250
 ```
 
-If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, specify the name with `-netAdapterName`.
+Verify kubelet is installed with `Get-service kubelet`.  Kubeadm will start kubelet during joining the node. 
 
-```powershell
-# Example
-.\Install-Containerd.ps1 -netAdapterName "Ethernet0 2"
+```
+Get-Service kubelet
+Status   Name     DisplayName
+------   ----     -----------
+Stopped  kubelet  kubelet
 ```
 
-{{< /note >}}
+#### Install kubeadm
 
-#### Install wins, kubelet, and kubeadm
+Start a Powershell session, set `$KubernetesVersion` to the desired version (ex: `$KubernetesVersion="v1.24.0"`), and then run the following commands:
 
-```PowerShell
-curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools/master/kubeadm/scripts/PrepareNode.ps1
-.\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}} -ContainerRuntime containerD
 ```
-
-[Install `crictl` from the cri-tools package](https://github.com/kubernetes-sigs/cri-tools)
-which is required so that kubeadm can talk to the CRI endpoint.
+curl.exe -L https://dl.k8s.io/$KubernetesVersion/bin/windows/amd64/kubeadm.exe -o c:\k\kubeadm.exe
+```
 
 #### Run `kubeadm` to join the node
 
-Use the command that was given to you when you ran `kubeadm init` on a control plane host.
+Use the command that was given to you when you ran `kubeadm init` on a control plane host. You will need to add  `--cri-socket "npipe:////./pipe/containerd-containerd"` to tell kubeadm to use the correct containerd pipe.
+
+The command will look like:
+
+```
+ ./kubeadm.exe join 10.240.0.10:6443 --token <your token> --discovery-token-ca-cert-hash sha256:<your hash> --cri-socket "npipe:////./pipe/containerd-containerd" 
+```
+
 If you no longer have this command, or the token has expired, you can run `kubeadm token create --print-join-command`
 (on a control plane host) to generate a new token and join command.
 
+#### Install Flannel CNI DaemonSet
+
+We will leverage host-process containers to run flannel as a DaemonSet:
+
+```
+kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay-hpc.yml
+```
+
+#### Install kube-proxy DaemonSet
+
+We will leverage host-process containers to run kube-proxy as a DaemonSet.  This can be run from any Linux machine that has kubectl installed with its context configured to your new cluster.
+
+``` powershell
+curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy-flannel-hpc.yml | sed 's/KUBERNETES_VERSION/v1.23.5/g' | kubectl apply -f -
+```
+
 {{% /tab %}}
 
 {{% tab name="Docker Engine" %}}
@@ -226,6 +252,35 @@ curl.exe -LO https://raw.githubusercontent.com/kubernetes-sigs/sig-windows-tools
 .\PrepareNode.ps1 -KubernetesVersion {{< param "fullversion" >}}
 ```
 
+#### Add Windows Flannel and kube-proxy DaemonSets
+
+Now you can add Windows-compatible versions of Flannel and kube-proxy. In order
+to ensure that you get a compatible version of kube-proxy, you'll need to substitute
+the tag of the image. The following example shows usage for Kubernetes {{< param "fullversion" >}},
+but you should adjust the version for your own deployment.
+
+```bash
+curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/kube-proxy.yml | sed 's/VERSION/{{< param "fullversion" >}}/g' | kubectl apply -f -
+kubectl apply -f https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml
+```
+{{< note >}}
+If you're using host-gateway use https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-host-gw.yml instead
+{{< /note >}}
+
+{{< note >}}
+If you're using a different interface rather than Ethernet (i.e. "Ethernet0 2") on the Windows nodes, you have to modify the line:
+
+```powershell
+wins cli process run --path /k/flannel/setup.exe --args "--mode=overlay --interface=Ethernet"
+```
+
+in the `flannel-host-gw.yml` or `flannel-overlay.yml` file and specify your interface accordingly.
+
+```bash
+# Example
+curl -L https://github.com/kubernetes-sigs/sig-windows-tools/releases/latest/download/flannel-overlay.yml | sed 's/Ethernet/Ethernet0 2/g' | kubectl apply -f -
+```
+
 #### Run `kubeadm` to join the node
 
 Use the command that was given to you when you ran `kubeadm init` on a control plane host.