Skip to content

Commit

Permalink
test: remove legacy Windows dockershim templates
Browse files Browse the repository at this point in the history
  • Loading branch information
jackfrancis committed May 17, 2022
1 parent 2a9b44c commit 80a6ddb
Show file tree
Hide file tree
Showing 42 changed files with 111 additions and 7,310 deletions.
2 changes: 0 additions & 2 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -272,10 +272,8 @@ create-management-cluster: $(KUSTOMIZE) $(ENVSUBST) ## Create a management clust
kubectl create configmap calico-ipv6-addon --from-file=templates/addons/calico-ipv6.yaml
kubectl create configmap calico-dual-stack-addon --from-file=templates/addons/calico-dual-stack.yaml
kubectl create configmap calico-windows-addon --from-file=templates/addons/windows/calico
kubectl create configmap flannel-windows-addon --from-file=templates/addons/windows/flannel

kubectl apply -f templates/addons/calico-resource-set.yaml
kubectl apply -f templates/addons/flannel-resource-set.yaml

# Wait for CAPZ deployments
kubectl wait --for=condition=Available --timeout=5m -n capz-system deployment -l cluster.x-k8s.io/provider=infrastructure-azure
Expand Down
3 changes: 0 additions & 3 deletions Tiltfile
Original file line number Diff line number Diff line change
Expand Up @@ -234,18 +234,15 @@ def create_crs():
local(kubectl_cmd + " create configmap calico-addon --from-file=templates/addons/calico.yaml")
local(kubectl_cmd + " delete configmaps calico-ipv6-addon --ignore-not-found=true")
local(kubectl_cmd + " create configmap calico-ipv6-addon --from-file=templates/addons/calico-ipv6.yaml")
local(kubectl_cmd + " delete configmaps flannel-windows-addon --ignore-not-found=true")
local(kubectl_cmd + " delete configmaps csi-proxy-addon --ignore-not-found=true")
local(kubectl_cmd + " create configmap csi-proxy-addon --from-file=templates/addons/windows/csi-proxy/csi-proxy.yaml")

# need to set version for kube-proxy on windows.
os.putenv("KUBERNETES_VERSION", settings.get("kubernetes_version", {}))
local(kubectl_cmd + " create configmap flannel-windows-addon --from-file=templates/addons/windows/flannel/ --dry-run=client -o yaml | " + envsubst_cmd + " | " + kubectl_cmd + " apply -f -")
local(kubectl_cmd + " create configmap calico-windows-addon --from-file=templates/addons/windows/calico/ --dry-run=client -o yaml | " + envsubst_cmd + " | " + kubectl_cmd + " apply -f -")

# set up crs
local(kubectl_cmd + " apply -f templates/addons/calico-resource-set.yaml")
local(kubectl_cmd + " apply -f templates/addons/flannel-resource-set.yaml")
local(kubectl_cmd + " apply -f templates/addons/windows/csi-proxy/csi-proxy-resource-set.yaml")

# create flavor resources from cluster-template files in the templates directory
Expand Down
45 changes: 9 additions & 36 deletions docs/book/src/topics/windows.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ CAPZ enables you to create Windows Kubernetes clusters on Microsoft Azure. We re

### Using Containerd for Windows Clusters

To deploy a cluster using Windows, use the [Windows Containerd flavor template](../../../../templates/cluster-template-machinepool-windows-containerd.yaml).
To deploy a cluster using Windows, use the [Windows flavor template](../../../../templates/cluster-template-machinepool-windows.yaml).

## Deploy a workload

Expand Down Expand Up @@ -82,7 +82,7 @@ The Windows HostProcess Container feature is Alpha for Kubernetes v1.22 and Beta
Current requirements:

- Kubernetes 1.22+
- containerd 1.6+
- containerd 1.6+
- `WindowsHostProcessContainers` feature-gate (Alpha for v1.22) turned on for kube-apiserver and kubelet if using Kubernetes 1.22

These requirements are satisfied by the Windows Containerd Template and Azure Marketplace reference image `cncf-upstream:capi-windows:k8s-1dot22dot1-windows-2019-containerd:2021.10.15`
Expand All @@ -93,21 +93,21 @@ See the CAPI proposal for implementation details: https://github.com/kubernetes-

### VM and VMSS naming

Azure does not support creating Windows VM's with names longer than 15 characters ([see additional details historical restrictions](https://github.com/kubernetes-sigs/cluster-api/issues/2217#issuecomment-743336941)).
Azure does not support creating Windows VM's with names longer than 15 characters ([see additional details historical restrictions](https://github.com/kubernetes-sigs/cluster-api/issues/2217#issuecomment-743336941)).

When creating a cluster with `AzureMachine` if the AzureMachine is longer than 15 characters then the first 9 characters of the cluster name and appends the last 5 characters of the machine to create a unique machine name.
When creating a cluster with `AzureMachine` if the AzureMachine is longer than 15 characters then the first 9 characters of the cluster name and appends the last 5 characters of the machine to create a unique machine name.

When creating a cluster with `Machinepool` if the Machine Pool name is longer than 9 characters then the Machine pool uses the prefix `win` and appends the last 5 characters of the machine pool name.

### VM password and access
The VM password is [random generated](https://cloudbase-init.readthedocs.io/en/latest/plugins.html#setting-password-main)
by Cloudbase-init during provisioning of the VM. For Access to the VM you can use ssh which will be configured with SSH
public key you provided during deployment.
public key you provided during deployment.

To SSH:

```
ssh -t -i .sshkey -o 'ProxyCommand ssh -i .sshkey -W %h:%p capi@<api-server-ip>' capi@<windows-ip>
ssh -t -i .sshkey -o 'ProxyCommand ssh -i .sshkey -W %h:%p capi@<api-server-ip>' capi@<windows-ip>
```

> There is also a [CAPZ kubectl plugin](https://github.com/kubernetes-sigs/cluster-api-provider-azure/blob/main/hack/debugging/Readme.md) that automates the ssh connection using the Management cluster
Expand All @@ -121,43 +121,16 @@ ssh -L 5555:<windows-ip>:3389 capi@<api-server-ip>
And then open an RDP client on your local machine to `localhost:5555`

### Image creation
The images are built using [image-builder](https://github.com/kubernetes-sigs/image-builder) and published the the Azure Market place. They use [Cloudbase-init](https://cloudbase-init.readthedocs.io/en/latest/) to bootstrap the machines via Kubeadm.
The images are built using [image-builder](https://github.com/kubernetes-sigs/image-builder) and published the the Azure Market place. They use [Cloudbase-init](https://cloudbase-init.readthedocs.io/en/latest/) to bootstrap the machines via Kubeadm.

Find the latest published images:
Find the latest published images:

```
az vm image list --publisher cncf-upstream --offer capi-windows -o table --all
az vm image list --publisher cncf-upstream --offer capi-windows -o table --all
Offer Publisher Sku Urn Version
------------ ------------- ---------------------------- ------------------------------------------------------------------ ----------
capi-windows cncf-upstream k8s-1dot22dot1-windows-2019-containerd cncf-upstream:capi-windows:k8s-1dot22dot1-windows-2019-containerd:2021.10.15 2021.10.15
capi-windows cncf-upstream k8s-1dot22dot2-windows-2019-containerd cncf-upstream:capi-windows:k8s-1dot22dot2-windows-2019-containerd:2021.10.15 2021.10.15
```

If you would like customize your images please refer to the documentation on building your own [custom images](custom-images.md).

### Using Docker EE and dockershim for Windows Clusters

<aside class="note warning">

<h1> Warning </h1>

Docker EE and dockershim for Windows Clusters is now deprecated for this project and will be removed in future releases, the recommended approach is to use [Containerd for Windows clusters](#using-containerd-for-windows-clusters). The default Windows template will be configured to used containerd in the future.

</aside>

Windows nodes can either run [Containerd (recommended)](#using-containerd-for-windows-clusters) or Docker EE as the container runtime.
Docker EE requires the dockershim which will be [removed starting with Kubernetes 1.24](https://kubernetes.io/blog/2020/12/02/dockershim-faq/#when-will-dockershim-be-removed) and
will be [maintained by Mirantis](https://www.mirantis.com/blog/mirantis-to-take-over-support-of-kubernetes-dockershim-2/) in the future. We do not plan to support dockershim
after its removal from upstream kubernetes in 1.24.

To deploy a cluster using Windows using dockershim, use the [Windows flavor template](https://raw.githubusercontent.com/kubernetes-sigs/cluster-api-provider-azure/main/templates/cluster-template-windows.yaml).

#### Kube-proxy and CNIs for dockershim

Kube-proxy and Windows CNIs are deployed via Cluster Resource Sets. Windows does not have a kube-proxy image due
to not having Privileged containers which would provide access to the host. The current solution is using wins.exe as
demonstrated in the [Kubeadm support for Windows](https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/adding-windows-nodes/) guide. You may choose to run components as Windows services directly on the node but will require a [custom image](#image-creation) and modifications to the default Docker EE windows template.

Flannel is being used as the default CNI with Docker EE and dockershim. An important note for Flannel vxlan deployments is that the MTU for the linux nodes must be set to 1400.
This is because [Azure's VNET MTU is 1400](https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-tcpip-performance-tuning#azure-and-vm-mtu) which can cause fragmentation on packets sent from the Linux node to Windows node resulting in dropped packets.
To mitigate this we set the Linux eth0 port match 1400 and Flannel will automatically pick this up and [subtract 50](https://github.com/flannel-io/flannel/issues/1011) for the flannel network created.
13 changes: 0 additions & 13 deletions templates/addons/flannel-resource-set.yaml

This file was deleted.

Loading

0 comments on commit 80a6ddb

Please sign in to comment.