[Backport release-1.25] Upgrade multus chart to v4.0.2-build2023081100 #4663

rancherbot · 2023-08-18T10:39:18Z

This is a backport issue for #4660, automatically created via rancherbot by @thomasferrandiz

Original issue description:

Bump multus image to v4.0.2-build20230811 to solve: Multus 4.x broken with the lastest rke versions #4568
Multus chart is now local because upstream was not maintained

Testing

To validate that the image fixes, the linked issue:

deploy rke2 with multus
save file /etc/cni/net.d/00-multus.conf
kill and restart the multus pod for the node
check that the file /etc/cni/net.d/00-multus.conf did not change after the pod restart

The text was updated successfully, but these errors were encountered:

VestigeJ · 2023-08-29T18:12:16Z

##Environment Details
VERSION=v1.25.12+rke2r1
VERSION=v1.25.13-rc1+rke2r1

Infrastructure

Cloud

Node(s) CPU architecture, OS, and version:

ami-0bbc06589f2e4f4f2

Linux 5.14.21-150500.53-default x86_64 GNU/Linux

PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration:

NAME               STATUS   ROLES                       AGE    VERSION
ip-12-31-19-17     Ready    control-plane,etcd,master   5m4s   v1.25.13+rke2r1

Config.yaml:

write-kubeconfig-mode: 644
debug: true
token: YOUR_TOKEN_HERE
profile: cis-1.23
selinux: true
cni: multus,cilium

Reproduction

$ curl https://get.rke2.io --output install-"rke2".sh
$ sudo chmod +x install-"rke2".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf or ~/90-kubelet.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/ or 90-kubelet.conf
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_RKE2_VERSION=v1.25.12+rke2r1 INSTALL_RKE2_EXEC=server ./install-rke2.sh 
$ go_rke2 // sudo systemctl enable rke2-server --now
$ set_kubefig //export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
$ kg no,po -A // kubectl get nodes,pods -A
$ kgp rke2-multus-ds-ftj4x -o yaml -n kube-system | grep -i image:

Results:

pay attention to the build date
$ kgp rke2-multus-ds-ftj4x -o yaml -n kube-system | grep -i image:

    image: rancher/hardened-multus-cni:v4.0.2-build20230707

Validation

$ curl https://get.rke2.io --output install-"rke2".sh
$ sudo chmod +x install-"rke2".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_RKE2_VERSION=v1.25.13-rc1+rke2r1 INSTALL_RKE2_EXEC=server ./install-rke2.sh 
$ go_rke2 // sudo systemctl enable rke2-server --now
$ set_kubefig // export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
$ kg no,po -A // kubectl get nodes,pods -A
$ kgp rke2-multus-ds-trnns -n kube-system -o yaml | grep -i image:

Results:
pay attention to the build date
$ kgp rke2-multus-ds-trnns -n kube-system -o yaml | grep -i image:

    image: rancher/hardened-multus-cni:v4.0.2-build20230811

VestigeJ · 2023-08-29T18:43:19Z

##Environment Details
VERSION=v1.25.12+rke2r1
VERSION=v1.25.13-rc1+rke2r1

Infrastructure

Cloud

Node(s) CPU architecture, OS, and version:

ami-0bbc06589f2e4f4f2

Linux 5.14.21-150500.53-default x86_64 GNU/Linux

PRETTY_NAME="SUSE Linux Enterprise Server 15 SP5"

Cluster Configuration:

NAME               STATUS   ROLES                       AGE    VERSION
ip-12-31-19-17     Ready    control-plane,etcd,master   5m4s   v1.25.13+rke2r1

Config.yaml:

write-kubeconfig-mode: 644
debug: true
token: YOUR_TOKEN_HERE
profile: cis-1.23
selinux: true
cni: multus,cilium

Reproduction

$ curl https://get.rke2.io --output install-"rke2".sh
$ sudo chmod +x install-"rke2".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf or ~/90-kubelet.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/ or 90-kubelet.conf
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_RKE2_VERSION=v1.25.12+rke2r1 INSTALL_RKE2_EXEC=server ./install-rke2.sh 
$ go_rke2 // sudo systemctl enable rke2-server --now
$ set_kubefig //export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
$ kg no,po -A // kubectl get nodes,pods -A
$ kgp rke2-multus-ds-ftj4x -o yaml -n kube-system | grep -i image:
$ sudo cat /etc/cni/net.d/00-multus.conf
$ k delete pod/rke2-multus-ds-ftj4x -n kube-system
$ kgp -n kube-system // observe new multus pod created
$ sudo cat /etc/cni/net.d/00-multus.conf //observe delegates configuration changes on file

Results:

pay attention to the build date
$ kgp rke2-multus-ds-ftj4x -o yaml -n kube-system | grep -i image:

    image: rancher/hardened-multus-cni:v4.0.2-build20230707

before killing/restarting pod
$ sudo cat /etc/cni/net.d/00-multus.conf

{
        "cniVersion": "0.3.1",
        "name": "multus-cni-network",
        "type": "multus",
        "capabilities": {"portMappings":true},
        "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig",
        "delegates": [
                {"cniVersion":"0.3.1","name":"portmap","plugins":[{"enable-debug":false,"log-file":"/var/run/cilium/cilium-cni.log","name":"cilium","type":"cilium-cni"},{"capabilities":{"portMappings":true},"type":"portmap"}]}
        ]
}

after restarting pod
$ k delete pod/rke2-multus-ds-ftj4x -n kube-system
pod "rke2-multus-ds-ftj4x" deleted
$ kgp -n kube-system

rke2-multus-ds-n4zjs                                    1/1     Running     0          36s

$ sudo cat /etc/cni/net.d/00-multus.conf // observe changes in config delegates

{
        "cniVersion": "0.3.1",
        "name": "multus-cni-network",
        "type": "multus",
        "capabilities": {"portMappings":true},
        "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig",
        "delegates": [
                {"capabilities":{"portMappings":true},"cniVersion":"0.3.1","delegates":[{"cniVersion":"0.3.1","name":"portmap","plugins":[{"enable-debug":false,"log-file":"/var/run/cilium/cilium-cni.log","name":"cilium","type":"cilium-cni"},{"capabilities":{"portMappings":true},"type":"portmap"}]}],"kubeconfig":"/etc/cni/net.d/multus.d/multus.kubeconfig","name":"multus-cni-network","type":"multus"}
        ]
}

Validation

$ curl https://get.rke2.io --output install-"rke2".sh
$ sudo chmod +x install-"rke2".sh
$ sudo groupadd --system etcd && sudo useradd -s /sbin/nologin --system -g etcd etcd
$ sudo modprobe ip_vs_rr
$ sudo modprobe ip_vs_wrr
$ sudo modprobe ip_vs_sh
$ sudo printf "on_oovm.panic_on_oom=0 \nvm.overcommit_memory=1 \nkernel.panic=10 \nkernel.panic_ps=1 \nkernel.panic_on_oops=1 \n" > ~/60-rke2-cis.conf
$ sudo cp 60-rke2-cis.conf /etc/sysctl.d/
$ sudo systemctl restart systemd-sysctl
$ sudo INSTALL_RKE2_VERSION=v1.25.13-rc1+rke2r1 INSTALL_RKE2_EXEC=server ./install-rke2.sh 
$ go_rke2 // sudo systemctl enable rke2-server --now
$ set_kubefig // export KUBECONFIG=/etc/rancher/rke2/rke2.yaml
$ kg no,po -A // kubectl get nodes,pods -A
$ kgp rke2-multus-ds-trnns -n kube-system -o yaml | grep -i image:
$ sudo cat /etc/cni/net.d/00-multus.conf.cilium_bak
$ k delete pod/rke2-multus-ds-trnns -n kube-system
$ kgp -n kube-system // observe new multus pod created
$ sudo cat /etc/cni/net.d/00-multus.conf.cilium_bak //observe no change

Results:
pay attention to the build date
$ kgp rke2-multus-ds-trnns -n kube-system -o yaml | grep -i image:

    image: rancher/hardened-multus-cni:v4.0.2-build20230811

Before manually restarting pod

$ sudo cat /etc/cni/net.d/00-multus.conf.cilium_bak

{
        "cniVersion": "0.3.1",
        "name": "multus-cni-network",
        "type": "multus",
        "capabilities": {"portMappings":true},
        "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig",
        "delegates": [
                {"cniVersion":"0.3.1","name":"portmap","plugins":[{"enable-debug":false,"log-file":"/var/run/cilium/cilium-cni.log","type":"cilium-cni"},{"capabilities":{"portMappings":true},"type":"portmap"}]}
        ]
}

$ k delete pod/rke2-multus-ds-trnns -n kube-system
$ kgp -n kube-system

kube-system   rke2-multus-ds-7txnd                                    1/1     Running     0          32s

$ sudo cat /etc/cni/net.d/00-multus.conf.cilium_bak

{
        "cniVersion": "0.3.1",
        "name": "multus-cni-network",
        "type": "multus",
        "capabilities": {"portMappings":true},
        "kubeconfig": "/etc/cni/net.d/multus.d/multus.kubeconfig",
        "delegates": [
                {"cniVersion":"0.3.1","name":"portmap","plugins":[{"enable-debug":false,"log-file":"/var/run/cilium/cilium-cni.log","type":"cilium-cni"},{"capabilities":{"portMappings":true},"type":"portmap"}]}
        ]
}

rancherbot added the area/cni label Aug 18, 2023

rancherbot added this to the v1.25.14+rke2r1 milestone Aug 18, 2023

rancherbot assigned thomasferrandiz Aug 18, 2023

thomasferrandiz mentioned this issue Aug 18, 2023

[Backport release-1.25] upgrade multus chart to v4.0.2-build2023081100 #4665

Merged

est-suse assigned VestigeJ Aug 23, 2023

VestigeJ closed this as completed Aug 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Backport release-1.25] Upgrade multus chart to v4.0.2-build2023081100 #4663

[Backport release-1.25] Upgrade multus chart to v4.0.2-build2023081100 #4663

rancherbot commented Aug 18, 2023

VestigeJ commented Aug 29, 2023

Reproduction

Validation

VestigeJ commented Aug 29, 2023

Reproduction

Validation

[Backport release-1.25] Upgrade multus chart to v4.0.2-build2023081100 #4663

[Backport release-1.25] Upgrade multus chart to v4.0.2-build2023081100 #4663

Comments

rancherbot commented Aug 18, 2023

Testing

VestigeJ commented Aug 29, 2023

Reproduction

Validation

VestigeJ commented Aug 29, 2023

Reproduction

Validation