entrypoint.adapter.sh
wraps around the official containerized DRBD kernel module loader script entry.sh
to achieve the following goals:
- Automatically adapt to the host operating system type;
- Automatically adapt to the host operating system kernel version
- For the host operating system kernel version supported by this system, drbd will be installed using the corresponding rpm/deb package in the code base.
- For host operating system kernel versions that are not currently supported by this system, download the corresponding version from the drbd official warehouse and install drbd.
- Use pre-built kernel modules for existing RHEL/CentOS hosts;
- Use dynamically built kernel modules for non-stock RHEL/CentOS hosts and Ubuntu hosts;
- Provide two deployment modes: Job (default) and DaemonSet
- Yellow: LINBIT's logic
- Blue: DaoCloud's Logic
LINBIT/drbd https://github.com/LINBIT/drbd/tree/drbd-9.1/docker
- kernel module v9.1.8 with utils v9.12.1
- (EXPERIMENTAL!) kernel module v9.1.11 with utils v9.21.4
- x86_64
- aarch64
x86
- CentOS 7.6/7.7/7.9
- Ubuntu 18.04/22.04
aarch64
- Kylin V10
- Ubuntu 22 Jammy ( experimental: will always install DRBD v9.1.11 )
NOT YET SUPPORTED
- 18+
For dynamically built kernels, the host must have kernel source installed.
# RHEL/CentOS
$ yum install -y kernel-devel-$(uname -r)
# Ubuntu
$ apt-get install -y linux-headers-$(uname -r)
Note:
For major releases of stock RHEL/CentOS 7 and 8,
kernel-devel
is not needed
By default, OS distros are auto-detected by helm lookup
function.
However, in DaemonSet
mode, it can be overridden by adding values to the array distros: []
in values.yaml
.
Distros that are not supported will be ignored.
For example:
distros:
- rhel7
- rhel8
- bionic
#- focal
Deploy the below DaemonSet
. It will bring up a pod on each kubernetes worker node to install DRBD modules and tools:
$ helm repo add drbd-adapter https://hwameistor.io/drbd-adapter/
$ helm repo update drbd-adapter
$ helm pull drbd-adapter/drbd-adapter --untar
$ helm install drbd-adapter ./drbd-adapter -n hwameistor --create-namespace
Users in China may use gcr.m.daocloud.io mirror to accelerate image pull:
$ helm install drbd-adapter ./drbd-adapter \
-n hwameistor --create-namespace \
--set imagePullPolicy=Always \
--set registry=gcr.m.daocloud.io/hwameistor
Only for Experiments!
$ helm install drbd-adapter ./drbd-adapter \
-n hwameistor --create-namespace \
--set imagePullPolicy=Always \
--set registry=gcr.m.daocloud.io/hwameistor\
--set drbdVersion=v9.1.11
Set DeployKind: job
in values.yaml
, which is the default:
$ kubectl get po -l app=drbd-adapter -o wide
NAME READY STATUS RESTARTS AGE IP NODE
drbd-adapter-k8s-worker-1-rhel7-fqpfg 0/2 Completed 0 36m 10.1.44.70 k8s-worker-1
drbd-adapter-k8s-worker-2-rhel8-k45hp 0/2 Completed 0 36m 10.1.82.97 k8s-worker-2
drbd-adapter-k8s-worker-3-bionic-rr7bv 0/2 Completed 0 36m 10.1.15.220 k8s-worker-3
drbd-adapter-k8s-worker-4-focal-xcmnx 0/2 Completed 0 36m 10.1.57.106 k8s-worker-4
drbd-adapter-k8s-worker-5-jammy-7xf4g 0/2 Completed 0 36m 10.1.42.42 k8s-worker-5
Set DeployKind: daemonset
in values.yaml
.
$ kubectl -n hwameistor get po -l app=drbd-adapter -o wide
NAME READY STATUS RESTARTS AGE IP NODE
drbd-adapter-5w74s 1/1 Running 0 11m 10.6.254.23 k8s-worker-3
drbd-adapter-7766x 1/1 Running 0 11m 10.6.254.21 k8s-worker-1
drbd-adapter-cq52p 1/1 Running 0 11m 10.6.254.24 k8s-worker-4
drbd-adapter-hlpvc 1/1 Running 0 11m 10.6.254.22 k8s-worker-2
drbd-adapter-slm5z 1/1 Running 0 11m 10.6.254.25 k8s-worker-5
On POD hosts
$ cat /proc/drbd
$ modinfo drbd
$ lsmod | grep drbd
$ drbdadm --ver
After expanding Kubernetes cluster, to install DRBD on new nodes
DaemonSet
will automatically expand to the new nodes unless affinity
and tolerations
forbid it.
Charts need to be reapplied for the helm lookup
function to identify the new nodes.
$ helm upgrade drbd-adapter ./drbd-adapter -n hwameistor
By default, master nodes are avoided. To deploy on master nodes, modify values.yaml
as below:
Set in nodeAffinity: {}
in values.yaml
Set deployOnMasters: "yes"
Cause:
A possible cause is that /etc/drbd.conf
and /etc/centos-release
get created as directories instead of files. This is a Kubernetes problem. You may find from kubectl describe pod
that those two paths fail to mount.
Solution: Delete those two directories on hosts, and then uninstall chart release and re-install it.
Cause: Secure Boot is not supported yet
Solution: Future releases will handle kernel module signing.