From 822aa44f7b4f27a511192523dff877ca9a2343fb Mon Sep 17 00:00:00 2001 From: Humble Chirammal Date: Tue, 21 Dec 2021 15:57:32 +0530 Subject: [PATCH] Add more debugging tips to the documentation Signed-off-by: Humble Chirammal --- docs/csi-debug.md | 54 ++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 44 insertions(+), 10 deletions(-) diff --git a/docs/csi-debug.md b/docs/csi-debug.md index 0429d982..b9140a69 100644 --- a/docs/csi-debug.md +++ b/docs/csi-debug.md @@ -1,15 +1,35 @@ -## CSI driver debug tips +## CSI ISCSI driver troubleshooting tips -### Case#1: volume create/delete failed -- -- locate csi iscsi driver pod +### volume attach/mount failed + +In this case, one can verify the ISCSI CSI driver pod is up and running and also +all the containers in the same POD are healthy. + +```console +kubectl get pods ``` -kubectl get csidriver -NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE -iscsi.csi.k8s.io false false false false Persistent 22m + +Once verified all containers in the POD are healthy, one can also check +problematic application pod `describe` output. + +```console +kubectl describe +``` + +You can also get detailed logging of the mount/attach failure from the ISCSI +node plugin POD container as shown below. + +- locate csi iscsi driver pod + +```api +kubectl get pods ``` +from above output make use of the pod name and check the logs of iscsi plugin +container as shown below + - get csi driver logs + ``` kubectl logs -f csi-iscsi-node-klh5c -c iscsi I1217 14:40:55.928307 7 driver.go:48] Driver: iscsi.csi.k8s.io version: 1.0.0 @@ -27,7 +47,13 @@ I1217 14:40:56.767445 7 utils.go:69] GRPC response: {"name":"iscsi.csi.k8s ``` #### Update driver version quickly by editing driver deployment directly -- update daemonset deployment + +iscsi node plugin has been deployed as a `deamonset` object in your cluster, if +a quick update of the plugin image is required, you can do that by editing +the `deamonset` deployment object for the new image as shown below. + +- update daemonset deployment + ```console kubectl get ds NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE @@ -35,13 +61,21 @@ csi-iscsi-node 1 1 1 1 1 kubernetes kubectl edit daemmonset csi-iscsi-node ``` + change below config, e.g. + ```console image: gcr.io/k8s-staging-sig-storage/iscsiplugin:canary imagePullPolicy: IfNotPresent ``` -```console -$ kubectl logs -f csi-iscsi-node-klh5c -c iscsi +#### Get more details about the ISCSI CSI driver object + +One can list the CSI driver object as shown below. + +``` +kubectl get csidriver +NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE +iscsi.csi.k8s.io false false false false Persistent 22m ```