Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add more debugging tips to the documentation #82

Merged
merged 1 commit into from
Dec 21, 2021
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 44 additions & 10 deletions docs/csi-debug.md
Original file line number Diff line number Diff line change
@@ -1,15 +1,35 @@
## CSI driver debug tips
## CSI ISCSI driver troubleshooting tips

### Case#1: volume create/delete failed
-
- locate csi iscsi driver pod
### volume attach/mount failed

In this case, one can verify the ISCSI CSI driver pod is up and running and also
all the containers in the same POD are healthy.

```console
kubectl get pods
```
kubectl get csidriver
NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE
iscsi.csi.k8s.io false false false <unset> false Persistent 22m

Once verified all containers in the POD are healthy, one can also check
problematic application pod `describe` output.

```console
kubectl describe <App POD>
```

You can also get detailed logging of the mount/attach failure from the ISCSI
node plugin POD container as shown below.

- locate csi iscsi driver pod

```api
kubectl get pods
```

from above output make use of the pod name and check the logs of iscsi plugin
container as shown below

- get csi driver logs

```
kubectl logs -f csi-iscsi-node-klh5c -c iscsi
I1217 14:40:55.928307 7 driver.go:48] Driver: iscsi.csi.k8s.io version: 1.0.0
Expand All @@ -27,21 +47,35 @@ I1217 14:40:56.767445 7 utils.go:69] GRPC response: {"name":"iscsi.csi.k8s
```

#### Update driver version quickly by editing driver deployment directly
- update daemonset deployment

iscsi node plugin has been deployed as a `deamonset` object in your cluster, if
a quick update of the plugin image is required, you can do that by editing
the `deamonset` deployment object for the new image as shown below.

- update daemonset deployment

```console
kubectl get ds
NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE
csi-iscsi-node 1 1 1 1 1 kubernetes.io/os=linux 51m

kubectl edit daemmonset csi-iscsi-node
```

change below config, e.g.

```console
image: gcr.io/k8s-staging-sig-storage/iscsiplugin:canary
imagePullPolicy: IfNotPresent

```

```console
$ kubectl logs -f csi-iscsi-node-klh5c -c iscsi
#### Get more details about the ISCSI CSI driver object

One can list the CSI driver object as shown below.

```
kubectl get csidriver
NAME ATTACHREQUIRED PODINFOONMOUNT STORAGECAPACITY TOKENREQUESTS REQUIRESREPUBLISH MODES AGE
iscsi.csi.k8s.io false false false <unset> false Persistent 22m
```