Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NDM fails to configure disks, after restarting the pod it works correctly #610

Open
dborman-hpe opened this issue Jul 30, 2021 · 0 comments
Open

NDM fails to configure disks, after restarting the pod it works correctly #610

dborman-hpe opened this issue Jul 30, 2021 · 0 comments
Labels
enhancement feature request release-note/open issue still open / being worked in a release

Comments

@dborman-hpe
Copy link
Contributor

What steps did you take and what happened:
During the initial install, there were two disks available for NDM to configure as block devices. Only one disk got configured, and it was in the Inactive state. When I deleted the ndm pod, the replacement pod came up, found, and activated both disks.

What did you expect to happen:
Normally ndm will find and activate the disks when it comes up.

The output of the following commands will help us better understand what's going on:
[Pasting long output into a GitHub gist or other pastebin is fine.]

  • kubectl get pods -n openebs
    Before restarting the ndm pod
k8s-worker1-bobloblaw-mercury01-bgth:/home/mercury # kubectl get pods -A | grep ndm
openebs          openebs-ndm-d6kzf                                                 1/1     Running             0          11h
openebs          openebs-ndm-operator-5566645876-kw9ck                             1/1     Running             0          11h

After restarting the ndm pod

bobloblaw-mercury01:/opt/cray/terraform # kubectl get pods -A | grep ndm
openebs          openebs-ndm-h6wh6                                                 1/1     Running   0          43m
openebs          openebs-ndm-operator-5566645876-kw9ck                             1/1     Running   0          12h
  • kubectl get blockdevices -n openebs -o yaml
    Before restarting the NDM pod:
k8s-worker1-bobloblaw-mercury01-bgth:/home/mercury # kubectl get bd -A
NAMESPACE   NAME                                           NODENAME                               SIZE            CLAIMSTATE   STATUS     AGE
openebs     blockdevice-8861d59f793ea6410f342d44d7b84446   k8s-worker1-bobloblaw-mercury01-bgth   1920382344704   Unclaimed    Inactive   11h

After restarting the NDM pod, and creating the cspi:

bobloblaw-mercury01:/opt/cray/terraform # kubectl get bd -A
NAMESPACE   NAME                                           NODENAME                               SIZE            CLAIMSTATE   STATUS   AGE
openebs     blockdevice-8861d59f793ea6410f342d44d7b84446   k8s-worker1-bobloblaw-mercury01-bgth   1920382344704   Claimed      Active   12h
openebs     blockdevice-d3c7cd981e7b3b1712816ff4a6b6f892   k8s-worker1-bobloblaw-mercury01-bgth   480102915584    Claimed      Active   44m
  • kubectl get blockdeviceclaims -n openebs -o yaml
    I did not get this information before restarting the NDM pod
  • kubectl logs <ndm daemon pod name> -n openebs - This is from the original NDM pod
[entrypoint.sh] launching ndm process.
E0730 01:46:06.525639       8 logs.go:35] unable to set flag, Error: no such flag -logtostderr
I0730 01:46:06.526100       8 commands.go:72] Starting Node Device Manager Daemon...
I0730 01:46:06.526124       8 commands.go:73] Version Tag : 1.5.0
I0730 01:46:06.526134       8 commands.go:74] GitCommit : 973d8052a36f3cef6594eac2cfc3945cb7df2dc8
I0730 01:46:06.526793       8 features.go:122] Feature gate: GPTBasedUUID, state: enable
I0730 01:46:06.526825       8 features.go:122] Feature gate: APIService, state: disable
I0730 01:46:06.526835       8 features.go:122] Feature gate: UseOSDisk, state: disable
I0730 01:46:06.591689       8 request.go:565] Throttling request took 52.857139ms, request: GET:https://10.16.0.1:443/apis/admissionregistration.k8s.io/v1beta1?timeout=32s
I0730 01:46:06.627628       8 request.go:565] Throttling request took 88.777512ms, request: GET:https://10.16.0.1:443/apis/events.k8s.io/v1beta1?timeout=32s
I0730 01:46:06.677693       8 request.go:565] Throttling request took 138.815529ms, request: GET:https://10.16.0.1:443/apis/apiextensions.k8s.io/v1?timeout=32s
I0730 01:46:06.727671       8 request.go:565] Throttling request took 188.774187ms, request: GET:https://10.16.0.1:443/apis/discovery.k8s.io/v1beta1?timeout=32s
I0730 01:46:06.777651       8 request.go:565] Throttling request took 238.779434ms, request: GET:https://10.16.0.1:443/apis/cstor.openebs.io/v1?timeout=32s
I0730 01:46:06.827626       8 request.go:565] Throttling request took 288.767091ms, request: GET:https://10.16.0.1:443/apis/snapshot.storage.k8s.io/v1?timeout=32s
I0730 01:46:06.877692       8 request.go:565] Throttling request took 338.809819ms, request: GET:https://10.16.0.1:443/apis/snapshot.storage.k8s.io/v1beta1?timeout=32s
I0730 01:46:06.927582       8 request.go:565] Throttling request took 388.688677ms, request: GET:https://10.16.0.1:443/apis/authentication.k8s.io/v1beta1?timeout=32s
I0730 01:46:06.977650       8 request.go:565] Throttling request took 438.733704ms, request: GET:https://10.16.0.1:443/apis/openebs.io/v1alpha1?timeout=32s
I0730 01:46:07.081702       8 request.go:565] Throttling request took 96.080802ms, request: GET:https://10.16.0.1:443/apis/rbac.authorization.k8s.io/v1?timeout=32s
I0730 01:46:07.131714       8 request.go:565] Throttling request took 146.083199ms, request: GET:https://10.16.0.1:443/apis/coordination.k8s.io/v1beta1?timeout=32s
I0730 01:46:07.181677       8 request.go:565] Throttling request took 196.042997ms, request: GET:https://10.16.0.1:443/apis/apiregistration.k8s.io/v1beta1?timeout=32s
I0730 01:46:07.231677       8 request.go:565] Throttling request took 246.032625ms, request: GET:https://10.16.0.1:443/apis/node.k8s.io/v1beta1?timeout=32s
I0730 01:46:07.282307       8 request.go:565] Throttling request took 296.637376ms, request: GET:https://10.16.0.1:443/apis/cstor.openebs.io/v1?timeout=32s
I0730 01:46:07.331736       8 request.go:565] Throttling request took 346.064569ms, request: GET:https://10.16.0.1:443/apis/snapshot.storage.k8s.io/v1beta1?timeout=32s
I0730 01:46:07.381728       8 request.go:565] Throttling request took 396.021016ms, request: GET:https://10.16.0.1:443/apis/discovery.k8s.io/v1beta1?timeout=32s
I0730 01:46:07.431682       8 request.go:565] Throttling request took 445.990453ms, request: GET:https://10.16.0.1:443/apis/snapshot.storage.k8s.io/v1?timeout=32s
I0730 01:46:07.440648       8 controller.go:252] BlockDevice CRD is available
I0730 01:46:07.446079       8 filter.go:60] registering filters
I0730 01:46:07.446107       8 filter.go:68] configured os disk exclude filter : state enable
E0730 01:46:07.446888       8 osdiskexcludefilter.go:92] unable to configure os disk filter for mountpoint: /etc/hosts, error: could not get device mount attributes, Path/MountPoint not present in mounts file
E0730 01:46:07.447151       8 osdiskexcludefilter.go:92] unable to configure os disk filter for mountpoint: /boot, error: could not get device mount attributes, Path/MountPoint not present in mounts file
E0730 01:46:07.447247       8 osdiskexcludefilter.go:104] unable to configure os disk filter for mountpoint: /, error: could not get device mount attributes, Path/MountPoint not present in mounts file
E0730 01:46:07.447399       8 osdiskexcludefilter.go:104] unable to configure os disk filter for mountpoint: /boot, error: could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.447421       8 filter.go:68] configured vendor filter : state enable
I0730 01:46:07.447431       8 filter.go:68] configured path filter : state enable
I0730 01:46:07.447438       8 filter.go:68] configured device validity filter : state enable
I0730 01:46:07.455060       8 probe.go:65] registering probes
I0730 01:46:07.455114       8 probe.go:85] configured seachest probe : state disable
I0730 01:46:07.455132       8 probe.go:85] configured smart probe : state enable
I0730 01:46:07.455143       8 probe.go:85] configured mount probe : state enable
I0730 01:46:07.455176       8 probe.go:85] configured udev probe : state enable
I0730 01:46:07.455236       8 udevprobe.go:337] starting udev probe listener
I0730 01:46:07.456716       8 udevprobe.go:202] device: /dev/sr0, Serial: QM00004 filled during udev scan
I0730 01:46:07.456738       8 udevprobe.go:214] device: /dev/sr0, FileSystemUUID: 2021-07-29-18-38-06-85 filled during udev scan
I0730 01:46:07.462363       8 udevprobe.go:231] Dependents of /dev/sr0 : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.462408       8 udevprobe.go:241] Device: /dev/sr0 is of type: disk
I0730 01:46:07.462615       8 udevprobe.go:206] device: /dev/vda, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 01:46:07.462940       8 udevprobe.go:231] Dependents of /dev/vda : {Parent: Partitions:[/dev/vda1 /dev/vda2] Holders:[] Slaves:[]}
I0730 01:46:07.462959       8 udevprobe.go:241] Device: /dev/vda is of type: disk
I0730 01:46:07.463094       8 udevprobe.go:206] device: /dev/vda1, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 01:46:07.463108       8 udevprobe.go:210] device: /dev/vda1, PartitionEntryUUID: 32163b98-7a6e-4dce-9bea-cab3f4c38282 filled during udev scan
I0730 01:46:07.463263       8 udevprobe.go:231] Dependents of /dev/vda1 : {Parent:/dev/vda Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.463279       8 udevprobe.go:241] Device: /dev/vda1 is of type: partition
I0730 01:46:07.463417       8 udevprobe.go:206] device: /dev/vda2, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 01:46:07.463431       8 udevprobe.go:210] device: /dev/vda2, PartitionEntryUUID: 9d7814c1-2455-4539-86d9-4966d277ba5f filled during udev scan
I0730 01:46:07.463437       8 udevprobe.go:214] device: /dev/vda2, FileSystemUUID: b0f46417-d251-4b95-9552-06fdde604e43 filled during udev scan
I0730 01:46:07.463602       8 udevprobe.go:231] Dependents of /dev/vda2 : {Parent:/dev/vda Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.463617       8 udevprobe.go:241] Device: /dev/vda2 is of type: partition
I0730 01:46:07.463949       8 udevprobe.go:231] Dependents of /dev/vdb : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.463969       8 udevprobe.go:241] Device: /dev/vdb is of type: disk
I0730 01:46:07.464325       8 udevprobe.go:231] Dependents of /dev/vdc : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.464344       8 udevprobe.go:241] Device: /dev/vdc is of type: disk
I0730 01:46:07.467534       8 probe.go:85] configured sysfs probe : state enable
I0730 01:46:07.467557       8 probe.go:85] configured used-by probe : state enable
I0730 01:46:07.467564       8 probe.go:85] configured Custom Tag Probe : state enable
I0730 01:46:07.467651       8 sparsefilegenerator.go:145] No sparse file path/size provided. Skip creating sparse files.
I0730 01:46:07.467810       8 controller.go:282] started the controller
I0730 01:46:07.470988       8 eventhandler.go:60] Processing details for /dev/sr0
I0730 01:46:07.471266       8 udevprobe.go:280] device: /dev/sr0, Model: QEMU_DVD-ROM filled by udev probe
I0730 01:46:07.471282       8 udevprobe.go:288] device: /dev/sr0, Serial: QM00004 filled by udev probe
I0730 01:46:07.471287       8 udevprobe.go:296] device: /dev/sr0, IDType: cd filled by udev probe
I0730 01:46:07.471308       8 probe.go:107] details filled by udev probe
I0730 01:46:07.471387       8 sysfsprobe.go:99] blockdevice path: /dev/sr0 capacity :401408 filled by sysfs probe.
I0730 01:46:07.471432       8 sysfsprobe.go:128] blockdevice path: /dev/sr0 logical block size :2048 filled by sysfs probe.
I0730 01:46:07.471465       8 sysfsprobe.go:140] blockdevice path: /dev/sr0 physical block size :2048 filled by sysfs probe.
I0730 01:46:07.471493       8 sysfsprobe.go:152] blockdevice path: /dev/sr0 hardware sector size :2048 filled by sysfs probe.
I0730 01:46:07.471523       8 sysfsprobe.go:163] blockdevice path: /dev/sr0 drive type :HDD filled by sysfs probe.
I0730 01:46:07.471532       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.471579       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.471598       8 probe.go:107] details filled by smart probe
E0730 01:46:07.471887       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.471897       8 probe.go:107] details filled by mount probe
I0730 01:46:07.471905       8 usedbyprobe.go:122] device: /dev/sr0 is not having any zfs partitions
I0730 01:46:07.477387       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.477418       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.477425       8 addhandler.go:51] device: /dev/sr0 does not exist in cache, the device is now connected to this node
I0730 01:46:07.477459       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sr0
I0730 01:46:07.477505       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sr0
I0730 01:46:07.477532       8 filter.go:89] /dev/sr0 ignored by path filter
I0730 01:46:07.477543       8 eventhandler.go:60] Processing details for /dev/vda
I0730 01:46:07.477751       8 probe.go:107] details filled by udev probe
I0730 01:46:07.477836       8 sysfsprobe.go:99] blockdevice path: /dev/vda capacity :107374182400 filled by sysfs probe.
I0730 01:46:07.477883       8 sysfsprobe.go:128] blockdevice path: /dev/vda logical block size :512 filled by sysfs probe.
I0730 01:46:07.477921       8 sysfsprobe.go:140] blockdevice path: /dev/vda physical block size :512 filled by sysfs probe.
I0730 01:46:07.477946       8 sysfsprobe.go:152] blockdevice path: /dev/vda hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.477969       8 sysfsprobe.go:163] blockdevice path: /dev/vda drive type :HDD filled by sysfs probe.
I0730 01:46:07.477978       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.478017       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.478041       8 probe.go:107] details filled by smart probe
E0730 01:46:07.478465       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.478477       8 probe.go:107] details filled by mount probe
I0730 01:46:07.478495       8 usedbyprobe.go:122] device: /dev/vda is not having any zfs partitions
I0730 01:46:07.478523       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.478533       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.478537       8 addhandler.go:51] device: /dev/vda does not exist in cache, the device is now connected to this node
I0730 01:46:07.478552       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda
I0730 01:46:07.478574       8 filter.go:89] /dev/vda ignored by os disk exclude filter
I0730 01:46:07.478587       8 eventhandler.go:60] Processing details for /dev/vda1
I0730 01:46:07.478815       8 probe.go:107] details filled by udev probe
I0730 01:46:07.478895       8 sysfsprobe.go:99] blockdevice path: /dev/vda1 capacity :8388608 filled by sysfs probe.
I0730 01:46:07.478951       8 sysfsprobe.go:128] blockdevice path: /dev/vda1 logical block size :512 filled by sysfs probe.
I0730 01:46:07.478977       8 sysfsprobe.go:140] blockdevice path: /dev/vda1 physical block size :512 filled by sysfs probe.
I0730 01:46:07.478997       8 sysfsprobe.go:152] blockdevice path: /dev/vda1 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.479016       8 sysfsprobe.go:163] blockdevice path: /dev/vda1 drive type :HDD filled by sysfs probe.
I0730 01:46:07.479020       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.479053       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.479066       8 probe.go:107] details filled by smart probe
E0730 01:46:07.479335       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.479347       8 probe.go:107] details filled by mount probe
I0730 01:46:07.761247       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.761271       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.761279       8 addhandler.go:51] device: /dev/vda1 does not exist in cache, the device is now connected to this node
I0730 01:46:07.761326       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda1
I0730 01:46:07.761360       8 filter.go:89] /dev/vda1 ignored by os disk exclude filter
I0730 01:46:07.761381       8 eventhandler.go:60] Processing details for /dev/vda2
I0730 01:46:07.761613       8 probe.go:107] details filled by udev probe
I0730 01:46:07.761708       8 sysfsprobe.go:99] blockdevice path: /dev/vda2 capacity :107364728320 filled by sysfs probe.
I0730 01:46:07.761786       8 sysfsprobe.go:128] blockdevice path: /dev/vda2 logical block size :512 filled by sysfs probe.
I0730 01:46:07.761822       8 sysfsprobe.go:140] blockdevice path: /dev/vda2 physical block size :512 filled by sysfs probe.
I0730 01:46:07.761867       8 sysfsprobe.go:152] blockdevice path: /dev/vda2 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.761908       8 sysfsprobe.go:163] blockdevice path: /dev/vda2 drive type :HDD filled by sysfs probe.
I0730 01:46:07.761919       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.761964       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.761986       8 probe.go:107] details filled by smart probe
I0730 01:46:07.762441       8 probe.go:107] details filled by mount probe
I0730 01:46:07.763950       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.763964       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.763969       8 addhandler.go:51] device: /dev/vda2 does not exist in cache, the device is now connected to this node
I0730 01:46:07.763988       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda2
I0730 01:46:07.764013       8 filter.go:89] /dev/vda2 ignored by os disk exclude filter
I0730 01:46:07.764024       8 eventhandler.go:60] Processing details for /dev/vdb
I0730 01:46:07.764170       8 probe.go:107] details filled by udev probe
I0730 01:46:07.764244       8 sysfsprobe.go:99] blockdevice path: /dev/vdb capacity :1920383410176 filled by sysfs probe.
I0730 01:46:07.764317       8 sysfsprobe.go:128] blockdevice path: /dev/vdb logical block size :512 filled by sysfs probe.
I0730 01:46:07.764346       8 sysfsprobe.go:140] blockdevice path: /dev/vdb physical block size :512 filled by sysfs probe.
I0730 01:46:07.764378       8 sysfsprobe.go:152] blockdevice path: /dev/vdb hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.764408       8 sysfsprobe.go:163] blockdevice path: /dev/vdb drive type :HDD filled by sysfs probe.
I0730 01:46:07.764417       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.764448       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.764458       8 probe.go:107] details filled by smart probe
E0730 01:46:07.764722       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.764733       8 probe.go:107] details filled by mount probe
I0730 01:46:07.764740       8 usedbyprobe.go:122] device: /dev/vdb is not having any zfs partitions
I0730 01:46:07.765407       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.765419       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.765423       8 addhandler.go:51] device: /dev/vdb does not exist in cache, the device is now connected to this node
I0730 01:46:07.765437       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb
I0730 01:46:07.765461       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb
I0730 01:46:07.765484       8 eventhandler.go:73] Processed details for /dev/vdb
I0730 01:46:07.765502       8 addhandler.go:104] checking if device: /dev/vdb can be uniquely identified
I0730 01:46:07.765515       8 addhandler.go:108] device: /dev/vdb cannot be uniquely identified
I0730 01:46:07.765529       8 addhandler.go:113] starting to create partition on device: /dev/vdb
I0730 01:46:07.771739       8 partition.go:164] created a single partition on disk /dev/vdb
I0730 01:46:07.771763       8 addhandler.go:123] created new partition in /dev/vdb
I0730 01:46:07.771768       8 eventhandler.go:60] Processing details for /dev/vdc
I0730 01:46:07.771995       8 probe.go:107] details filled by udev probe
I0730 01:46:07.772095       8 sysfsprobe.go:99] blockdevice path: /dev/vdc capacity :480103981056 filled by sysfs probe.
I0730 01:46:07.772140       8 sysfsprobe.go:128] blockdevice path: /dev/vdc logical block size :512 filled by sysfs probe.
I0730 01:46:07.772190       8 sysfsprobe.go:140] blockdevice path: /dev/vdc physical block size :512 filled by sysfs probe.
I0730 01:46:07.772833       8 sysfsprobe.go:152] blockdevice path: /dev/vdc hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.772897       8 sysfsprobe.go:163] blockdevice path: /dev/vdc drive type :HDD filled by sysfs probe.
I0730 01:46:07.772911       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.772948       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.772965       8 probe.go:107] details filled by smart probe
E0730 01:46:07.773324       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.773343       8 probe.go:107] details filled by mount probe
I0730 01:46:07.773350       8 usedbyprobe.go:122] device: /dev/vdc is not having any zfs partitions
I0730 01:46:07.773565       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.773576       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.773581       8 addhandler.go:51] device: /dev/vdc does not exist in cache, the device is now connected to this node
I0730 01:46:07.773598       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc
I0730 01:46:07.773622       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc
I0730 01:46:07.773652       8 eventhandler.go:73] Processed details for /dev/vdc
I0730 01:46:07.773664       8 addhandler.go:104] checking if device: /dev/vdc can be uniquely identified
I0730 01:46:07.773678       8 addhandler.go:108] device: /dev/vdc cannot be uniquely identified
I0730 01:46:07.773695       8 addhandler.go:113] starting to create partition on device: /dev/vdc
I0730 01:46:07.778814       8 event.go:47] processing new event for (/dev/vdb) action type change
I0730 01:46:07.779618       8 event.go:84] Dependents of /dev/vdb : {Parent: Partitions:[/dev/vdb1] Holders:[] Slaves:[]}
I0730 01:46:07.779710       8 event.go:93] Device: /dev/vdb is of type: disk
I0730 01:46:07.782864       8 partition.go:164] created a single partition on disk /dev/vdc
I0730 01:46:07.782897       8 addhandler.go:123] created new partition in /dev/vdc
I0730 01:46:07.784708       8 event.go:47] processing new event for (/dev/vdb1) action type add
I0730 01:46:07.784986       8 event.go:84] Dependents of /dev/vdb1 : {Parent:/dev/vdb Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.785020       8 event.go:93] Device: /dev/vdb1 is of type: partition
I0730 01:46:07.791194       8 event.go:47] processing new event for (/dev/vdc) action type change
I0730 01:46:07.791579       8 event.go:84] Dependents of /dev/vdc : {Parent: Partitions:[/dev/vdc1] Holders:[] Slaves:[]}
I0730 01:46:07.791607       8 event.go:93] Device: /dev/vdc is of type: disk
I0730 01:46:07.793056       8 eventhandler.go:60] Processing details for /dev/vdb1
I0730 01:46:07.793443       8 probe.go:107] details filled by udev probe
I0730 01:46:07.793561       8 sysfsprobe.go:99] blockdevice path: /dev/vdb1 capacity :1920382344704 filled by sysfs probe.
I0730 01:46:07.793648       8 sysfsprobe.go:128] blockdevice path: /dev/vdb1 logical block size :512 filled by sysfs probe.
I0730 01:46:07.793705       8 sysfsprobe.go:140] blockdevice path: /dev/vdb1 physical block size :512 filled by sysfs probe.
I0730 01:46:07.793733       8 sysfsprobe.go:152] blockdevice path: /dev/vdb1 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.793778       8 sysfsprobe.go:163] blockdevice path: /dev/vdb1 drive type :HDD filled by sysfs probe.
I0730 01:46:07.793790       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.793861       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.793881       8 probe.go:107] details filled by smart probe
E0730 01:46:07.794430       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.794442       8 probe.go:107] details filled by mount probe
I0730 01:46:07.799639       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.799654       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.799661       8 addhandler.go:51] device: /dev/vdb1 does not exist in cache, the device is now connected to this node
I0730 01:46:07.799693       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 01:46:07.799729       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 01:46:07.799758       8 eventhandler.go:73] Processed details for /dev/vdb1
I0730 01:46:07.799875       8 addhandler.go:104] checking if device: /dev/vdb1 can be uniquely identified
I0730 01:46:07.799893       8 uuid.go:72] device(/dev/vdb1) is a partition, using partition UUID: bf2f8fe1-f2e7-4a85-b063-3c15b675c16b
I0730 01:46:07.799903       8 uuid.go:91] generated uuid: blockdevice-8861d59f793ea6410f342d44d7b84446 for device: /dev/vdb1
I0730 01:46:07.799911       8 addhandler.go:128] uuid: blockdevice-8861d59f793ea6410f342d44d7b84446 has been generated for device: /dev/vdb1
E0730 01:46:07.802922       8 blockdevicestore.go:141] Unable to get blockdevice object : blockdevices.openebs.io "blockdevice-8861d59f793ea6410f342d44d7b84446" not found
I0730 01:46:07.802949       8 addhandler.go:132] device: /dev/vdb1, uuid: blockdevice-8861d59f793ea6410f342d44d7b84446 not found in etcd
I0730 01:46:07.802959       8 addhandler.go:140] device: /dev/vdb1 is partition
I0730 01:46:07.802971       8 addhandler.go:141] checking if device has a parent
I0730 01:46:07.802983       8 addhandler.go:149] parent device: /dev/vdb found for device: /dev/vdb1
I0730 01:46:07.802999       8 addhandler.go:150] checking if parent device can be uniquely identified
I0730 01:46:07.803012       8 addhandler.go:153] unable to generate UUID for parent device, may be a device without WWN
I0730 01:46:07.803026       8 addhandler.go:256] creating block device resource for device: /dev/vdb1 with uuid: blockdevice-8861d59f793ea6410f342d44d7b84446
I0730 01:46:07.814003       8 blockdevicestore.go:43] eventcode=ndm.blockdevice.create.success msg=Created blockdevice object in etcd rname=blockdevice-8861d59f793ea6410f342d44d7b84446
I0730 01:46:07.814148       8 event.go:47] processing new event for (/dev/vdc1) action type add
E0730 01:46:07.814270       8 event.go:76] could not get sysfs device for /dev/vdc1, err: unable to create sysfs device from devpath for device: /dev/vdc1, error: lstat /sys/class/block/vdc1: no such file or directory
I0730 01:46:07.819580       8 eventhandler.go:60] Processing details for /dev/vdc1
I0730 01:46:07.819837       8 event.go:47] processing new event for (/dev/vdc1) action type remove
I0730 01:46:07.819859       8 probe.go:107] details filled by udev probe
I0730 01:46:07.819963       8 sysfsprobe.go:99] blockdevice path: /dev/vdc1 capacity :480102915584 filled by sysfs probe.
E0730 01:46:07.819975       8 sysfsprobe.go:109] cannot find the parent disk for partition: /dev/vdc1
I0730 01:46:07.819979       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.820042       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.820055       8 probe.go:107] details filled by smart probe
E0730 01:46:07.821197       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.821223       8 probe.go:107] details filled by mount probe
I0730 01:46:07.825929       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.825954       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.825962       8 addhandler.go:51] device: /dev/vdc1 does not exist in cache, the device is now connected to this node
I0730 01:46:07.826010       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 01:46:07.826061       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 01:46:07.826122       8 eventhandler.go:73] Processed details for /dev/vdc1
E0730 01:46:07.826136       8 addhandler.go:343] unable to find parent device for /dev/vdc1
E0730 01:46:07.826157       8 addhandler.go:68] error handling unmanaged device /dev/vdc1. error: error in getting parent device for /dev/vdc1 from device hierarchy
E0730 01:46:07.826169       8 eventhandler.go:84] error in getting parent device for /dev/vdc1 from device hierarchy
I0730 01:46:07.826511       8 event.go:47] processing new event for (/dev/vdb1) action type remove
I0730 01:46:07.827839       8 udevprobe.go:202] device: /dev/sr0, Serial: QM00004 filled during udev scan
I0730 01:46:07.827863       8 udevprobe.go:214] device: /dev/sr0, FileSystemUUID: 2021-07-29-18-38-06-85 filled during udev scan
I0730 01:46:07.828215       8 udevprobe.go:231] Dependents of /dev/sr0 : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.828289       8 udevprobe.go:241] Device: /dev/sr0 is of type: disk
I0730 01:46:07.828460       8 udevprobe.go:206] device: /dev/vda, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 01:46:07.828742       8 udevprobe.go:231] Dependents of /dev/vda : {Parent: Partitions:[/dev/vda1 /dev/vda2] Holders:[] Slaves:[]}
I0730 01:46:07.828759       8 udevprobe.go:241] Device: /dev/vda is of type: disk
I0730 01:46:07.828923       8 udevprobe.go:206] device: /dev/vda1, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 01:46:07.828940       8 udevprobe.go:210] device: /dev/vda1, PartitionEntryUUID: 32163b98-7a6e-4dce-9bea-cab3f4c38282 filled during udev scan
I0730 01:46:07.829090       8 udevprobe.go:231] Dependents of /dev/vda1 : {Parent:/dev/vda Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.829103       8 udevprobe.go:241] Device: /dev/vda1 is of type: partition
I0730 01:46:07.829238       8 udevprobe.go:206] device: /dev/vda2, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 01:46:07.829250       8 udevprobe.go:210] device: /dev/vda2, PartitionEntryUUID: 9d7814c1-2455-4539-86d9-4966d277ba5f filled during udev scan
I0730 01:46:07.829256       8 udevprobe.go:214] device: /dev/vda2, FileSystemUUID: b0f46417-d251-4b95-9552-06fdde604e43 filled during udev scan
I0730 01:46:07.829462       8 udevprobe.go:231] Dependents of /dev/vda2 : {Parent:/dev/vda Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.829480       8 udevprobe.go:241] Device: /dev/vda2 is of type: partition
I0730 01:46:07.829644       8 udevprobe.go:206] device: /dev/vdb, PartitionTableUUID: 5c638168-78c5-4b6f-880e-5de7e4936592 filled during udev scan
I0730 01:46:07.829889       8 udevprobe.go:231] Dependents of /dev/vdb : {Parent: Partitions:[/dev/vdb1] Holders:[] Slaves:[]}
I0730 01:46:07.829904       8 udevprobe.go:241] Device: /dev/vdb is of type: disk
I0730 01:46:07.830235       8 udevprobe.go:231] Dependents of /dev/vdb1 : {Parent:/dev/vdb Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.830253       8 udevprobe.go:241] Device: /dev/vdb1 is of type: partition
I0730 01:46:07.830384       8 udevprobe.go:206] device: /dev/vdc, PartitionTableUUID: 03a05148-43a1-4c2c-9d0d-d82c28f22d77 filled during udev scan
I0730 01:46:07.831754       8 deletehandler.go:31] Disk /dev/vdc1 not in hierarchy
I0730 01:46:07.832069       8 event.go:47] processing new event for (/dev/vdc) action type change
I0730 01:46:07.832386       8 event.go:84] Dependents of /dev/vdc : {Parent: Partitions:[/dev/vdc1] Holders:[] Slaves:[]}
I0730 01:46:07.832432       8 event.go:93] Device: /dev/vdc is of type: disk
I0730 01:46:07.832514       8 udevprobe.go:231] Dependents of /dev/vdc : {Parent: Partitions:[/dev/vdc1] Holders:[] Slaves:[]}
I0730 01:46:07.832730       8 udevprobe.go:241] Device: /dev/vdc is of type: disk
I0730 01:46:07.835850       8 deletehandler.go:31] Disk /dev/vdb1 not in hierarchy
I0730 01:46:07.835932       8 event.go:47] processing new event for (/dev/vdb) action type change
I0730 01:46:07.836391       8 event.go:84] Dependents of /dev/vdb : {Parent: Partitions:[/dev/vdb1] Holders:[] Slaves:[]}
I0730 01:46:07.836410       8 event.go:93] Device: /dev/vdb is of type: disk
I0730 01:46:07.837336       8 event.go:47] processing new event for (/dev/vdc1) action type add
I0730 01:46:07.837536       8 event.go:84] Dependents of /dev/vdc1 : {Parent:/dev/vdc Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.837546       8 event.go:93] Device: /dev/vdc1 is of type: partition
I0730 01:46:07.838503       8 event.go:47] processing new event for (/dev/vdb1) action type add
I0730 01:46:07.838716       8 event.go:84] Dependents of /dev/vdb1 : {Parent:/dev/vdb Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.838731       8 event.go:93] Device: /dev/vdb1 is of type: partition
I0730 01:46:07.841142       8 eventhandler.go:60] Processing details for /dev/vdc1
I0730 01:46:07.841137       8 udevprobe.go:231] Dependents of /dev/vdc1 : {Parent:/dev/vdc Partitions:[] Holders:[] Slaves:[]}
I0730 01:46:07.841408       8 udevprobe.go:241] Device: /dev/vdc1 is of type: partition
I0730 01:46:07.841460       8 probe.go:107] details filled by udev probe
I0730 01:46:07.841585       8 sysfsprobe.go:99] blockdevice path: /dev/vdc1 capacity :480102915584 filled by sysfs probe.
I0730 01:46:07.841663       8 sysfsprobe.go:128] blockdevice path: /dev/vdc1 logical block size :512 filled by sysfs probe.
I0730 01:46:07.841704       8 sysfsprobe.go:140] blockdevice path: /dev/vdc1 physical block size :512 filled by sysfs probe.
I0730 01:46:07.841742       8 sysfsprobe.go:152] blockdevice path: /dev/vdc1 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.841778       8 sysfsprobe.go:163] blockdevice path: /dev/vdc1 drive type :HDD filled by sysfs probe.
I0730 01:46:07.841786       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.841849       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.841873       8 probe.go:107] details filled by smart probe
E0730 01:46:07.842285       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.842299       8 probe.go:107] details filled by mount probe
I0730 01:46:07.849298       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.849320       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.849327       8 addhandler.go:51] device: /dev/vdc1 does not exist in cache, the device is now connected to this node
I0730 01:46:07.849381       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 01:46:07.849414       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 01:46:07.849437       8 eventhandler.go:73] Processed details for /dev/vdc1
E0730 01:46:07.849447       8 addhandler.go:343] unable to find parent device for /dev/vdc1
E0730 01:46:07.849475       8 addhandler.go:68] error handling unmanaged device /dev/vdc1. error: error in getting parent device for /dev/vdc1 from device hierarchy
E0730 01:46:07.849485       8 eventhandler.go:84] error in getting parent device for /dev/vdc1 from device hierarchy
E0730 01:46:07.849555       8 udevprobe.go:135] Scan is in progress
I0730 01:46:07.852219       8 blockdevicestore.go:129] eventcode=ndm.blockdevice.deactivate.success msg=Deactivated blockdevice rname=blockdevice-8861d59f793ea6410f342d44d7b84446
I0730 01:46:07.852688       8 eventhandler.go:60] Processing details for /dev/vdb1
I0730 01:46:07.852920       8 probe.go:107] details filled by udev probe
I0730 01:46:07.853001       8 sysfsprobe.go:99] blockdevice path: /dev/vdb1 capacity :1920382344704 filled by sysfs probe.
I0730 01:46:07.853056       8 sysfsprobe.go:128] blockdevice path: /dev/vdb1 logical block size :512 filled by sysfs probe.
I0730 01:46:07.853084       8 sysfsprobe.go:140] blockdevice path: /dev/vdb1 physical block size :512 filled by sysfs probe.
I0730 01:46:07.853126       8 sysfsprobe.go:152] blockdevice path: /dev/vdb1 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.853153       8 sysfsprobe.go:163] blockdevice path: /dev/vdb1 drive type :HDD filled by sysfs probe.
I0730 01:46:07.853158       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.853200       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.853212       8 probe.go:107] details filled by smart probe
E0730 01:46:07.853564       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.853569       8 probe.go:107] details filled by mount probe
I0730 01:46:07.858211       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.858231       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.858238       8 addhandler.go:51] device: /dev/vdb1 does not exist in cache, the device is now connected to this node
I0730 01:46:07.858271       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 01:46:07.858301       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 01:46:07.858321       8 eventhandler.go:73] Processed details for /dev/vdb1
E0730 01:46:07.858331       8 addhandler.go:343] unable to find parent device for /dev/vdb1
E0730 01:46:07.858347       8 addhandler.go:68] error handling unmanaged device /dev/vdb1. error: error in getting parent device for /dev/vdb1 from device hierarchy
E0730 01:46:07.858355       8 eventhandler.go:84] error in getting parent device for /dev/vdb1 from device hierarchy
E0730 01:46:07.858416       8 udevprobe.go:135] Scan is in progress
I0730 01:46:07.861729       8 eventhandler.go:60] Processing details for /dev/sr0
I0730 01:46:07.861997       8 udevprobe.go:280] device: /dev/sr0, Model: QEMU_DVD-ROM filled by udev probe
I0730 01:46:07.862012       8 udevprobe.go:288] device: /dev/sr0, Serial: QM00004 filled by udev probe
I0730 01:46:07.862016       8 udevprobe.go:296] device: /dev/sr0, IDType: cd filled by udev probe
I0730 01:46:07.862032       8 probe.go:107] details filled by udev probe
I0730 01:46:07.862116       8 sysfsprobe.go:99] blockdevice path: /dev/sr0 capacity :401408 filled by sysfs probe.
I0730 01:46:07.862155       8 sysfsprobe.go:128] blockdevice path: /dev/sr0 logical block size :2048 filled by sysfs probe.
I0730 01:46:07.862180       8 sysfsprobe.go:140] blockdevice path: /dev/sr0 physical block size :2048 filled by sysfs probe.
I0730 01:46:07.862201       8 sysfsprobe.go:152] blockdevice path: /dev/sr0 hardware sector size :2048 filled by sysfs probe.
I0730 01:46:07.862225       8 sysfsprobe.go:163] blockdevice path: /dev/sr0 drive type :HDD filled by sysfs probe.
I0730 01:46:07.862231       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.862273       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.862289       8 probe.go:107] details filled by smart probe
E0730 01:46:07.862573       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.862584       8 probe.go:107] details filled by mount probe
I0730 01:46:07.862591       8 usedbyprobe.go:122] device: /dev/sr0 is not having any zfs partitions
I0730 01:46:07.865274       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.865299       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.865309       8 addhandler.go:51] device: /dev/sr0 does not exist in cache, the device is now connected to this node
I0730 01:46:07.865522       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sr0
I0730 01:46:07.865584       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sr0
I0730 01:46:07.865613       8 filter.go:89] /dev/sr0 ignored by path filter
I0730 01:46:07.865625       8 eventhandler.go:60] Processing details for /dev/vda
I0730 01:46:07.865829       8 probe.go:107] details filled by udev probe
I0730 01:46:07.865920       8 sysfsprobe.go:99] blockdevice path: /dev/vda capacity :107374182400 filled by sysfs probe.
I0730 01:46:07.865960       8 sysfsprobe.go:128] blockdevice path: /dev/vda logical block size :512 filled by sysfs probe.
I0730 01:46:07.865992       8 sysfsprobe.go:140] blockdevice path: /dev/vda physical block size :512 filled by sysfs probe.
I0730 01:46:07.866029       8 sysfsprobe.go:152] blockdevice path: /dev/vda hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.866060       8 sysfsprobe.go:163] blockdevice path: /dev/vda drive type :HDD filled by sysfs probe.
I0730 01:46:07.866069       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.866105       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.866125       8 probe.go:107] details filled by smart probe
E0730 01:46:07.866535       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.866547       8 probe.go:107] details filled by mount probe
I0730 01:46:07.866564       8 usedbyprobe.go:122] device: /dev/vda is not having any zfs partitions
I0730 01:46:07.866591       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.866602       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.866607       8 addhandler.go:51] device: /dev/vda does not exist in cache, the device is now connected to this node
I0730 01:46:07.866618       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda
I0730 01:46:07.866645       8 filter.go:89] /dev/vda ignored by os disk exclude filter
I0730 01:46:07.866655       8 eventhandler.go:60] Processing details for /dev/vda1
I0730 01:46:07.866906       8 probe.go:107] details filled by udev probe
I0730 01:46:07.866980       8 sysfsprobe.go:99] blockdevice path: /dev/vda1 capacity :8388608 filled by sysfs probe.
I0730 01:46:07.867061       8 sysfsprobe.go:128] blockdevice path: /dev/vda1 logical block size :512 filled by sysfs probe.
I0730 01:46:07.867096       8 sysfsprobe.go:140] blockdevice path: /dev/vda1 physical block size :512 filled by sysfs probe.
I0730 01:46:07.867122       8 sysfsprobe.go:152] blockdevice path: /dev/vda1 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.867148       8 sysfsprobe.go:163] blockdevice path: /dev/vda1 drive type :HDD filled by sysfs probe.
I0730 01:46:07.867153       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.867183       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.867197       8 probe.go:107] details filled by smart probe
E0730 01:46:07.867618       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.867634       8 probe.go:107] details filled by mount probe
I0730 01:46:07.881469       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.881487       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.881494       8 addhandler.go:51] device: /dev/vda1 does not exist in cache, the device is now connected to this node
I0730 01:46:07.881525       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda1
I0730 01:46:07.881552       8 filter.go:89] /dev/vda1 ignored by os disk exclude filter
I0730 01:46:07.881568       8 eventhandler.go:60] Processing details for /dev/vda2
I0730 01:46:07.881763       8 probe.go:107] details filled by udev probe
I0730 01:46:07.881860       8 sysfsprobe.go:99] blockdevice path: /dev/vda2 capacity :107364728320 filled by sysfs probe.
I0730 01:46:07.881949       8 sysfsprobe.go:128] blockdevice path: /dev/vda2 logical block size :512 filled by sysfs probe.
I0730 01:46:07.881992       8 sysfsprobe.go:140] blockdevice path: /dev/vda2 physical block size :512 filled by sysfs probe.
I0730 01:46:07.882024       8 sysfsprobe.go:152] blockdevice path: /dev/vda2 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.882062       8 sysfsprobe.go:163] blockdevice path: /dev/vda2 drive type :HDD filled by sysfs probe.
I0730 01:46:07.882073       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.882110       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.882127       8 probe.go:107] details filled by smart probe
I0730 01:46:07.882548       8 probe.go:107] details filled by mount probe
I0730 01:46:07.882614       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.882623       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.882628       8 addhandler.go:51] device: /dev/vda2 does not exist in cache, the device is now connected to this node
I0730 01:46:07.882648       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda2
I0730 01:46:07.882741       8 filter.go:89] /dev/vda2 ignored by os disk exclude filter
I0730 01:46:07.882760       8 eventhandler.go:60] Processing details for /dev/vdb
I0730 01:46:07.883131       8 probe.go:107] details filled by udev probe
I0730 01:46:07.883255       8 sysfsprobe.go:99] blockdevice path: /dev/vdb capacity :1920383410176 filled by sysfs probe.
I0730 01:46:07.883319       8 sysfsprobe.go:128] blockdevice path: /dev/vdb logical block size :512 filled by sysfs probe.
I0730 01:46:07.883382       8 sysfsprobe.go:140] blockdevice path: /dev/vdb physical block size :512 filled by sysfs probe.
I0730 01:46:07.883438       8 sysfsprobe.go:152] blockdevice path: /dev/vdb hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.883486       8 sysfsprobe.go:163] blockdevice path: /dev/vdb drive type :HDD filled by sysfs probe.
I0730 01:46:07.883502       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.883555       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.883583       8 probe.go:107] details filled by smart probe
E0730 01:46:07.884144       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.884161       8 probe.go:107] details filled by mount probe
I0730 01:46:07.884173       8 usedbyprobe.go:122] device: /dev/vdb is not having any zfs partitions
I0730 01:46:07.884551       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.884568       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.884575       8 addhandler.go:51] device: /dev/vdb does not exist in cache, the device is now connected to this node
I0730 01:46:07.884604       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb
I0730 01:46:07.884649       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb
I0730 01:46:07.884681       8 eventhandler.go:73] Processed details for /dev/vdb
I0730 01:46:07.884700       8 addhandler.go:104] checking if device: /dev/vdb can be uniquely identified
I0730 01:46:07.884722       8 addhandler.go:108] device: /dev/vdb cannot be uniquely identified
I0730 01:46:07.884748       8 addhandler.go:111] device: /dev/vdb has holders/partitions. {Parent: Partitions:[/dev/vdb1] Holders:[] Slaves:[]}
I0730 01:46:07.884767       8 eventhandler.go:60] Processing details for /dev/vdb1
I0730 01:46:07.885023       8 probe.go:107] details filled by udev probe
I0730 01:46:07.885157       8 sysfsprobe.go:99] blockdevice path: /dev/vdb1 capacity :1920382344704 filled by sysfs probe.
I0730 01:46:07.885264       8 sysfsprobe.go:128] blockdevice path: /dev/vdb1 logical block size :512 filled by sysfs probe.
I0730 01:46:07.885319       8 sysfsprobe.go:140] blockdevice path: /dev/vdb1 physical block size :512 filled by sysfs probe.
I0730 01:46:07.885371       8 sysfsprobe.go:152] blockdevice path: /dev/vdb1 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.885419       8 sysfsprobe.go:163] blockdevice path: /dev/vdb1 drive type :HDD filled by sysfs probe.
I0730 01:46:07.885436       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.885489       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.885513       8 probe.go:107] details filled by smart probe
E0730 01:46:07.886005       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.886023       8 probe.go:107] details filled by mount probe
I0730 01:46:07.891249       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.891273       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.891282       8 addhandler.go:46] device: /dev/vdb1 already exists in cache, the event was likely generated by a partition table re-read
I0730 01:46:07.891315       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 01:46:07.891388       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 01:46:07.891425       8 devicevalidityfilter.go:122] device: /dev/vdb1 of device-type partition has invalid partition UUID
I0730 01:46:07.891435       8 filter.go:89] /dev/vdb1 ignored by device validity filter
I0730 01:46:07.891443       8 eventhandler.go:60] Processing details for /dev/vdc
I0730 01:46:07.891655       8 probe.go:107] details filled by udev probe
I0730 01:46:07.891737       8 sysfsprobe.go:99] blockdevice path: /dev/vdc capacity :480103981056 filled by sysfs probe.
I0730 01:46:07.891777       8 sysfsprobe.go:128] blockdevice path: /dev/vdc logical block size :512 filled by sysfs probe.
I0730 01:46:07.891810       8 sysfsprobe.go:140] blockdevice path: /dev/vdc physical block size :512 filled by sysfs probe.
I0730 01:46:07.891857       8 sysfsprobe.go:152] blockdevice path: /dev/vdc hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.891889       8 sysfsprobe.go:163] blockdevice path: /dev/vdc drive type :HDD filled by sysfs probe.
I0730 01:46:07.891901       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.891939       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.891955       8 probe.go:107] details filled by smart probe
E0730 01:46:07.892356       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.892370       8 probe.go:107] details filled by mount probe
I0730 01:46:07.892377       8 usedbyprobe.go:122] device: /dev/vdc is not having any zfs partitions
I0730 01:46:07.892568       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.892579       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.892583       8 addhandler.go:51] device: /dev/vdc does not exist in cache, the device is now connected to this node
I0730 01:46:07.892599       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc
I0730 01:46:07.892626       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc
I0730 01:46:07.892649       8 eventhandler.go:73] Processed details for /dev/vdc
I0730 01:46:07.892661       8 addhandler.go:104] checking if device: /dev/vdc can be uniquely identified
I0730 01:46:07.892673       8 addhandler.go:108] device: /dev/vdc cannot be uniquely identified
I0730 01:46:07.892690       8 addhandler.go:111] device: /dev/vdc has holders/partitions. {Parent: Partitions:[/dev/vdc1] Holders:[] Slaves:[]}
I0730 01:46:07.892704       8 eventhandler.go:60] Processing details for /dev/vdc1
I0730 01:46:07.892860       8 probe.go:107] details filled by udev probe
I0730 01:46:07.892940       8 sysfsprobe.go:99] blockdevice path: /dev/vdc1 capacity :480102915584 filled by sysfs probe.
I0730 01:46:07.893013       8 sysfsprobe.go:128] blockdevice path: /dev/vdc1 logical block size :512 filled by sysfs probe.
I0730 01:46:07.893047       8 sysfsprobe.go:140] blockdevice path: /dev/vdc1 physical block size :512 filled by sysfs probe.
I0730 01:46:07.893077       8 sysfsprobe.go:152] blockdevice path: /dev/vdc1 hardware sector size :512 filled by sysfs probe.
I0730 01:46:07.893105       8 sysfsprobe.go:163] blockdevice path: /dev/vdc1 drive type :HDD filled by sysfs probe.
I0730 01:46:07.893116       8 probe.go:107] details filled by sysfs probe
E0730 01:46:07.893148       8 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 01:46:07.893163       8 probe.go:107] details filled by smart probe
E0730 01:46:07.893504       8 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 01:46:07.893516       8 probe.go:107] details filled by mount probe
I0730 01:46:07.897025       8 probe.go:107] details filled by used-by probe
I0730 01:46:07.897041       8 probe.go:107] details filled by Custom Tag Probe
I0730 01:46:07.897047       8 addhandler.go:46] device: /dev/vdc1 already exists in cache, the event was likely generated by a partition table re-read
I0730 01:46:07.897063       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 01:46:07.897088       8 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 01:46:07.897109       8 devicevalidityfilter.go:122] device: /dev/vdc1 of device-type partition has invalid partition UUID
I0730 01:46:07.897117       8 filter.go:89] /dev/vdc1 ignored by device validity filter
  • kubectl logs <ndm daemon pod name> -n openebs - This is from the replacement NDM pod
[entrypoint.sh] launching ndm process.
E0730 13:23:10.121530       7 logs.go:35] unable to set flag, Error: no such flag -logtostderr
I0730 13:23:10.122263       7 commands.go:72] Starting Node Device Manager Daemon...
I0730 13:23:10.122277       7 commands.go:73] Version Tag : 1.5.0
I0730 13:23:10.122281       7 commands.go:74] GitCommit : 973d8052a36f3cef6594eac2cfc3945cb7df2dc8
I0730 13:23:10.122295       7 features.go:122] Feature gate: UseOSDisk, state: disable
I0730 13:23:10.122308       7 features.go:122] Feature gate: GPTBasedUUID, state: enable
I0730 13:23:10.122313       7 features.go:122] Feature gate: APIService, state: disable
I0730 13:23:10.222919       7 request.go:565] Throttling request took 86.731491ms, request: GET:https://10.16.0.1:443/apis/snapshot.storage.k8s.io/v1beta1?timeout=32s
I0730 13:23:10.272996       7 request.go:565] Throttling request took 136.774179ms, request: GET:https://10.16.0.1:443/apis/rbac.authorization.k8s.io/v1?timeout=32s
I0730 13:23:10.323205       7 request.go:565] Throttling request took 187.011486ms, request: GET:https://10.16.0.1:443/apis/volumesnapshot.external-storage.k8s.io/v1?timeout=32s
I0730 13:23:10.373055       7 request.go:565] Throttling request took 236.849787ms, request: GET:https://10.16.0.1:443/apis/rbac.authorization.k8s.io/v1beta1?timeout=32s
I0730 13:23:10.422930       7 request.go:565] Throttling request took 286.719858ms, request: GET:https://10.16.0.1:443/apis/storage.k8s.io/v1?timeout=32s
I0730 13:23:10.472978       7 request.go:565] Throttling request took 336.763277ms, request: GET:https://10.16.0.1:443/apis/storage.k8s.io/v1beta1?timeout=32s
I0730 13:23:10.522968       7 request.go:565] Throttling request took 386.741156ms, request: GET:https://10.16.0.1:443/apis/node.k8s.io/v1beta1?timeout=32s
I0730 13:23:10.573035       7 request.go:565] Throttling request took 436.818425ms, request: GET:https://10.16.0.1:443/apis/coordination.k8s.io/v1beta1?timeout=32s
I0730 13:23:10.622942       7 request.go:565] Throttling request took 486.704296ms, request: GET:https://10.16.0.1:443/apis/discovery.k8s.io/v1beta1?timeout=32s
I0730 13:23:10.728500       7 request.go:565] Throttling request took 94.792653ms, request: GET:https://10.16.0.1:443/apis/autoscaling/v2beta2?timeout=32s
I0730 13:23:10.778485       7 request.go:565] Throttling request took 144.775832ms, request: GET:https://10.16.0.1:443/apis/authorization.k8s.io/v1?timeout=32s
I0730 13:23:10.828492       7 request.go:565] Throttling request took 194.775752ms, request: GET:https://10.16.0.1:443/apis/certificates.k8s.io/v1beta1?timeout=32s
I0730 13:23:10.878402       7 request.go:565] Throttling request took 244.662542ms, request: GET:https://10.16.0.1:443/apis/autoscaling/v1?timeout=32s
I0730 13:23:10.928511       7 request.go:565] Throttling request took 294.790891ms, request: GET:https://10.16.0.1:443/apis/batch/v1?timeout=32s
I0730 13:23:10.978333       7 request.go:565] Throttling request took 344.602582ms, request: GET:https://10.16.0.1:443/apis/networking.k8s.io/v1?timeout=32s
I0730 13:23:11.028454       7 request.go:565] Throttling request took 394.69048ms, request: GET:https://10.16.0.1:443/apis/networking.k8s.io/v1beta1?timeout=32s
I0730 13:23:11.078384       7 request.go:565] Throttling request took 444.625421ms, request: GET:https://10.16.0.1:443/apis/autoscaling/v2beta1?timeout=32s
I0730 13:23:11.128357       7 request.go:565] Throttling request took 494.410012ms, request: GET:https://10.16.0.1:443/apis/storage.k8s.io/v1?timeout=32s
I0730 13:23:11.140380       7 controller.go:252] BlockDevice CRD is available
I0730 13:23:11.147263       7 filter.go:60] registering filters
I0730 13:23:11.147310       7 filter.go:68] configured os disk exclude filter : state enable
E0730 13:23:11.149749       7 osdiskexcludefilter.go:92] unable to configure os disk filter for mountpoint: /etc/hosts, error: could not get device mount attributes, Path/MountPoint not present in mounts file
E0730 13:23:11.150850       7 osdiskexcludefilter.go:92] unable to configure os disk filter for mountpoint: /boot, error: could not get device mount attributes, Path/MountPoint not present in mounts file
E0730 13:23:11.151098       7 osdiskexcludefilter.go:104] unable to configure os disk filter for mountpoint: /, error: could not get device mount attributes, Path/MountPoint not present in mounts file
E0730 13:23:11.151578       7 osdiskexcludefilter.go:104] unable to configure os disk filter for mountpoint: /boot, error: could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.151625       7 filter.go:68] configured vendor filter : state enable
I0730 13:23:11.151650       7 filter.go:68] configured path filter : state enable
I0730 13:23:11.151666       7 filter.go:68] configured device validity filter : state enable
I0730 13:23:11.151688       7 probe.go:65] registering probes
I0730 13:23:11.151712       7 probe.go:85] configured seachest probe : state disable
I0730 13:23:11.151739       7 probe.go:85] configured smart probe : state enable
I0730 13:23:11.151761       7 probe.go:85] configured mount probe : state enable
I0730 13:23:11.151808       7 probe.go:85] configured udev probe : state enable
I0730 13:23:11.151883       7 udevprobe.go:337] starting udev probe listener
I0730 13:23:11.153794       7 udevprobe.go:202] device: /dev/sr0, Serial: QM00004 filled during udev scan
I0730 13:23:11.153817       7 udevprobe.go:214] device: /dev/sr0, FileSystemUUID: 2021-07-29-18-38-06-85 filled during udev scan
I0730 13:23:11.154060       7 udevprobe.go:231] Dependents of /dev/sr0 : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 13:23:11.154087       7 udevprobe.go:241] Device: /dev/sr0 is of type: disk
I0730 13:23:11.154227       7 udevprobe.go:206] device: /dev/vda, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 13:23:11.154473       7 udevprobe.go:231] Dependents of /dev/vda : {Parent: Partitions:[/dev/vda1 /dev/vda2] Holders:[] Slaves:[]}
I0730 13:23:11.154492       7 udevprobe.go:241] Device: /dev/vda is of type: disk
I0730 13:23:11.154642       7 udevprobe.go:206] device: /dev/vda1, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 13:23:11.154657       7 udevprobe.go:210] device: /dev/vda1, PartitionEntryUUID: 32163b98-7a6e-4dce-9bea-cab3f4c38282 filled during udev scan
I0730 13:23:11.154843       7 udevprobe.go:231] Dependents of /dev/vda1 : {Parent:/dev/vda Partitions:[] Holders:[] Slaves:[]}
I0730 13:23:11.154859       7 udevprobe.go:241] Device: /dev/vda1 is of type: partition
I0730 13:23:11.155016       7 udevprobe.go:206] device: /dev/vda2, PartitionTableUUID: 202a081f-2006-40fd-ad20-0cb9f05bc8f9 filled during udev scan
I0730 13:23:11.155030       7 udevprobe.go:210] device: /dev/vda2, PartitionEntryUUID: 9d7814c1-2455-4539-86d9-4966d277ba5f filled during udev scan
I0730 13:23:11.155039       7 udevprobe.go:214] device: /dev/vda2, FileSystemUUID: b0f46417-d251-4b95-9552-06fdde604e43 filled during udev scan
I0730 13:23:11.155204       7 udevprobe.go:231] Dependents of /dev/vda2 : {Parent:/dev/vda Partitions:[] Holders:[] Slaves:[]}
I0730 13:23:11.155219       7 udevprobe.go:241] Device: /dev/vda2 is of type: partition
I0730 13:23:11.155335       7 udevprobe.go:206] device: /dev/vdb, PartitionTableUUID: 5c638168-78c5-4b6f-880e-5de7e4936592 filled during udev scan
I0730 13:23:11.155581       7 udevprobe.go:231] Dependents of /dev/vdb : {Parent: Partitions:[/dev/vdb1] Holders:[] Slaves:[]}
I0730 13:23:11.155598       7 udevprobe.go:241] Device: /dev/vdb is of type: disk
I0730 13:23:11.155736       7 udevprobe.go:206] device: /dev/vdb1, PartitionTableUUID: 5c638168-78c5-4b6f-880e-5de7e4936592 filled during udev scan
I0730 13:23:11.155751       7 udevprobe.go:210] device: /dev/vdb1, PartitionEntryUUID: bf2f8fe1-f2e7-4a85-b063-3c15b675c16b filled during udev scan
I0730 13:23:11.155981       7 udevprobe.go:231] Dependents of /dev/vdb1 : {Parent:/dev/vdb Partitions:[] Holders:[] Slaves:[]}
I0730 13:23:11.155998       7 udevprobe.go:241] Device: /dev/vdb1 is of type: partition
I0730 13:23:11.156110       7 udevprobe.go:206] device: /dev/vdc, PartitionTableUUID: 03a05148-43a1-4c2c-9d0d-d82c28f22d77 filled during udev scan
I0730 13:23:11.156334       7 udevprobe.go:231] Dependents of /dev/vdc : {Parent: Partitions:[/dev/vdc1] Holders:[] Slaves:[]}
I0730 13:23:11.156352       7 udevprobe.go:241] Device: /dev/vdc is of type: disk
I0730 13:23:11.156500       7 udevprobe.go:206] device: /dev/vdc1, PartitionTableUUID: 03a05148-43a1-4c2c-9d0d-d82c28f22d77 filled during udev scan
I0730 13:23:11.156519       7 udevprobe.go:210] device: /dev/vdc1, PartitionEntryUUID: 34ae7e26-c41b-439f-bb64-1bfafe396eae filled during udev scan
I0730 13:23:11.156677       7 udevprobe.go:231] Dependents of /dev/vdc1 : {Parent:/dev/vdc Partitions:[] Holders:[] Slaves:[]}
I0730 13:23:11.156694       7 udevprobe.go:241] Device: /dev/vdc1 is of type: partition
I0730 13:23:11.170248       7 blockdevicestore.go:129] eventcode=ndm.blockdevice.deactivate.success msg=Deactivated blockdevice rname=blockdevice-8861d59f793ea6410f342d44d7b84446
I0730 13:23:11.170327       7 probe.go:85] configured sysfs probe : state enable
I0730 13:23:11.170346       7 probe.go:85] configured used-by probe : state enable
I0730 13:23:11.170359       7 probe.go:85] configured Custom Tag Probe : state enable
I0730 13:23:11.170400       7 sparsefilegenerator.go:145] No sparse file path/size provided. Skip creating sparse files.
I0730 13:23:11.170649       7 controller.go:282] started the controller
I0730 13:23:11.173353       7 eventhandler.go:60] Processing details for /dev/sr0
I0730 13:23:11.173854       7 udevprobe.go:280] device: /dev/sr0, Model: QEMU_DVD-ROM filled by udev probe
I0730 13:23:11.173885       7 udevprobe.go:288] device: /dev/sr0, Serial: QM00004 filled by udev probe
I0730 13:23:11.173900       7 udevprobe.go:296] device: /dev/sr0, IDType: cd filled by udev probe
I0730 13:23:11.173938       7 probe.go:107] details filled by udev probe
I0730 13:23:11.174889       7 sysfsprobe.go:99] blockdevice path: /dev/sr0 capacity :401408 filled by sysfs probe.
I0730 13:23:11.175053       7 sysfsprobe.go:128] blockdevice path: /dev/sr0 logical block size :2048 filled by sysfs probe.
I0730 13:23:11.175116       7 sysfsprobe.go:140] blockdevice path: /dev/sr0 physical block size :2048 filled by sysfs probe.
I0730 13:23:11.175188       7 sysfsprobe.go:152] blockdevice path: /dev/sr0 hardware sector size :2048 filled by sysfs probe.
I0730 13:23:11.175250       7 sysfsprobe.go:163] blockdevice path: /dev/sr0 drive type :HDD filled by sysfs probe.
I0730 13:23:11.175279       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.175689       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.175740       7 probe.go:107] details filled by smart probe
E0730 13:23:11.177223       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.177253       7 probe.go:107] details filled by mount probe
I0730 13:23:11.177274       7 usedbyprobe.go:122] device: /dev/sr0 is not having any zfs partitions
I0730 13:23:11.189349       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.189385       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.189404       7 addhandler.go:51] device: /dev/sr0 does not exist in cache, the device is now connected to this node
I0730 13:23:11.189539       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sr0
I0730 13:23:11.189642       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sr0
I0730 13:23:11.189697       7 filter.go:89] /dev/sr0 ignored by path filter
I0730 13:23:11.189724       7 eventhandler.go:60] Processing details for /dev/vda
I0730 13:23:11.190143       7 probe.go:107] details filled by udev probe
I0730 13:23:11.190321       7 sysfsprobe.go:99] blockdevice path: /dev/vda capacity :107374182400 filled by sysfs probe.
I0730 13:23:11.190398       7 sysfsprobe.go:128] blockdevice path: /dev/vda logical block size :512 filled by sysfs probe.
I0730 13:23:11.190475       7 sysfsprobe.go:140] blockdevice path: /dev/vda physical block size :512 filled by sysfs probe.
I0730 13:23:11.190541       7 sysfsprobe.go:152] blockdevice path: /dev/vda hardware sector size :512 filled by sysfs probe.
I0730 13:23:11.190593       7 sysfsprobe.go:163] blockdevice path: /dev/vda drive type :HDD filled by sysfs probe.
I0730 13:23:11.190606       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.190733       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.190769       7 probe.go:107] details filled by smart probe
E0730 13:23:11.192027       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.192040       7 probe.go:107] details filled by mount probe
I0730 13:23:11.192073       7 usedbyprobe.go:122] device: /dev/vda is not having any zfs partitions
I0730 13:23:11.192119       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.192142       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.192153       7 addhandler.go:51] device: /dev/vda does not exist in cache, the device is now connected to this node
I0730 13:23:11.192207       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda
I0730 13:23:11.192249       7 filter.go:89] /dev/vda ignored by os disk exclude filter
I0730 13:23:11.192259       7 eventhandler.go:60] Processing details for /dev/vda1
I0730 13:23:11.192625       7 probe.go:107] details filled by udev probe
I0730 13:23:11.192769       7 sysfsprobe.go:99] blockdevice path: /dev/vda1 capacity :8388608 filled by sysfs probe.
I0730 13:23:11.192878       7 sysfsprobe.go:128] blockdevice path: /dev/vda1 logical block size :512 filled by sysfs probe.
I0730 13:23:11.192938       7 sysfsprobe.go:140] blockdevice path: /dev/vda1 physical block size :512 filled by sysfs probe.
I0730 13:23:11.192988       7 sysfsprobe.go:152] blockdevice path: /dev/vda1 hardware sector size :512 filled by sysfs probe.
I0730 13:23:11.193035       7 sysfsprobe.go:163] blockdevice path: /dev/vda1 drive type :HDD filled by sysfs probe.
I0730 13:23:11.193049       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.193108       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.193134       7 probe.go:107] details filled by smart probe
E0730 13:23:11.193675       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.193688       7 probe.go:107] details filled by mount probe
I0730 13:23:11.197089       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.197104       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.197108       7 addhandler.go:51] device: /dev/vda1 does not exist in cache, the device is now connected to this node
I0730 13:23:11.197123       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda1
I0730 13:23:11.197141       7 filter.go:89] /dev/vda1 ignored by os disk exclude filter
I0730 13:23:11.197146       7 eventhandler.go:60] Processing details for /dev/vda2
I0730 13:23:11.197293       7 probe.go:107] details filled by udev probe
I0730 13:23:11.197359       7 sysfsprobe.go:99] blockdevice path: /dev/vda2 capacity :107364728320 filled by sysfs probe.
I0730 13:23:11.197402       7 sysfsprobe.go:128] blockdevice path: /dev/vda2 logical block size :512 filled by sysfs probe.
I0730 13:23:11.197420       7 sysfsprobe.go:140] blockdevice path: /dev/vda2 physical block size :512 filled by sysfs probe.
I0730 13:23:11.197483       7 sysfsprobe.go:152] blockdevice path: /dev/vda2 hardware sector size :512 filled by sysfs probe.
I0730 13:23:11.197507       7 sysfsprobe.go:163] blockdevice path: /dev/vda2 drive type :HDD filled by sysfs probe.
I0730 13:23:11.197512       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.197541       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.197550       7 probe.go:107] details filled by smart probe
I0730 13:23:11.198003       7 probe.go:107] details filled by mount probe
I0730 13:23:11.200112       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.200128       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.200134       7 addhandler.go:51] device: /dev/vda2 does not exist in cache, the device is now connected to this node
I0730 13:23:11.200157       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vda2
I0730 13:23:11.200181       7 filter.go:89] /dev/vda2 ignored by os disk exclude filter
I0730 13:23:11.200186       7 eventhandler.go:60] Processing details for /dev/vdb
I0730 13:23:11.200316       7 probe.go:107] details filled by udev probe
I0730 13:23:11.200382       7 sysfsprobe.go:99] blockdevice path: /dev/vdb capacity :1920383410176 filled by sysfs probe.
I0730 13:23:11.200459       7 sysfsprobe.go:128] blockdevice path: /dev/vdb logical block size :512 filled by sysfs probe.
I0730 13:23:11.200486       7 sysfsprobe.go:140] blockdevice path: /dev/vdb physical block size :512 filled by sysfs probe.
I0730 13:23:11.200523       7 sysfsprobe.go:152] blockdevice path: /dev/vdb hardware sector size :512 filled by sysfs probe.
I0730 13:23:11.200550       7 sysfsprobe.go:163] blockdevice path: /dev/vdb drive type :HDD filled by sysfs probe.
I0730 13:23:11.200555       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.200587       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.200602       7 probe.go:107] details filled by smart probe
E0730 13:23:11.201152       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.201162       7 probe.go:107] details filled by mount probe
I0730 13:23:11.201169       7 usedbyprobe.go:122] device: /dev/vdb is not having any zfs partitions
I0730 13:23:11.201466       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.201476       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.201480       7 addhandler.go:51] device: /dev/vdb does not exist in cache, the device is now connected to this node
I0730 13:23:11.201496       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb
I0730 13:23:11.201521       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb
I0730 13:23:11.201545       7 eventhandler.go:73] Processed details for /dev/vdb
I0730 13:23:11.201554       7 addhandler.go:104] checking if device: /dev/vdb can be uniquely identified
I0730 13:23:11.201571       7 addhandler.go:108] device: /dev/vdb cannot be uniquely identified
I0730 13:23:11.201583       7 addhandler.go:111] device: /dev/vdb has holders/partitions. {Parent: Partitions:[/dev/vdb1] Holders:[] Slaves:[]}
I0730 13:23:11.201599       7 eventhandler.go:60] Processing details for /dev/vdb1
I0730 13:23:11.201742       7 probe.go:107] details filled by udev probe
I0730 13:23:11.201800       7 sysfsprobe.go:99] blockdevice path: /dev/vdb1 capacity :1920382344704 filled by sysfs probe.
I0730 13:23:11.201848       7 sysfsprobe.go:128] blockdevice path: /dev/vdb1 logical block size :512 filled by sysfs probe.
I0730 13:23:11.201872       7 sysfsprobe.go:140] blockdevice path: /dev/vdb1 physical block size :512 filled by sysfs probe.
I0730 13:23:11.201891       7 sysfsprobe.go:152] blockdevice path: /dev/vdb1 hardware sector size :512 filled by sysfs probe.
I0730 13:23:11.201919       7 sysfsprobe.go:163] blockdevice path: /dev/vdb1 drive type :HDD filled by sysfs probe.
I0730 13:23:11.201924       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.201950       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.201963       7 probe.go:107] details filled by smart probe
E0730 13:23:11.202429       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.202439       7 probe.go:107] details filled by mount probe
I0730 13:23:11.207150       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.207163       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.207168       7 addhandler.go:51] device: /dev/vdb1 does not exist in cache, the device is now connected to this node
I0730 13:23:11.207183       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 13:23:11.207207       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdb1
I0730 13:23:11.207231       7 eventhandler.go:73] Processed details for /dev/vdb1
I0730 13:23:11.207240       7 addhandler.go:104] checking if device: /dev/vdb1 can be uniquely identified
I0730 13:23:11.207245       7 uuid.go:72] device(/dev/vdb1) is a partition, using partition UUID: bf2f8fe1-f2e7-4a85-b063-3c15b675c16b
I0730 13:23:11.207251       7 uuid.go:91] generated uuid: blockdevice-8861d59f793ea6410f342d44d7b84446 for device: /dev/vdb1
I0730 13:23:11.207258       7 addhandler.go:128] uuid: blockdevice-8861d59f793ea6410f342d44d7b84446 has been generated for device: /dev/vdb1
I0730 13:23:11.212335       7 blockdevicestore.go:144] Got blockdevice object : blockdevice-8861d59f793ea6410f342d44d7b84446
I0730 13:23:11.212363       7 addhandler.go:231] creating resource for device: /dev/vdb1 with uuid: blockdevice-8861d59f793ea6410f342d44d7b84446
I0730 13:23:11.222658       7 blockdevicestore.go:111] eventcode=ndm.blockdevice.update.success msg=Updated blockdevice object rname=blockdevice-8861d59f793ea6410f342d44d7b84446
I0730 13:23:11.222748       7 eventhandler.go:60] Processing details for /dev/vdc
I0730 13:23:11.223047       7 probe.go:107] details filled by udev probe
I0730 13:23:11.223204       7 sysfsprobe.go:99] blockdevice path: /dev/vdc capacity :480103981056 filled by sysfs probe.
I0730 13:23:11.223293       7 sysfsprobe.go:128] blockdevice path: /dev/vdc logical block size :512 filled by sysfs probe.
I0730 13:23:11.223364       7 sysfsprobe.go:140] blockdevice path: /dev/vdc physical block size :512 filled by sysfs probe.
I0730 13:23:11.223451       7 sysfsprobe.go:152] blockdevice path: /dev/vdc hardware sector size :512 filled by sysfs probe.
I0730 13:23:11.223540       7 sysfsprobe.go:163] blockdevice path: /dev/vdc drive type :HDD filled by sysfs probe.
I0730 13:23:11.223601       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.223701       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.223746       7 probe.go:107] details filled by smart probe
E0730 13:23:11.224556       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.224600       7 probe.go:107] details filled by mount probe
I0730 13:23:11.224623       7 usedbyprobe.go:122] device: /dev/vdc is not having any zfs partitions
I0730 13:23:11.225180       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.225229       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.225245       7 addhandler.go:51] device: /dev/vdc does not exist in cache, the device is now connected to this node
I0730 13:23:11.225280       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc
I0730 13:23:11.225348       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc
I0730 13:23:11.225418       7 eventhandler.go:73] Processed details for /dev/vdc
I0730 13:23:11.225448       7 addhandler.go:104] checking if device: /dev/vdc can be uniquely identified
I0730 13:23:11.225491       7 addhandler.go:108] device: /dev/vdc cannot be uniquely identified
I0730 13:23:11.225529       7 addhandler.go:111] device: /dev/vdc has holders/partitions. {Parent: Partitions:[/dev/vdc1] Holders:[] Slaves:[]}
I0730 13:23:11.225586       7 eventhandler.go:60] Processing details for /dev/vdc1
I0730 13:23:11.225781       7 probe.go:107] details filled by udev probe
I0730 13:23:11.225906       7 sysfsprobe.go:99] blockdevice path: /dev/vdc1 capacity :480102915584 filled by sysfs probe.
I0730 13:23:11.226018       7 sysfsprobe.go:128] blockdevice path: /dev/vdc1 logical block size :512 filled by sysfs probe.
I0730 13:23:11.226093       7 sysfsprobe.go:140] blockdevice path: /dev/vdc1 physical block size :512 filled by sysfs probe.
I0730 13:23:11.226175       7 sysfsprobe.go:152] blockdevice path: /dev/vdc1 hardware sector size :512 filled by sysfs probe.
I0730 13:23:11.226250       7 sysfsprobe.go:163] blockdevice path: /dev/vdc1 drive type :HDD filled by sysfs probe.
I0730 13:23:11.226273       7 probe.go:107] details filled by sysfs probe
E0730 13:23:11.226344       7 smartprobe.go:101] map[errorCheckingConditions:the device type is not supported yet, device type: "unknown"]
I0730 13:23:11.226373       7 probe.go:107] details filled by smart probe
E0730 13:23:11.227001       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:23:11.227042       7 probe.go:107] details filled by mount probe
I0730 13:23:11.231328       7 probe.go:107] details filled by used-by probe
I0730 13:23:11.231379       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:23:11.231395       7 addhandler.go:51] device: /dev/vdc1 does not exist in cache, the device is now connected to this node
I0730 13:23:11.231432       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 13:23:11.231470       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/vdc1
I0730 13:23:11.231510       7 eventhandler.go:73] Processed details for /dev/vdc1
I0730 13:23:11.231532       7 addhandler.go:104] checking if device: /dev/vdc1 can be uniquely identified
I0730 13:23:11.231547       7 uuid.go:72] device(/dev/vdc1) is a partition, using partition UUID: 34ae7e26-c41b-439f-bb64-1bfafe396eae
I0730 13:23:11.231568       7 uuid.go:91] generated uuid: blockdevice-d3c7cd981e7b3b1712816ff4a6b6f892 for device: /dev/vdc1
I0730 13:23:11.231604       7 addhandler.go:128] uuid: blockdevice-d3c7cd981e7b3b1712816ff4a6b6f892 has been generated for device: /dev/vdc1
E0730 13:23:11.235448       7 blockdevicestore.go:141] Unable to get blockdevice object : blockdevices.openebs.io "blockdevice-d3c7cd981e7b3b1712816ff4a6b6f892" not found
I0730 13:23:11.235491       7 addhandler.go:132] device: /dev/vdc1, uuid: blockdevice-d3c7cd981e7b3b1712816ff4a6b6f892 not found in etcd
I0730 13:23:11.235509       7 addhandler.go:140] device: /dev/vdc1 is partition
I0730 13:23:11.235534       7 addhandler.go:141] checking if device has a parent
I0730 13:23:11.235559       7 addhandler.go:149] parent device: /dev/vdc found for device: /dev/vdc1
I0730 13:23:11.235581       7 addhandler.go:150] checking if parent device can be uniquely identified
I0730 13:23:11.235606       7 addhandler.go:153] unable to generate UUID for parent device, may be a device without WWN
I0730 13:23:11.235621       7 addhandler.go:256] creating block device resource for device: /dev/vdc1 with uuid: blockdevice-d3c7cd981e7b3b1712816ff4a6b6f892
I0730 13:23:11.243535       7 blockdevicestore.go:43] eventcode=ndm.blockdevice.create.success msg=Created blockdevice object in etcd rname=blockdevice-d3c7cd981e7b3b1712816ff4a6b6f892
I0730 13:24:42.827377       7 event.go:47] processing new event for (/dev/vdc1) action type change
I0730 13:24:42.827786       7 event.go:84] Dependents of /dev/vdc1 : {Parent:/dev/vdc Partitions:[] Holders:[] Slaves:[]}
I0730 13:24:42.827864       7 event.go:93] Device: /dev/vdc1 is of type: partition
I0730 13:24:42.828454       7 event.go:47] processing new event for (/dev/vdb1) action type change
I0730 13:24:42.829471       7 event.go:84] Dependents of /dev/vdb1 : {Parent:/dev/vdb Partitions:[] Holders:[] Slaves:[]}
I0730 13:24:42.829507       7 event.go:93] Device: /dev/vdb1 is of type: partition
I0730 13:25:23.794742       7 event.go:47] processing new event for (/dev/sda) action type add
I0730 13:25:23.795361       7 event.go:84] Dependents of /dev/sda : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 13:25:23.795405       7 event.go:93] Device: /dev/sda is of type: disk
I0730 13:25:23.810257       7 eventhandler.go:60] Processing details for /dev/sda
I0730 13:25:23.810663       7 udevprobe.go:280] device: /dev/sda, Model: iscsi filled by udev probe
I0730 13:25:23.810740       7 udevprobe.go:284] device: /dev/sda, WWN: 0x3000000079e4dc18 filled by udev probe
I0730 13:25:23.810755       7 udevprobe.go:288] device: /dev/sda, Serial: 3000000079e4dc18 filled by udev probe
I0730 13:25:23.810766       7 udevprobe.go:292] device: /dev/sda, Vendor: OpenEBS filled by udev probe
I0730 13:25:23.810780       7 udevprobe.go:296] device: /dev/sda, IDType: disk filled by udev probe
I0730 13:25:23.810807       7 probe.go:107] details filled by udev probe
I0730 13:25:23.810962       7 sysfsprobe.go:99] blockdevice path: /dev/sda capacity :107374182400 filled by sysfs probe.
I0730 13:25:23.811038       7 sysfsprobe.go:128] blockdevice path: /dev/sda logical block size :512 filled by sysfs probe.
I0730 13:25:23.811099       7 sysfsprobe.go:140] blockdevice path: /dev/sda physical block size :4096 filled by sysfs probe.
I0730 13:25:23.811153       7 sysfsprobe.go:152] blockdevice path: /dev/sda hardware sector size :512 filled by sysfs probe.
I0730 13:25:23.811214       7 sysfsprobe.go:163] blockdevice path: /dev/sda drive type :HDD filled by sysfs probe.
I0730 13:25:23.811232       7 probe.go:107] details filled by sysfs probe
I0730 13:25:23.813278       7 probe.go:107] details filled by smart probe
E0730 13:25:23.815259       7 mountprobe.go:100] could not get device mount attributes, Path/MountPoint not present in mounts file
I0730 13:25:23.815298       7 probe.go:107] details filled by mount probe
I0730 13:25:23.815319       7 usedbyprobe.go:122] device: /dev/sda is not having any zfs partitions
I0730 13:25:23.825645       7 probe.go:107] details filled by used-by probe
I0730 13:25:23.825670       7 probe.go:107] details filled by Custom Tag Probe
I0730 13:25:23.825679       7 addhandler.go:51] device: /dev/sda does not exist in cache, the device is now connected to this node
I0730 13:25:23.825718       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sda
I0730 13:25:23.825763       7 osdiskexcludefilter.go:131] applying os-filter regex ^/dev/vda[0-9]*$ on /dev/sda
I0730 13:25:23.825795       7 filter.go:89] /dev/sda ignored by vendor filter
I0730 13:25:23.861978       7 event.go:47] processing new event for (/dev/sda) action type change
I0730 13:25:23.862246       7 event.go:84] Dependents of /dev/sda : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 13:25:23.862272       7 event.go:93] Device: /dev/sda is of type: disk
I0730 13:25:36.661028       7 event.go:47] processing new event for (/dev/sda) action type change
I0730 13:25:36.661592       7 event.go:84] Dependents of /dev/sda : {Parent: Partitions:[] Holders:[] Slaves:[]}
I0730 13:25:36.661647       7 event.go:93] Device: /dev/sda is of type: disk
  • lsblk from nodes where ndm daemonset is running
k8s-worker1-bobloblaw-mercury01-bgth:/home/mercury # lsblk
NAME   MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
sr0     11:0    1   392K  0 rom  
vda    254:0    0   100G  0 disk 
├─vda1 254:1    0     8M  0 part 
└─vda2 254:2    0   100G  0 part /
vdb    254:16   0   1.8T  0 disk 
└─vdb1 254:17   0   1.8T  0 part 
vdc    254:32   0 447.1G  0 disk 
└─vdc1 254:33   0 447.1G  0 part

Anything else you would like to add:
This does not happen very often, and restarting the NDM pod has fixed it each time. Our startup scripts make sure that the block devices are zeroed out prior to starting things, through a combination of wipefs, sgdisk --zap-all and using dd to clear the beginning and tail of the disk.

Environment:

  • OpenEBS version: 2.10, openebs/node-disk-manager:1.5.0
  • Kubernetes version (use kubectl version):
Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:58:53Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:51:04Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
  • Kubernetes installer & version:
kubeadm version: &version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.6", GitCommit:"dff82dc0de47299ab66c83c626e08b245ab19037", GitTreeState:"clean", BuildDate:"2020-07-15T16:56:34Z", GoVersion:"go1.13.9", Compiler:"gc", Platform:"linux/amd64"}
  • Cloud provider or hardware configuration:
    Baremetal, k8s nodes are running in VMs.
  • Type of disks connected to the nodes (eg: Virtual Disks, GCE/EBS Volumes, Physical drives etc)
    SSDs passed into the VM as virtual disks:
 Target   Source
-------------------------------------------------------------------------
 vdb      /dev/disk/by-id/ata-SAMSUNG_MZ7LH1T9HMLT-00005_S455NY0MB41995
 vdc      /dev/disk/by-id/ata-SAMSUNG_MZ7LH480HAHQ-00005_S45PNA0MC37492
  • OS (e.g. from /etc/os-release):
    openSUSE Leap 15.1
@akhilerm akhilerm added enhancement feature request release-note/open issue still open / being worked in a release labels Aug 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement feature request release-note/open issue still open / being worked in a release
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@akhilerm @dborman-hpe