Volumesnapshots component missing?

[ishioni]

I wanted to roll out velero for backups which seems to require volumesnapshots

However, if I try to test it out with an example volumesnapshot from the link above I get

no matches for kind "VolumeSnapshot" in version "snapshot.storage.k8s.io/v1beta1"

This is on k3s 1.23.12. I see in your e2e test you have a volumesnapshotclass, and I see democratic-csi deploys the snapshot-controller as a component for this to work. I know this is not strictly truenas-csp related, but which component am I missing for snapshots here?

[datamattsson]

Where did you find the example VolumeSnapshot stanza? You need to call out "snapshot.storage.k8s.io/v1" since Kubernetes 1.20 (well, I'm not sure when "v1beta1" got fully removed). Check example here: https://kubernetes.io/docs/concepts/storage/volume-snapshots/#volumesnapshots

(I also noticed my error in the e2e test suite that calls out the wrong API version)

Anyhow, it's supposed to be up to the Kubernetes distribution vendor to install the external snapshot controller. Not many distributions actually provide this. So, if you want to enable volume snapshots for the TrueNAS CSP you may follow the instructions found with the HPE CSI Driver.

[ishioni]

Where did you find the example VolumeSnapshot stanza?

Exactly the same link from the one you posted :)

So, if you want to enable volume snapshots for the TrueNAS CSP you may follow the instructions found with the HPE CSI Driver.

Ah, so as I suspected, I need external-snapshot-controller. Could you maybe link that info in the Readme somewhere?

Thanks for the support!

[datamattsson]

Thanks for the suggestion. I'll have a look to have this outlined on the INSTALL to ensure there are pointers to enable and use VolumeSnapshots.

[ishioni]

@datamattsson I'm playing around with this now, snapshot-controller in tow. However, I think that's not how it's supposed to work

Consider this scenario
I create a pvc - my-pv
I create a volumesnapshot out of it - my-pv-snapshot -
I delete the original pvc
I create a new pvc out of the snapshot - It now references (in my case) SSD_k3s_pvc-f72e0ff5-33cf-40fb-b488-6b960928f53b@snapshot-2f5d1c03-9dba-448b-b426-c71ac2f1c3f9, which is a zfs snapshot of a zvol that doesn't exist anymore, and subsequently fails

Hence all snapshots are bound to a backing zvol. If that zvol were to be deleted by for example killing the underlying pv, then those snapshots are dead, providing no protection

[datamattsson]

Would we like zvols which has snapshots created by the CSI driver to be retained on the backend? I'm not sure why PVs with snapshotcontent derived from them aren't protected. It's odd.

[ishioni]

Yeah, it's a quirk of how snapshots are handled by zfs, where the snapshots are bound to the originating zvol/dataset. Democratic CSI has an option to clone the zvol/dataset outright.

That being said, you should know i've moved on to using nvmeof, so if you do change the handling I won't be able to test it out

[datamattsson]

Democratic CSI has an option to clone the zvol/dataset outright.

Interesting, like doing a send/recv asynchronously within the same pool?

[datamattsson]

Add hints to INSTALL.md.