The {pve} storage model is very flexible. Virtual machine images can either be stored on one or several local storages, or on shared storage like NFS or iSCSI (NAS, SAN). There are no limits, and you may configure as many storage pools as you like. You can use all storage technologies available for Debian Linux.
One major benefit of storing VMs on shared storage is the ability to live-migrate running machines without any downtime, as all nodes in the cluster have direct access to VM disk images. There is no need to copy VM image data, so live migration is very fast in that case.
The storage library (package libpve-storage-perl) uses a flexible
plugin system to provide a common interface to all storage types. This
can be easily adopted to include further storage types in the future.
There are basically two different classes of storage types:
- File level storage
-
File level based storage technologies allow access to a fully featured (POSIX) file system. They are in general more flexible than any Block level storage (see below), and allow you to store content of any type. ZFS is probably the most advanced system, and it has full support for snapshots and clones.
- Block level storage
-
Allows to store large raw images. It is usually not possible to store other files (ISO, backups, ..) on such storage types. Most modern block level storage implementations support snapshots and clones. RADOS and GlusterFS are distributed systems, replicating storage data to different nodes.
| Description | PVE type | Level | Shared | Snapshots | Stable |
|---|---|---|---|---|---|
ZFS (local) |
|
file |
no |
yes |
yes |
Directory |
|
file |
no |
no1 |
yes |
NFS |
|
file |
yes |
no1 |
yes |
CIFS |
|
file |
yes |
no1 |
yes |
GlusterFS |
|
file |
yes |
no1 |
yes |
CephFS |
|
file |
yes |
yes |
yes |
LVM |
|
block |
no2 |
no |
yes |
LVM-thin |
|
block |
no |
yes |
yes |
iSCSI/kernel |
|
block |
yes |
no |
yes |
iSCSI/libiscsi |
|
block |
yes |
no |
yes |
Ceph/RBD |
|
block |
yes |
yes |
yes |
ZFS over iSCSI |
|
block |
yes |
yes |
yes |