KVM incremental snapshot feature #9270
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #9270 +/- ##
============================================
- Coverage 15.80% 4.04% -11.77%
============================================
Files 5627 392 -5235
Lines 492341 32175 -460166
Branches 59693 5678 -54015
============================================
- Hits 77827 1301 -76526
+ Misses 405990 30726 -375264
+ Partials 8524 148 -8376
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
|
Good job @JoaoJandre |
second that, tnx |
core/src/main/java/com/cloud/storage/resource/StorageSubsystemCommandHandlerBase.java
Outdated
Show resolved
Hide resolved
|
not gotten through all of it yet but looks good so far. |
|
@blueorangutan package |
|
@DaanHoogland a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el7 ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 10011 |
|
@blueorangutan test |
|
@DaanHoogland a [SL] Trillian-Jenkins test job (centos7 mgmt + kvm-centos7) has been kicked to run smoke tests |
|
[SF] Trillian test result (tid-10507)
|
|
@blueorangutan test rocky8 kvm-rocky8 |
|
@weizhouapache a [SL] Trillian-Jenkins test job (rocky8 mgmt + kvm-rocky8) has been kicked to run smoke tests |
|
@JoaoJandre nice one. Just one remark, the following sentence sounds contradictory to me "Additionally, this functionality is only available when using file based storage, such as shared mount-point (iSCSI and FC)", if it supports iSCSI and FC (through a shared mountpoint) it does support block storage, I think the phrasing could cause some confusion as to which types of storage are supported. |
Hey @alexandremattioli, I understand your confusion. However, when using shared mount-point, as far as ACS is concerned, the storage is file-based, we will not be working with blocks directly, only files (as ACS does already for shared mount point). The mentions on parenthesis are there to give an example of underlying storages that might be behind the shared mount-point. I have updated the description to add a little more context. |
|
[SF] Trillian test result (tid-10523)
|
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
|
@slavkap I have addressed all of your comments. Could you take another look at the PR? |
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
|
@blueorangutan package |
|
@JoaoJandre a [SL] Jenkins job has been kicked to build packages. It will be bundled with KVM, XenServer and VMware SystemVM templates. I'll keep you posted as I make progress. |
|
Packaging result [SF]: ✔️ el8 ✔️ el9 ✔️ debian ✔️ suse15. SL-JID 11532 |
|
@DaanHoogland could we run the CI here? |
|
@blueorangutan test |
|
@shwstppr a [SL] Trillian-Jenkins test job (ol8 mgmt + kvm-ol8) has been kicked to run smoke tests |
|
[SF] Trillian test result (tid-11768)
|
|
This pull request has merge conflicts. Dear author, please fix the conflicts and sync your branch with the base branch. |
|
@slavkap @winterhazel there have been multiple changes since you last reviewed, could you take another look to see if everything looks good to you? |
Description
This PR solves issue #8907.
Currently, when taking a volume snapshot/backup with KVM as the hypervisor, it is always a full snapshot/backup. However, always taking full snapshots of volumes is costly for both the storage network and storage systems. To solve the aforementioned issues, this PR extends the volume snapshot feature in KVM, allowing users to create incremental volume snapshots using KVM as a hypervisor.
To give operators control over which type of snapshot is being created, a new global setting
kvm.incremental.snapshothas been added, which can be changed at zone and cluster scopes; this setting is false by default. Also, thesnapshot.delta.maxconfiguration, used to control the maximum deltas when using XenServer, was extended to also limit the size of the backing chain of snapshots on primary/secondary storage.This functionality is only available in environments with Libvirt 7.6.0+ and qemu 6.1+. If the
kvm.incremental.snapshotsetting is true, and the hosts do not have the required Libvirt and qemu versions, an error will be thrown when trying to take a snapshot. Additionally, this functionality is only available when using file based storage, such as shared mount-point (iSCSI and FC that require a shared mount-point storage file system for KVM such as OCFS2 or GlusterFS), NFS, and local storage. Other storage types for KVM, such as CLVM and RBD, need different approaches to enable incremental backups; therefore, these are not currently supported.Issue #8907 has more details and flowcharts of all the mapped workflows.
Types of changes
Feature/Enhancement Scale or Bug Severity
Feature/Enhancement Scale
Bug Severity
Screenshots (if appropriate):
How Has This Been Tested?
Description of tests
During testing, the
kvm.incremental.snapshotsetting was changed totrueand thesnapshot.delta.maxsetting was changed to 3.Tests with snapshot.backup.to.secondary = false
For the tests in this section, a test VM was created and reused for all tests.
Snapshot creation tests
Snapshot restore tests
Snapshot removal tests
Template creation test
Tests with snapshot.backup.to.secondary = true
All tests performed in the previous sections were repeated with snapshot.backup.to.secondary = false, in addition, two additional tests were performed. For the tests in this section, a test VM was created and reused for all tests.
Snapshot creation tests
I have also tested that the bitmaps are removed once the snapshots are deleted.