Merge branch 'vmware-tanzu:main' into main

kaovilai · Oct 3, 2022 · a15bd15 · a15bd15 · vercel · Oct 3, 2022
2 parents aa26702 + f3e3cfc
commit a15bd15
Show file tree

Hide file tree

Showing 379 changed files with 18,032 additions and 6,061 deletions.
diff --git a/ADOPTERS.md b/ADOPTERS.md
@@ -40,7 +40,9 @@ We have integrated our [solution with Velero][11] to provide our customers with
 Kyma [integrates with Velero][41] to effortlessly back up and restore Kyma clusters with all its resources. Velero capabilities allow Kyma users to define and run manual and scheduled backups in order to successfully handle a disaster-recovery scenario.
 
 **[Red Hat][50]**  
-Red Hat has developed the [Cluster Application Migration Tool][51] which uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
+Red Hat has developed 2 operators for the OpenShift platform:
+- [Migration Toolkit for Containers][51] (Crane): This operator uses [Velero and Restic][52] to drive the migration of applications between OpenShift clusters.
+- [OADP (OpenShift API for Data Protection) Operator][53]: This operator sets up and installs Velero on the OpenShift platform, allowing users to backup and restore applications.
 
 **[Dell EMC][70]**  
 For Kubernetes environments, [PowerProtect Data Manager][71] leverages the Container Storage Interface (CSI) framework to take snapshots to back up the persistent data or the data that the application creates e.g. databases. [Dell EMC leverages Velero][72] to backup the namespace configuration files (also known as Namespace meta data) for enterprise grade data protection.
@@ -89,6 +91,7 @@ If you would like to add your logo to a future `Adopters of Velero` section on [
 [50]: https://redhat.com
 [51]: https://github.com/fusor/mig-operator
 [52]: https://github.com/fusor/mig-operator/blob/master/docs/usage/2.md
+[53]: https://github.com/openshift/oadp-operator
 
 [60]: https://banzaicloud.com
 [61]: https://banzaicloud.com/products/pipeline/

diff --git a/changelogs/unreleased/4864-sseago b/changelogs/unreleased/4864-sseago
@@ -0,0 +1 @@
+Add credentials to volume snapshot locations
diff --git a/changelogs/unreleased/4943-phuongatemc b/changelogs/unreleased/4943-phuongatemc
@@ -0,0 +1 @@
+Refactor BackupItemAction proto and related code to backupitemaction/v1 package.  This is part of implementation of the plugin version design https://github.com/vmware-tanzu/velero/blob/main/design/plugin-versioning.md
diff --git a/changelogs/unreleased/5064-jxun b/changelogs/unreleased/5064-jxun
@@ -0,0 +1 @@
+Exclude "csinodes.storage.k8s.io" and "volumeattachments.storage.k8s.io" from restore by default.
diff --git a/changelogs/unreleased/5214-qiuming-best b/changelogs/unreleased/5214-qiuming-best
@@ -0,0 +1 @@
+Uploader Implementation: Restic backup and restore
diff --git a/changelogs/unreleased/5215-allenxu404 b/changelogs/unreleased/5215-allenxu404
@@ -0,0 +1 @@
+Refactor GCController with kubebuilder
diff --git a/changelogs/unreleased/5217-lilongfeng0902 b/changelogs/unreleased/5217-lilongfeng0902
@@ -0,0 +1 @@
+check vsc null pointer
diff --git a/changelogs/unreleased/5218-jxun b/changelogs/unreleased/5218-jxun
@@ -0,0 +1 @@
+Migrate backup sync controller from code-generator to kubebuilder.
diff --git a/changelogs/unreleased/5221-qiuming-best b/changelogs/unreleased/5221-qiuming-best
@@ -0,0 +1 @@
+Uploader Implementation: Kopia backup and restore
diff --git a/changelogs/unreleased/5231-lyndon b/changelogs/unreleased/5231-lyndon
@@ -0,0 +1 @@
+Add changes for Kopia Integration: Kopia Lib - initialize Kopia repo
diff --git a/changelogs/unreleased/5233-lyndon b/changelogs/unreleased/5233-lyndon
@@ -0,0 +1,2 @@
+Add changes for Kopia Integration: Kopia Lib - method implementation
+Add changes to write Kopia Repository logs to Velero log
diff --git a/changelogs/unreleased/5234-reasonerjt b/changelogs/unreleased/5234-reasonerjt
@@ -0,0 +1 @@
+Remove reference to non-existent doc
diff --git a/changelogs/unreleased/5236-sseago b/changelogs/unreleased/5236-sseago
@@ -0,0 +1 @@
+Check for empty ns list before checking nslist[0]
diff --git a/changelogs/unreleased/5239-shubham-pampattiwar b/changelogs/unreleased/5239-shubham-pampattiwar
@@ -0,0 +1 @@
+Fix edge cases for already exists resources
diff --git a/changelogs/unreleased/5241-jxun b/changelogs/unreleased/5241-jxun
@@ -0,0 +1 @@
+Controller refactor code modifications.
diff --git a/changelogs/unreleased/5247-divolgin b/changelogs/unreleased/5247-divolgin
@@ -0,0 +1 @@
+Fix nil pointer panic when restoring StatefulSets
diff --git a/changelogs/unreleased/5248-allenxu404 b/changelogs/unreleased/5248-allenxu404
@@ -0,0 +1 @@
+equip gc controller with configurable frequency
diff --git a/changelogs/unreleased/5252-cleverhu b/changelogs/unreleased/5252-cleverhu
@@ -0,0 +1 @@
+Add csiSnapshotTimeout for describe backup
diff --git a/changelogs/unreleased/5259-lyndon b/changelogs/unreleased/5259-lyndon
@@ -0,0 +1 @@
+Fill gaps for Kopia path of PVBR: integrate Repo Manager with Unified Repo; pass UploaderType to PVBR backupper and restorer; pass RepositoryType to BackupRepository controller and Repo Ensurer
diff --git a/changelogs/unreleased/5263-sseago b/changelogs/unreleased/5263-sseago
@@ -0,0 +1 @@
+Don't move velero v1 plugins to new proto dir
diff --git a/changelogs/unreleased/5271-sseago b/changelogs/unreleased/5271-sseago
@@ -0,0 +1 @@
+ plugin/clientmgmt refactoring for BackupItemAction v1
diff --git a/changelogs/unreleased/5279-ywk253100 b/changelogs/unreleased/5279-ywk253100
@@ -0,0 +1 @@
+Support pause/unpause schedules
diff --git a/changelogs/unreleased/5282-lyndon b/changelogs/unreleased/5282-lyndon
@@ -0,0 +1 @@
+Add changes for problems/enhancements found during smoking test for Kopia pod volume backup/restore
diff --git a/changelogs/unreleased/5283-blackpiglet b/changelogs/unreleased/5283-blackpiglet
@@ -0,0 +1 @@
+Add backup status checking in schedule controller.
diff --git a/changelogs/unreleased/5291-cleverhu b/changelogs/unreleased/5291-cleverhu
@@ -0,0 +1 @@
+Add more detailed comments for labels columns.
diff --git a/changelogs/unreleased/5293-cleverhu b/changelogs/unreleased/5293-cleverhu
@@ -0,0 +1 @@
+Optimize code for restore exists resources.
diff --git a/changelogs/unreleased/5297-blackpiglet b/changelogs/unreleased/5297-blackpiglet
@@ -0,0 +1 @@
+Remove github.com/apex/log logger.
diff --git a/changelogs/unreleased/5299-jxun b/changelogs/unreleased/5299-jxun
@@ -0,0 +1 @@
+Remove snapshot related lister, informer and client from backup controller.
diff --git a/changelogs/unreleased/5308-lyndon b/changelogs/unreleased/5308-lyndon
@@ -0,0 +1 @@
+Refactor the repoEnsurer code to use controller runtime client and wrap some common BackupRepository operations to share with other modules
diff --git a/changelogs/unreleased/5312-sseago b/changelogs/unreleased/5312-sseago
@@ -0,0 +1 @@
+RestoreItemAction v1 refactoring for plugin api versioning
diff --git a/changelogs/unreleased/5314-allenxu404 b/changelogs/unreleased/5314-allenxu404
@@ -0,0 +1 @@
+Change the status of restore to completed from partially failed when restore empty backup
diff --git a/changelogs/unreleased/5318-sseago b/changelogs/unreleased/5318-sseago
@@ -0,0 +1 @@
+plugin versioning v1 refactor for VolumeSnapshotter
diff --git a/changelogs/unreleased/5319-lyndon b/changelogs/unreleased/5319-lyndon
@@ -0,0 +1 @@
+Fix issue 4874 and 4752: check the daemonset pod is running in the node where the workload pod resides before running the PVB for the pod
diff --git a/changelogs/unreleased/5322-qiuming-best b/changelogs/unreleased/5322-qiuming-best
@@ -0,0 +1 @@
+Fix PVB finds wrong parent snapshot
diff --git a/changelogs/unreleased/5329-kaovilai b/changelogs/unreleased/5329-kaovilai
@@ -0,0 +1 @@
+Cancel downloadRequest when timeout without downloadURL
diff --git a/changelogs/unreleased/5331-danfengliu b/changelogs/unreleased/5331-danfengliu
@@ -0,0 +1 @@
+Add opt-in and opt-out PersistentVolume backup to E2E tests
diff --git a/changelogs/unreleased/5347-qiuming-best b/changelogs/unreleased/5347-qiuming-best
@@ -0,0 +1 @@
+Fix restore cmd extraflag overwrite bug
diff --git a/changelogs/unreleased/5350-blackpiglet b/changelogs/unreleased/5350-blackpiglet
@@ -0,0 +1 @@
+Clarify the help message for the default value of parameter --snapshot-volumes, when it's not set.
diff --git a/changelogs/unreleased/5353-cleverhu b/changelogs/unreleased/5353-cleverhu
@@ -0,0 +1 @@
+Add useOwnerReferencesInBackup field doc for schedule.
diff --git a/changelogs/unreleased/5355-danfengliu b/changelogs/unreleased/5355-danfengliu
@@ -0,0 +1 @@
+Add E2E test for schedule backup
diff --git a/changelogs/unreleased/5359-lyndon b/changelogs/unreleased/5359-lyndon
@@ -0,0 +1 @@
+Fix a repoEnsurer problem introduced by the refactor - The repoEnsurer didn't check "" state of BackupRepository, as a result, the function GetBackupRepository always returns without an error even though the ensreReady is specified.
diff --git a/changelogs/unreleased/5362-niulechuan b/changelogs/unreleased/5362-niulechuan
@@ -0,0 +1 @@
+Added backupController's UT to test the prepareBackupRequest() method BackupStorageLocation processing logic
diff --git a/changelogs/unreleased/5370-lyndon b/changelogs/unreleased/5370-lyndon
@@ -0,0 +1 @@
+Pod Volume Backup/Restore Refactor: Rename parameters in CRDs and commands to remove "Restic" word
diff --git a/changelogs/unreleased/5377-qiuming-best b/changelogs/unreleased/5377-qiuming-best
@@ -0,0 +1 @@
+Fix restore error with flag namespace-mappings
diff --git a/changelogs/unreleased/5387-lyndon b/changelogs/unreleased/5387-lyndon
@@ -0,0 +1 @@
+Fix issue 5386: Velero providers a full URL as the S3Url while the underlying minio client only accept the host part of the URL as the endpoint and the schema should be specified separately.
diff --git a/changelogs/unreleased/5394-blackpiglet b/changelogs/unreleased/5394-blackpiglet
@@ -0,0 +1 @@
+Fix Test_prepareBackupRequest_BackupStorageLocation UT failure.
diff --git a/changelogs/unreleased/5396-blackpiglet b/changelogs/unreleased/5396-blackpiglet
@@ -0,0 +1 @@
+update velero using klog to version v2.9.0
diff --git a/changelogs/unreleased/5412-allenxu404 b/changelogs/unreleased/5412-allenxu404
@@ -0,0 +1 @@
+Change B/R describe CLI to support Kopia
diff --git a/cmd/velero/velero.go b/cmd/velero/velero.go
@@ -20,7 +20,7 @@ import (
 	"os"
 	"path/filepath"
 
-	"k8s.io/klog"
+	"k8s.io/klog/v2"
 
 	"github.com/vmware-tanzu/velero/pkg/cmd"
 	"github.com/vmware-tanzu/velero/pkg/cmd/velero"

diff --git a/config/crd/v1/bases/velero.io_backups.yaml b/config/crd/v1/bases/velero.io_backups.yaml
@@ -42,9 +42,17 @@ spec:
                   CSI VolumeSnapshot status turns to ReadyToUse during creation, before
                   returning error as timeout. The default value is 10 minute.
                 type: string
+              defaultVolumesToFsBackup:
+                description: DefaultVolumesToFsBackup specifies whether pod volume
+                  file system backup should be used for all volumes by default.
+                nullable: true
+                type: boolean
               defaultVolumesToRestic:
-                description: DefaultVolumesToRestic specifies whether restic should
-                  be used to take a backup of all pod volumes by default.
+                description: "DefaultVolumesToRestic specifies whether restic should
+                  be used to take a backup of all pod volumes by default. \n Deprecated:
+                  this field is no longer used and will be removed entirely in future.
+                  Use DefaultVolumesToFsBackup instead."
+                nullable: true
                 type: boolean
               excludedNamespaces:
                 description: ExcludedNamespaces contains a list of namespaces that

diff --git a/config/crd/v1/bases/velero.io_podvolumerestores.yaml b/config/crd/v1/bases/velero.io_podvolumerestores.yaml
@@ -116,6 +116,10 @@ spec:
               snapshotID:
                 description: SnapshotID is the ID of the volume snapshot to be restored.
                 type: string
+              sourceNamespace:
+                description: SourceNamespace is the original namespace for namaspace
+                  mapping.
+                type: string
               uploaderType:
                 description: UploaderType is the type of the uploader to handle the
                   data transfer.
@@ -133,6 +137,7 @@ spec:
             - pod
             - repoIdentifier
             - snapshotID
+            - sourceNamespace
             - volume
             type: object
           status:

diff --git a/config/crd/v1/bases/velero.io_schedules.yaml b/config/crd/v1/bases/velero.io_schedules.yaml
@@ -32,6 +32,9 @@ spec:
     - jsonPath: .metadata.creationTimestamp
       name: Age
       type: date
+    - jsonPath: .spec.paused
+      name: Paused
+      type: boolean
     name: v1
     schema:
       openAPIV3Schema:
@@ -53,6 +56,9 @@ spec:
           spec:
             description: ScheduleSpec defines the specification for a Velero schedule
             properties:
+              paused:
+                description: Paused specifies whether the schedule is paused or not
+                type: boolean
               schedule:
                 description: Schedule is a Cron expression defining when to run the
                   Backup.
@@ -66,9 +72,17 @@ spec:
                       for CSI VolumeSnapshot status turns to ReadyToUse during creation,
                       before returning error as timeout. The default value is 10 minute.
                     type: string
+                  defaultVolumesToFsBackup:
+                    description: DefaultVolumesToFsBackup specifies whether pod volume
+                      file system backup should be used for all volumes by default.
+                    nullable: true
+                    type: boolean
                   defaultVolumesToRestic:
-                    description: DefaultVolumesToRestic specifies whether restic should
-                      be used to take a backup of all pod volumes by default.
+                    description: "DefaultVolumesToRestic specifies whether restic
+                      should be used to take a backup of all pod volumes by default.
+                      \n Deprecated: this field is no longer used and will be removed
+                      entirely in future. Use DefaultVolumesToFsBackup instead."
+                    nullable: true
                     type: boolean
                   excludedNamespaces:
                     description: ExcludedNamespaces contains a list of namespaces

diff --git a/config/crd/v1/bases/velero.io_volumesnapshotlocations.yaml b/config/crd/v1/bases/velero.io_volumesnapshotlocations.yaml
@@ -45,6 +45,24 @@ spec:
                   type: string
                 description: Config is for provider-specific configuration fields.
                 type: object
+              credential:
+                description: Credential contains the credential information intended
+                  to be used with this location
+                properties:
+                  key:
+                    description: The key of the secret to select from.  Must be a
+                      valid secret key.
+                    type: string
+                  name:
+                    description: 'Name of the referent. More info: https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
+                      TODO: Add other useful fields. apiVersion, kind, uid?'
+                    type: string
+                  optional:
+                    description: Specify whether the Secret or its key must be defined
+                    type: boolean
+                required:
+                - key
+                type: object
               provider:
                 description: Provider is the provider of the volume storage.
                 type: string
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Refactor BackupItemAction proto and related code to backupitemaction/v1 package. This is part of implementation of the plugin version design https://github.com/vmware-tanzu/velero/blob/main/design/plugin-versioning.md
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Exclude "csinodes.storage.k8s.io" and "volumeattachments.storage.k8s.io" from restore by default.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Uploader Implementation: Restic backup and restore
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Migrate backup sync controller from code-generator to kubebuilder.
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Uploader Implementation: Kopia backup and restore
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Add changes for Kopia Integration: Kopia Lib - initialize Kopia repo
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1,2 @@
		Add changes for Kopia Integration: Kopia Lib - method implementation
		Add changes to write Kopia Repository logs to Velero log
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Check for empty ns list before checking nslist[0]
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		Fix nil pointer panic when restoring StatefulSets
Original file line number	Diff line number	Diff line change
		@@ -0,0 +1 @@
		equip gc controller with configurable frequency