Tenants cannot be restored from velero, k8up, restic PVC backups because chart does not accept an existing claim

[cloudymax]

The documented process for restoring Minio from Valero, Resit, and K8up is to create a PVC, copy backups to the PVC, then specify that PVC as an existing claim for Minio.

The ability to specify an existing claim was removed when migrating from the vanilla chart to the new operator/tenant.

reference: https://github.com/minio/minio/blob/22d59e757da6e83ccb954fc6abc094375d73d444/helm/minio/values.yaml#L148

Expected Behavior

I would expect to be able to specify an existing claim as is possible in the vanilla chart and use the tools Mino member harshavardhana says are the endorsed ways to do backups in minio/minio#4135:

There some tools which we recommend such as

    Restic https://docs.minio.io/docs/restic-with-minio
    Rclone https://docs.minio.io/docs/rclone-with-minio-server

Current Behavior

There are no-longer any options for specifying which PVC is used by the tenant.

Possible Solution

Allow a pool to contain 1 server which will respect the spec.dataSource field of volumeClaimTemplate such that the pool will attach to an existing PVC

PVC:

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: backup-restore
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      # Must be sufficient to hold your data
      storage: 5Gi

Pool:

  pools:
    - servers: 1
      name: pool-0
      volumesPerServer: 1
      volumeClaimTemplate:
        spec:
          dataSource:
            backup-restore

Context

The lack of control over PVCs breaks Valero, K8up, and Restic PVC backups

[jessebot]

Want to also ask: if we're not supposed to use PVCs to do backups, what are we supposed to use for backups in the operator/tenant architecture? Happy to switch, but couldn't find docs on it.

[cloudymax]

Want to also ask: if we're not supposed to use PVCs to do backups, what are we supposed to use for backups in the operator/tenant architecture? Happy to switch, but couldn't find docs on it.

As far as I can tell, this was answered long ago in minio/minio#4135. The links provided in the original answer are all dead now though, so its impossible to know what used to be there. Someone asked again specifically regarding PVCs in minio/minio#17247 but never received an answer.

The only other references I have found are:

• Unable to use the drive with persistence.existingClaim minio#18096 which addresses permissions errors that result from mounting an existing PVC containing minio data
• some people have suggested using mc mirror however in my testing this does not properly copy ownership metadata which results in the restored data being moved to the restore PVC, but is unreadable by the UI.

[jessebot]

• some people have suggested using mc mirror however in my testing this does not properly copy ownership metadata which results in the restored data being moved to the restore PVC, but is unreadable by the UI.

yeah, from their docs:

mc mirror only synchronizes the current object without any version information or metadata. To synchronize an object’s version history and metadata, consider using mc replicate or mc admin replicate.

But if you look at the doc for mc replicate, it says:

The mc replicate command configures and manages the Server-Side Bucket Replication for a MinIO deployment, including active-active replication configurations and resynchronization.

and if you click the Server-Side Bucket Replication link, it says it only works for minio to minio:

...synchronizes new objects and object mutations, such as new object versions or changes to object metadata.
MinIO server-side bucket replication only supports a MinIO cluster on an identical release for the remote replication target.

This is a little confusing.

Is there some sort of way to to sync the objects AND metadata to a third party s3 compatible object store? This would be really helpful for syncing to something like Wasabi, Backblaze B2, etc on say like a nightly basis or something, that way, if your entire site went down, you could spin it back up from the data that was backed up to another s3 host.

[cloudymax]

Spent some time testing,

mc replicate and mc admin replicate only work from Minio to Minio, o we cant use them to backup or restore from B2, Wasabi etc...

mc mirror doesnt transfer metadata or version history so when you try to restore from a backup you get data that Minio doesn't recognize.

[harshavardhana]

Spent some time testing,

mc replicate and mc admin replicate only work from Minio to Minio, o we cant use them to backup or restore from B2, Wasabi etc...

mc mirror doesnt transfer metadata or version history so when you try to restore from a backup you get data that Minio doesn't recognize.

That is correct please setup another MinIO to keep your backups and DR needs via replication.

You can also batch replication to replicate to AWS S3 if needed.

https://min.io/docs/minio/kubernetes/upstream/administration/concepts.html#how-can-i-backup-and-restore-objects-on-minio