Backup reduce downtime - Restart nextcloud - do borg maintenance/pruning in parallel

[frederikb96]

I understand that the Nextcloud containers must be shut down while the borg backup is running. However, I guess they can be restarted once the backup is done. There is no need to wait until the borg backup maintenance is finished.

Here is an example where the backup itself only took 18 minutes, but the maintenance (pruning) took 04 hours:

Performing backup...
Starting the backup...
Creating archive at "/mnt/borgbackup/borg::20230807_020134-nextcloud-aio"
------------------------------------------------------------------------------
Repository: /mnt/borgbackup/borg
Archive name: 20230807_020134-nextcloud-aio
Archive fingerprint: 45ac5135ee737f7a46b86cc445ee1e62ea6ebdb28f750cba9d2f5463c1deae27
Time (start): Mon, 2023-08-07 02:01:35
Time (end):   Mon, 2023-08-07 02:20:16
Duration: 18 minutes 40.87 seconds
Number of files: 1263983
Utilization of max. archive size: 1%
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
This archive:              924.14 GB            879.43 GB              4.27 GB
All archives:               15.43 TB             14.70 TB            841.78 GB

                       Unique chunks         Total chunks
Chunk index:                 1404419             24173414
------------------------------------------------------------------------------
BORG_PRUNE_OPTS are --keep-within=7d --keep-weekly=4 --keep-monthly=6
Pruning the archives...
------------------------------------------------------------------------------
                       Original size      Compressed size    Deduplicated size
Deleted data:             -421.31 GB           -405.16 GB            -44.22 GB
All archives:               15.01 TB             14.29 TB            797.57 GB

                       Unique chunks         Total chunks
Chunk index:                 1372575             23819088
------------------------------------------------------------------------------
Compacting the archives...
Backup finished successfully on 07.08.2023 - 06:03:57 (04 hours 02 minutes 23 seconds)

[szaimen]

Is it always taking 4 hours in your case or only sometimes?

[frederikb96]

Only sometimes, in some runs a lot of data is pruned. But not always :)

[szaimen]

How long does it take usually?

[szaimen]

Also what kind of storage is the target backup dir? Is this a network FS or e.g. HDD or SSD?

[frederikb96]

Normally it only takes some minutes. It is a lvm partition composed of a ssd as cache and a network storage as main disk. Thus, operations which are adding data, like the real backup process are cached on the fast ssd and do not take long. However, operations like pruning old archive data, might take longer since they cannot be cached on the fast SSD.

[szaimen]

Normally it only takes some minutes.

All right. This sounds sane then.

a network storage as main disk.

This is peobably the bottleneck then as you already figured out - switching to a local storage would probably fix this.

---

Unfortunately I dont see us changing the logic around pruning as it would mean a bigger rewrite and many possible side-effects which at least I dont have time for.

[frederikb96]

Ok but thanks anyways :)

[szaimen]

FYI: with v10.0.0 beta now borg remote backups are supported