Skip to content
This repository has been archived by the owner on Oct 19, 2023. It is now read-only.

Backup strategy when cache is involved #11

Open
elmuz opened this issue Feb 18, 2022 · 0 comments
Open

Backup strategy when cache is involved #11

elmuz opened this issue Feb 18, 2022 · 0 comments

Comments

@elmuz
Copy link

elmuz commented Feb 18, 2022

First, let me see if I understood the logic behind your filesystem layout (once MergerFS is in action):

  • all over, the services usually point to {{ mergerfs_root }} ...maybe the only exception being the transcoding folder that bypasses cache directly.
  • MergerFS root folder is an array of [cache1 + cache2 + ... + disk1 + disk2 ...].
  • Because of MergerFS policies the cache has priority and is filled first. I could see that MergerFS policy is LFS (i.e. gives priority to the disk with the least free space).
  • The {{ cache_root }} has MFS policy, which balances the occupancy between disks. However, it doesn't really seem to be directly invoked by services.
  • A smart cronjob moves not-accessed files from cache disks to non-cache disks so that the cache available space is at least a specific percentage.
  • At beginning of time cache has priority due to the fact of being smaller in size. This somehow imposes a sorting mechanism from the smallest (in capacity) to the biggest. There is also a hidden necessary condition that wants the slow disks never to be filled more than the cache, or they could gain priority over the cache. This is accomplished by the minfreespace=500G, which must be bigger than the percentage of free space preserved by the un-caching script.
  • Snapraid works on (slow) disk level, hence the cache is not considered.

Correct?

My question now is:

  1. did you consider a backup strategy for those elements that will live (only) in the cache? Maybe they are accessed often and they never have the chance of being "promoted" to the slow disks. For instance docker data backup folders.
  2. Are you aware of an inverse solution for caching back files that have been moved to slow disks that for any reason become useful again?
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant