Skip to content

Replication Q&A

Allan Roger Reid edited this page Apr 19, 2024 · 1 revision

Questions: does _MINIO_BATCH_REPLICATION_WORKERS affect the replication workers used my mc replicate ? I know that by default the replication workers are set to N/2 (where n is number of CPUs on the host where the batch is running), and the env var _MINIO_BATCH_REPLICATION_WORKERS can be used to tweak that to any number

the batch env does not affect minio's bucket replication workers ( i.e done with mc admin config set sitea api replication_priority )

Will mc replicate add ... priority or mc admin config ... replication_priority help in anyway to improve throughput - assuming there are no other replication rules?

replication automatically scales up and down workers according to the priority chosen. The priority here is very specific to bucket/site replication which is heavily parallel . batch is controlled only by the defaults and the _MINIO_BATCH_REPLICATION_WORKERS

I also just saw mc admin config set ALIAS api replication_max_workers 500 that is not for general use - it is mostly for some advanced replication cases where workers need to be limited .

This was specifically added for Disney - they wanted to cap the number of workers

Last thing - its not clear how replication_priority works. Is there a range of integers to use? And in the absence of other repl rules, how does minio use this number?

https://github.com/minio/minio/blob/master/docs/config/README.md slow -> cap at 50, default(auto) -> 100, fast -> 500 or max workers set the max-workers if set will override the max limit

Clone this wiki locally