Add --preallocate to rsync options

[graemes]

Please add --preallocate to the options for rsync.

With file-systems that support fallocate (most modern ones - tested with btrfs) it pre-reserves the space required for the plot.

This has two benefits when using multiple plotters against a single drive:
a) minimises fragmentation when interleaving writes
b) and also flags up out of space situation immediately on first to be last writer.

[randellhodges]

Nice. I didn't even know about that option. Out of curiosity, how much delay does that add before the transfer starts? I need to do more research into that (using ext4 myself)

[graemes]

On my quick test seemed to be approx. 30 seconds before the space was registered through df but conversely the transfer felt faster (I'm using btrfs and 10G). YMMV.
Interestingly - if you do a ls -larth on the destination directory the actual size of the file is incremented reflecting the actual transfer.

[graemes]

FYI - rsync usies fallocate under the hood: https://www.linux.org/docs/man1/fallocate.html

[syadnom]

upvote.

[randellhodges]

I added this option to my rsync command a couple days ago, and while I haven't paid enough attention to see if made anything better, I haven't seen any problems since adding.

[syadnom]

randellhodges, which file in plotman did you mod (I'm lazy, save me a minute of searching ;) )

[syadnom]

I'm using aufs set to write to the drive with the most free space, so pre-allocating is key to balancing target drives out in the sync process.

[randellhodges]

randellhodges, which file in plotman did you mod (I'm lazy, save me a minute of searching ;) )

Speaking of lazy, I just modified the file from the official pip install. Yup, I know this isn't the best way of doing it, but it was quick and easy. Just remember that if you are running plotman to stop and start it again.

/home/chia/chia-blockchain/venv/lib/python3.8/site-packages/plotman/archive.py

Replace /home/chia/chia-blockchain/venv with whatever folder the python virtual environment installed into.

[syadnom]

thanks. I'll experiment with you too

[graemes]

From the comments in #565 it seems that the system version of rsync on the mac is missing the --preallocate option due to it's age.

[graemes]

Already added in 0.4 but can be added independently anyway.