Archiving

Introduction

Plotman's archiving feature can be used to distribute your plots to fill however many drives you setup. Users should either use archiving or provide their own plot distribution mechanism. The dst directories are not intended to be the final storage location for plots.

Instructions and comments are based on standard Ubuntu Server installation. Commands and file locations may differ for other Linux distributions.

A significantly reworked archival configuration is being developed. Presently it is only available in https://github.com/ericaltendorf/plotman/pull/627, but is close to being merged, further tested, and then released. If you are using that then see https://gist.github.com/altendky/162700aac293ae72fdcb0e0fa4a33523 until it is merged here.

Machine set up

There are two main pieces to plotting. Creating the plots and getting them to where you want them to be farmed. In some cases these will both be on the same machine, in other cases there will be one or more dedicated plotters with a separate farmer. We will start by setting up the Plot Storage and then configure everything on the Plotter.

1) Plot Storage set up

On your Plot Storage machine, make sure that all the storage drives are mounted, rsync daemon is running and SSH is set up to accept incoming connections from your Plotter.

Mount drives

Archiving expects there to be a directory that contains the mounts for the drives you want to archive to and no other drives mounted there. /mnt itself is unlikely to be a good choice since it is a standard place to mount anything. In this tutorial the following mount points will be used:

/mnt/farm/plots1
/mnt/farm/plots2

Set up rsync daemon (rsyncd)

The Plot Storage machine needs the rsync daemon running and correctly configured using these steps:

1. Install rsync (Ubuntu Server already comes with it installed)

2. Create /etc/rsyncd.conf:

lock file = /var/run/rsync.lock
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid

# don't change the port, plotman (as of version 0.2) has the port hard coded
port = 12000

# rsync module name
[chia]
    # Path with your mounted drives
    path = /mnt/farm
    comment = Chia
    # Use the username that you log into Ubuntu with or create a new one
    uid = username
    # User group (by default same as username)
    gid = username
    read only = no
    list = yes
    # dont uncomment this, 
    #auth users = none
    # plotman does not work with authentication
    #secrets file = none
    # since we dont use auth only accept connections from plotter's ip
    hosts allow = plotter.ip.address

3. Start rsync daemon by typing sudo systemctl start rsync

4. If you automatically want the daemon to start after a reboot type: sudo systemctl enable rsync

Clarification about 'path' variable in rsyncd.conf

The path variable represents a path where all your storage drives are mount. So in our example in /mnt/farm we have two drives mounted, namely plots1 and plots2.

If you only have a single drive mounted for archiving (e.g. /media/username/plots) your path should not point to /media/username/plots, but rather to /media/username. Note that you should still consider that any other mount points in that directory will be considered for archiving. This should generally drive you to use explicit mount points instead of any default locations.

SSH configuration

Make sure you configure ssh in a way that you can connect from your Plotter without having to use a password or keyfile passphrase.

The best way to do this is to create a ssh-key without a passphrase on the Plotter and copy the public key to your Plot Storage.

2) Plotter set up

Update your plotman.yaml

Update the archive section of your plotman.yaml (Default Configuration File) file. If the Plotter and the Plot storage are on the same machine then you can use rsyncd_host: localhost. In our example the config would look as follows:

archive:
        rsyncd_module: chia               # Module name specified in the Plot Storage's rsyncd.conf
        rsyncd_path: /mnt/farm            # Path where your storage drives are mounted (same as in rsyncd.conf)
        rsyncd_bwlimit: 100000            # Bandwidth limit in KB/s
        rsyncd_host: plot.storage.ip      # IP address or hostname of your Plot Storage, localhost if local
        rsyncd_user: username             # Username that can ssh into your Plot Storage

Test your setup / trouble shooting

Before starting plotman you should make sure that both SSH and rsync is set up correctly. If you can't successfully run the tests below, plotman's archiving will not work.

Make sure SSH is set up correctly

The machines should be set up in such a way that you can SSH from your Plotter to your Plot Storage without having to enter a password. In order to do this you should use a ssh public/private keypair that doesn't require entering a passphrase.

To test if you set this up correctly type the following command on your Plotter: ssh [email protected] df -aBK | grep /mnt/farm/

The command above should give you a list of all the mounted drives of your Plot Storage. If it doesn't, or if it asks for a password or a passphrase then SSH is not set up as required by plotman and archiving will not work.

Make sure rsync works

1. Create a testfile on your Plotter using echo "testing" > testfile.test
2. Enter the following on your Plotter: rsync -P testfile.test rsync://[email protected]:12000/chia/plots1
3. Check your Plot Storage and make sure testfile.test exists in /mnt/farm/plots1

Manually run plotman's rsync command

If both of the tests above pass but archiving still doesn't work, you can look at the rsync output in your console.

1. Start plotman interactive
2. Locate the rsync line in the Log section at the bottom of the screen, e.g.:

05-03 08:37:46 Starting archive: rsync --bwlimit=80000 --remove-source-files -P /mnt/dst1/plot-k32-2021-05-03-01-50-b4271f88a74b36b516c242151e00fdda20e3f31ce1f8624465bf05a195009ecd.plot rsync://[email protected]:12000/chia/plots2

3. Copy the part after 05-03 08:37:46 Starting archive: . In our example that would be:

rsync --bwlimit=80000 --remove-source-files -P /mnt/plots1/plot-k32-2021-05-03-01-50-b4271f88a74b36b516c242151e00fdda20e3f31ce1f8624465bf05a195009ecd.plot rsync://[email protected]:12000/chia/plots2

4. Run the command in your terminal and use the output for finding any errors you may have in your configuration.

Disable archiving in plotman

In order to disable archiving, completely comment out the corresponding archive: section in your .config/plotman/plotman.yaml. Users should either use archiving or provide their own plot distribution mechanism. The dst directories are not intended to be the final storage location for plots.