How to automatically move plots on completion

[nyxynyx]

Hi, I'm new to Chai and plotman is awesome!

How can we edit plotman.yaml such that the completed plots on an Ubuntu 20.04 system are automatically copied to a drive on a Windows 10 system on the same local network?

There is no rsync server on the Windows 10 system. If there must be an rsync daemon running on Windows 10, what is the recommended software that is easy to setup?

Thank you for your help!

plotman.yaml:

# Default/example plotman.yaml configuration file

# Options for display and rendering
user_interface:
        # Call out to the `stty` program to determine terminal size, instead of
        # relying on what is reported by the curses library.   In some cases,
        # the curses library fails to update on SIGWINCH signals.  If the
        # `plotman interactive` curses interface does not properly adjust when
        # you resize the terminal window, you can try setting this to True. 
        use_stty_size: True

# Where to plot and log.
directories:
        # One directory in which to store all plot job logs (the STDOUT/
        # STDERR of all plot jobs).  In order to monitor progress, plotman
        # reads these logs on a regular basis, so using a fast drive is
        # recommended.
        log: /home/chia/chia/logs

        # One or more directories to use as tmp dirs for plotting.  The
        # scheduler will use all of them and distribute jobs among them.
        # It assumes that IO is independent for each one (i.e., that each
        # one is on a different physical device).
        #
        # If multiple directories share a common prefix, reports will
        # abbreviate and show just the uniquely identifying suffix.
        tmp:
                - /plots/tmp/
s
        # Optional: Allows overriding some characteristics of certain tmp
        # directories. This contains a map of tmp directory names to
        # attributes. If a tmp directory and attribute is not listed here,
        # it uses the default attribute setting from the main configuration.
        #
        # Currently support override parameters:
        #     - tmpdir_max_jobs
        tmp_overrides:
                # In this example, /mnt/tmp/00 is larger than the other tmp
                # dirs and it can hold more plots than the default.
                "/mnt/tmp/00":
                        tmpdir_max_jobs: 5

        # Optional: tmp2 directory.  If specified, will be passed to
        # chia plots create as -2.  Only one tmp2 directory is supported.
        # tmp2: /mnt/tmp/a

        # One or more directories; the scheduler will use all of them.
        # These again are presumed to be on independent physical devices,
        # so writes (plot jobs) and reads (archivals) can be scheduled
        # to minimize IO contention.
        dst:
                - /plots/dst/00

        # Archival configuration.  Optional; if you do not wish to run the
        # archiving operation, comment this section out.
        #
        # Currently archival depends on an rsync daemon running on the remote
        # host, and that the module is configured to match the local path.
        # See code for details.
        # archive:
        #         rsyncd_module: plots
        #         rsyncd_path: /plots
        #         rsyncd_bwlimit: 80000  # Bandwidth limit in KB/s
        #         rsyncd_host: myfarmer
        #         rsyncd_user: chia
                # Optional index.  If omitted or set to 0, plotman will archive
                # to the first archive dir with free space.  If specified,
                # plotman will skip forward up to 'index' drives (if they exist).
                # This can be useful to reduce io contention on a drive on the
                # archive host if you have multiple plotters (simultaneous io
                # can still happen at the time a drive fills up.)  E.g., if you
                # have four plotters, you could set this to 0, 1, 2, and 3, on
                # the 4 machines, or 0, 1, 0, 1.
                #   index: 0


# Plotting scheduling parameters
scheduling:
        # Run a job on a particular temp dir only if the number of existing jobs
        # before tmpdir_stagger_phase_major tmpdir_stagger_phase_minor
        # is less than tmpdir_stagger_phase_limit.
        # Phase major corresponds to the plot phase, phase minor corresponds to
        # the table or table pair in sequence, phase limit corresponds to
        # the number of plots allowed before [phase major, phase minor]
        tmpdir_stagger_phase_major: 2
        tmpdir_stagger_phase_minor: 1
        # Optional: default is 1
        tmpdir_stagger_phase_limit: 1

        # Don't run more than this many jobs at a time on a single temp dir.
        tmpdir_max_jobs: 3

        # Don't run more than this many jobs at a time in total.
        global_max_jobs: 12

        # Don't run any jobs (across all temp dirs) more often than this.
        global_stagger_m: 30

        # How often the daemon wakes to consider starting a new plot job
        polling_time_s: 20


# Plotting parameters.  These are pass-through parameters to chia plots create.
# See documentation at
# https://github.com/Chia-Network/chia-blockchain/wiki/CLI-Commands-Reference#create
plotting:
        k: 32
        e: False             # Use -e plotting option
        n_threads: 2         # Threads per job
        n_buckets: 128       # Number of buckets to split data into
        job_buffer: 4608     # Per job memory
        # If specified, pass through to the -f and -p options.  See CLI reference.
        #   farmer_pk: ...
        #   pool_pk: ...

[ericaltendorf]

I would say this is another vote for #110 (comment)

[altendky]

I suspect that there are also various ways to get rsyncd 'on Windows'. To start there are WSL, msys2, and Cygwin. I haven't checked the package list for the latter two but suspect they'll have an rsyncd.

But yeah, more options would be good.

[nyxynyx]

@ericaltendorf @altendky Managed to get rsync daemon setup in Windows 10 using WSL and mounted the ext drives into WSL.

Now, how can we cofigure Plotman to use a non-standard SSH port to do its rsync?

[altendky]

@nyxynyx plotman has 12000 hardcoded for the rsyncd port (which isn't ssh).

plotman/src/plotman/archive.py

Line 69 in c0fe10c

rsync_url = 'rsync://%s@%s:12000/%s' % (

If you cheat and set rsyncd_user: -p 123 chia or such you might be able to sneak in a port spec to ssh... which is really part of why formatting command lines isn't great.

plotman/src/plotman/archive.py

Line 52 in c0fe10c

df_cmd = ('ssh %s@%s df -aBK | grep " %s/"' %

This whole situation is the topic underlying #110.