Remote executors

Machine management

Note: This is documentation for an experimental feature which is under active development, it should not be used in production environments.

dvc machine provides a set of DVC commands for provisioning and managing remote machines which will eventually be used for executing DVC experiments.

Currently dvc machine implementation utilizes https://github.com/iterative/terraform-provider-iterative and requires the terraform client be installed and available in your PATH.

Installation/Configuration

• (Optional) Download & install terraform client for your platform

• (Optional) Install latest tpi from master (pip install -e)

• Install DVC deps (preferably using pip install -e from master:

  pip install dvc[terraform]
  

  • This will install tpi from pypi if you did not already install it from source

Note: If you do not install a terraform client yourself, it will be downloaded and installed for you (via tpi)

• Enable the dvc machine feature (either per-repo or globally):

dvc config [--global] feature.machine true

Machine configuration

Machines are configured similarly to DVC remotes, and configuration usage generally mirrors dvc remote add/modify/remove.

• dvc machine add - adds a machine to your repo configuration (note that no machine instance will actually be created until dvc machine create is run).
• dvc machine modify - modify the configuration for an existing machine. For a full list of available options, refer to the documentation for https://github.com/iterative/terraform-provider-iterative#machine
• dvc machine list - List the configuration of one/all machines.
• dvc machine remove - removes a machine from your repo configuration (note that any running machine instances should be destroyed with dvc machine destroy before removing the machine from your repo configuration.

Instance management

• dvc machine create - create and start an instance of a configured machine.
• dvc machine status - List the running status of the instances from one specified or all machines.
• dvc machine destroy - stop and destroy a previously created machine instance.
• dvc machine ssh - connect to a machine via SSH.
  • Your default ssh client will be used if available in your PATH.
  • Otherwise a limited functionality client session will be provided via asyncssh - Note that interactive programs (particularly line editors like vi) may not work as expected when run in this shell session.