Skip to content

Remote executors

Peter Rowlands (변기호) edited this page Aug 30, 2021 · 12 revisions

Machine management

Note: This is documentation for an experimental feature which is under active development, it should not be used in production environments.

dvc machine provides a set of DVC commands for provisioning and managing remote machines which will eventually be used for executing DVC experiments.

Currently dvc machine implementation utilizes https://github.com/iterative/terraform-provider-iterative and requires the terraform client be installed and available in your PATH.

Installation/Configuration

  • Download & install terraform client for your platform
  • Install DVC deps (preferably using install -e from master):
pip install dvc[terraform]
  • Enable the dvc machine feature (either per-repo or globally):
dvc config [--global] feature.machine true

Machine configuration

Machines are configured similarly to DVC remotes, and configuration usage generally mirrors dvc remote add/modify/remove.

  • dvc machine add - adds a machine to your repo configuration (note that no machine instance will actually be created until dvc machine create is run).
  • dvc machine modify - modify the configuration for an existing machine. For a full list of available options, refer to the documentation for https://github.com/iterative/terraform-provider-iterative#machine
  • dvc machine remove - removes a machine from your repo configuration (note that any running machine instances should be destroyed with dvc machine destroy before removing the machine from your repo configuration.

Instance management

  • dvc machine create - create and start an instance of a configured machine.
  • dvc machine destroy - stop and destroy a previously created machine instance.
  • dvc machine ssh - connect to a machine via SSH.
Clone this wiki locally