Skip to content

Commit

Permalink
docs: locking-related updates
Browse files Browse the repository at this point in the history
Related to #860
  • Loading branch information
efiop committed Jan 6, 2020
1 parent 0b607e9 commit 80bfcd2
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 0 deletions.
33 changes: 33 additions & 0 deletions public/static/docs/command-reference/repro.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,39 @@ files, intermediate or final results. It saves all the data files, intermediate
or final results into the <abbr>DVC cache</abbr> (unless `--no-commit` option is
specified), and updates stage files with the new checksum information.

### Parallel stage execution

Currently, `dvc repro` is not able to parallelize stage execution automatically.
If you need to do this, you can launch `dvc repro` multiple times manually.

For example, let's say a <abbr>pipeline</abbr> graph looks something like this:

```
$ dvc pipeline show --ascii result.py
+--------+ +--------+
| A1.dvc | | B1.dvc |
+--------+ +--------+
* *
* *
* *
+--------+ +--------+
| A2.dvc | | B2.dvc |
+--------+ +--------+
* *
** **
* *
+------------+
| result.dvc |
+------------+
```

This pipeline consists of two parallel branches (`A` and `B`), and the final
"result" stage, where the branches merge. To reproduce both branches at the same
time, you could run `dvc repro A2.dvc` and `dvc repro B2.dvc` at the same time
(e.g. in separate terminals). After both finish successfully, you can then run
`dvc repro result.dvc`: DVC will know that both branches are already up-to-date
and only execute the final stage.

## Options

- `-f`, `--force` - reproduce a pipeline, regenerating its results, even if no
Expand Down
6 changes: 6 additions & 0 deletions public/static/docs/user-guide/dvc-files-and-directories.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ operation:

- `.dvc/lock`: Lock file for the entire DVC project

- `.dvc/tmp`: Directory for miscellaneous temporary files

- `.dvc/tmp/rwlock`: JSON file that contains read and write locks for specific
dependencies and outputs, to allow safely running multiple DVC commands in
parallel.

## Structure of cache directory

There are two ways in which the data is stored in <abbr>cache</abbr>: As a
Expand Down

0 comments on commit 80bfcd2

Please sign in to comment.