Skip to content

Commit

Permalink
docs: locking-related updates
Browse files Browse the repository at this point in the history
Related to #860
  • Loading branch information
efiop committed Jan 6, 2020
1 parent 0b607e9 commit 5c353c6
Show file tree
Hide file tree
Showing 2 changed files with 39 additions and 0 deletions.
33 changes: 33 additions & 0 deletions public/static/docs/command-reference/repro.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,39 @@ files, intermediate or final results. It saves all the data files, intermediate
or final results into the <abbr>DVC cache</abbr> (unless `--no-commit` option is
specified), and updates stage files with the new checksum information.

### Parallel stage execution

Currently `dvc repro` is not able to parallelize execution by itself (see
[iterative/dvc#755](https://github.com/iterative/dvc/issues/755)), so if you
need to do that you could launch multiple `dvc repro`s yourself. For example,
say your DAG looks something like:

```
$ dvc pipeline show --ascii result.py
+--------+ +--------+
| A1.dvc | | B1.dvc |
+--------+ +--------+
* *
* *
* *
+--------+ +--------+
| A2.dvc | | B2.dvc |
+--------+ +--------+
* *
** **
* *
+------------+
| result.dvc |
+------------+
```

so it consists of two pipeline branches (pipeline `A` and pipeline `B`) and the
final `result` stage. To reproduce both branches at the same time, you could run
`dvc repro A2.dvc` and `dvc repro B2.dvc` at the same time (e.g. by running them
in separate terminals). After both are done running, you could then run
`dvc repro result.dvc` that will see that both branches are already up-to-date
and will only run the final stage.

## Options

- `-f`, `--force` - reproduce a pipeline, regenerating its results, even if no
Expand Down
6 changes: 6 additions & 0 deletions public/static/docs/user-guide/dvc-files-and-directories.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,12 @@ operation:

- `.dvc/lock`: Lock file for the entire DVC project

- `.dvc/tmp`: Directory for arbitrary temporary files

- `.dvc/tmp/rwlock`: JSON file that contains read and write locks for specific
dependencies and outputs to allow safely running multiple DVC commands in
parallel.

## Structure of cache directory

There are two ways in which the data is stored in <abbr>cache</abbr>: As a
Expand Down

0 comments on commit 5c353c6

Please sign in to comment.