Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update the cmd entry of the dvc.yaml file to add cmd as list option #1980

Merged
merged 21 commits into from
Dec 12, 2020
Merged
Show file tree
Hide file tree
Changes from 14 commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
278dab9
Update dvc-files-and-directories.md
ClementWalter Nov 29, 2020
7b796f5
Fix styling and add more details to new cmd behavior
Dec 2, 2020
6c83768
Fix singular command form in repro doc
Dec 2, 2020
e080432
Update command output in repro doc
Dec 2, 2020
b01aba4
Fix typo
Dec 2, 2020
1849588
Add an example of a usage of a list of cmd
Dec 2, 2020
2f937b8
Shorten cmd description and ref to repro
Dec 3, 2020
2384372
Update content/docs/user-guide/dvc-files-and-directories.md
jorgeorpinel Dec 7, 2020
1df6550
Update content/docs/user-guide/dvc-files-and-directories.md
jorgeorpinel Dec 7, 2020
abb79ab
cmd: update repro sample output
jorgeorpinel Dec 7, 2020
12d730c
cmd: update repro sample outputs
jorgeorpinel Dec 7, 2020
362d9e9
Update content/docs/user-guide/dvc-files-and-directories.md
jorgeorpinel Dec 7, 2020
e4ca1bd
Merge branch 'patch-1' of github.com:ClementWalter/dvc.org into patch-1
jorgeorpinel Dec 8, 2020
56252e3
cmd: say how multiple cmd's are reproed
jorgeorpinel Dec 8, 2020
c4aa493
cmd: remove repro --cwd (again)
jorgeorpinel Dec 8, 2020
41ef3ef
Merge branch 'master' into patch-1
jorgeorpinel Dec 8, 2020
4ee75b0
Update content/docs/command-reference/repro.md
jorgeorpinel Dec 12, 2020
ad2dc39
Update content/docs/command-reference/repro.md
jorgeorpinel Dec 12, 2020
1ef4203
Update content/docs/command-reference/repro.md
jorgeorpinel Dec 12, 2020
0c89f68
Update content/docs/command-reference/repro.md
jorgeorpinel Dec 12, 2020
49faf55
Update content/docs/command-reference/repro.md
jorgeorpinel Dec 12, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 12 additions & 7 deletions content/docs/command-reference/repro.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Reproduce complete or partial [pipelines](/doc/command-reference/dag) by
executing commands defined in their [stages](/doc/command-reference/run) in the
correct order. The commands to be executed are determined by recursively
correct order. The stage commands to be executed are determined by recursively
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
analyzing dependencies and <abbr>outputs</abbr> of the target stages.

## Synopsis
Expand Down Expand Up @@ -44,8 +44,13 @@ files, intermediate or final results (except if the `--pull` option is used).

By default, this command checks all [pipeline](/doc/command-reference/dag)
stages to determine which ones have changed. Then it executes the corresponding
commands. [Stage](/doc/command-reference/run) outputs are deleted from the
<abbr>workspace</abbr> before executing the stage commands that produce them.
commands (`cmd` field of `dvc.yaml`). [Stage](/doc/command-reference/run)
outputs are deleted from the <abbr>workspace</abbr> before executing the stage
commands that produce them.

For stages with multiple commands (having a list in the `cmd` field), commands
are run one after the other in the order they are defined. The failure of any
command will halt the remaining stage execution, and raises an error.

There are a few ways to restrict what will be regenerated by this command: by
specifying stages as `targets`, or by using the `--single-item`, `--cwd`, among
Expand Down Expand Up @@ -170,7 +175,7 @@ up-to-date and only execute the final stage.

- `-q`, `--quiet` - do not write anything to standard output. Exit with 0 if all
stages are up to date or if all stages are successfully executed, otherwise
exit with 1. The command defined in the stage is free to write output
exit with 1. The commands defined in the stage are free to write output
regardless of this flag.

- `-v`, `--verbose` - displays detailed tracing information.
Expand Down Expand Up @@ -254,7 +259,7 @@ If we now run `dvc repro`, we should see this:
$ dvc repro
Stage 'filter' didn't change, skipping
Running stage 'count' with command:
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
python process.py numbers.txt > count.txt
python process.py numbers.txt > count.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we need an example with multiple commands in repro...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved to #2045

jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
Updating lock file 'dvc.lock'
```

Expand Down Expand Up @@ -291,7 +296,7 @@ of only the target (`count`) and following stages (none in this case):
```dvc
$ dvc repro --downstream count
Running stage 'count' with command:
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
python process.py numbers.txt > count.txt
python process.py numbers.txt > count.txt
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
Updating lock file 'dvc.lock'
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
```

Expand All @@ -315,5 +320,5 @@ $ dvc dag
```

> Note that using `dvc repro` without `--downstream` in the above example
> results in the execution of the target (`count`), and the preceeding stages
> results in the execution of the target (`count`), and the preceding stages
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
> (only 'filter' in this case).
9 changes: 7 additions & 2 deletions content/docs/user-guide/dvc-files-and-directories.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,7 +132,9 @@ stages:
- performance.json
training:
desc: train your model
cmd: python train.py
cmd:
- pip install -r requirements.txt
- python train.py
deps:
- train.py
- features
Expand All @@ -151,7 +153,10 @@ stages:
by the user with the `--name` (`-n`) option of `dvc run`. Each stage can contain
the possible following fields:

- `cmd` (always present): Executable command defined in this stage
- `cmd` (always present): One or more commands executed by the stage (may
contain either a single value, or a list). Commands are executed sequentially
until all are finished or until one of them fails (see
[`dvc repro`](/doc/command-reference/repro) for details).
jorgeorpinel marked this conversation as resolved.
Show resolved Hide resolved
- `wdir`: Working directory for the stage command to run in (relative to the
file's location). If this field is not present explicitly, it defaults to `.`
(the file's location).
Expand Down