Update the cmd entry of the dvc.yaml file to add cmd as list option #1980

ClementWalter · 2020-11-29T12:14:13Z

Update the dvc.yaml documentation with the list possibility for the cmd entry.

Core PR #4934

You may disregard these recommendations if you used the Edit on GitHub button from dvc.org to improve a doc in place.

❗ Please read the guidelines in the Contributing to the Documentation list if you make any substantial changes to the documentation or JS engine.

🐛 Please make sure to mention Fix #issue (if applicable) in the description of the PR. This causes GitHub to close it automatically when the PR is merged.

Please choose to allow us to edit your branch when creating the PR.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Update the dvc.yaml documentation with the list possibility for the `cmd` entry.

jorgeorpinel · 2020-11-29T21:07:18Z

Fix #4373

I don't think this docs PR fixes a core issue 🙂 please link to the core PR this matches instead (don't link them). Thanks

content/docs/user-guide/dvc-files-and-directories.md

jorgeorpinel

Left a suggestion ^ Also:

The dvc.yaml example should show multiple commands, probably. Any other dvc.yaml examples throughout docs that could feature this?
And what about the cmd refs? At least https://dvc.org/doc/command-reference/run needs an update, I think. Maybe repro

Thanks

jorgeorpinel · 2020-11-29T23:19:42Z

Oh and BTW the PR checks are failing, please follow https://dvc.org/doc/user-guide/contributing/docs (we can also merge the restyled PR #1981 if needed but preferably, it shouldn't be necessary).

ClementWalter · 2020-11-30T17:56:01Z

And what about the cmd refs? At least https://dvc.org/doc/command-reference/run needs an update, I think. Maybe repro

Do you think that the dvc run should handle a command, with, for instance, && so as to make it eventually a list in the dvc.yaml? I may be nice, but I am not sure it is the desired behavior because it requires some parsing of the user command, which, indeed, does not seem to be a preferred choice. @efiop, an idea about that?

efiop · 2020-11-30T19:43:43Z

@ClementWalter I agree, that is not a desired behavior. List of commands could be only specified by hand by editing dvcfile, dvc run doesn't need to support it.

jorgeorpinel · 2020-12-01T08:07:10Z

dvc run doesn't need to support it.

Sounds good. But at the very least this will need checking the usage of the term "command" in repro (e.g. see -q option) and maybe other refs so it goes from singular to plural. Also it seems like the output of dvc repro will change slightly? There are a few examples that it would be ideal to update per this. Thanks

ClementWalter · 2020-12-02T19:57:19Z

@jorgeorpinel I try to rephrase, hopefully it is clearer. I have added a use case for a list of commands, hopefully relevant as well. What do you think?

content/docs/user-guide/dvc-files-and-directories.md

jorgeorpinel · 2020-12-03T19:39:29Z

content/docs/command-reference/repro.md

+Running stage 'count':
+$ python process.py numbers.txt > count.txt


This is the actual output? Including a $ character?

yes, it has changed from the previous "with command: \t {command}" to this with line beginning with $

So repro becomes some sort of background process that actually enters commands into the user's shell? Or we specifically print the $ character + command string?

we print it. Indeed it went from previous repeated "running command", to "> {command}" suggested by @skshetry to "$ {command}" because the ">" were confusing with possible ">" inside a command (like echo > file)

@ClementWalter @skshetry @efiop this can create a mess across all docs where we use $ as a starting symbol for all shell commands. We'll need some mechanism to escape it, and even if we do we'll end up with confusing text anyway - mix of actual commands and outputs. Let's reconsider this please. Also out of curiosity how exactly is it related to the PR (multiple commands), I mean this particular change in UI/UX? what was the reason to change it?

Implemented this on iterative/dvc#5041.

not sure > is great since it can be part of the command

For now, we add > instead of $. Let's discuss this on a different issue.

Let's discuss this on a different issue.

Created iterative/dvc#5043 to discuss this further.

For now, we add > instead of $.

Actually right now it's a tab, why change it if we're not sure? I'd vote to roll back that change and yes to the discussion.

Actually I'm back to having an opinion 😆 — see iterative/dvc#5043 (comment)

content/docs/command-reference/repro.md

jorgeorpinel

Thanks for all the details! Let's try to summarize this last bit about the cmd field though. And no worries, I can finish the writing at some point but I do have some questions about it before I can do that. My suggestions for now:

content/docs/user-guide/dvc-files-and-directories.md

per iterative#1980 (review)

jorgeorpinel · 2020-12-07T05:46:25Z

content/docs/command-reference/repro.md

 $ dvc repro
 Stage 'filter' didn't change, skipping
 Running stage 'count' with command:
-        python process.py numbers.txt > count.txt
+	python process.py numbers.txt > count.txt


I wonder if we need an example with multiple commands in repro...

Moved to #2045

per iterative#1980 (review)

skshetry

Rest looks good, just one issue with the output format.

content/docs/command-reference/repro.md

ClementWalter · 2020-12-09T18:05:05Z

I am not sure about the remaining changes to be brought to this PR.

jorgeorpinel · 2020-12-11T23:50:59Z

Hey no worries @ClementWalter I've assigned myself to finish up this one. Thanks!

content/docs/command-reference/repro.md

jorgeorpinel · 2020-12-12T00:20:13Z

content/docs/command-reference/repro.md

-Running stage 'count' with command:
-        python process.py numbers.txt > count.txt
+Running stage 'count':


OK I removed "with command" from this file (for now, see iterative/dvc#5043 (comment)) but there's another 4 instances/files with this text in the repo.

content/docs/command-reference/repro.md

jorgeorpinel

I think this is mergeable pending #1980 (review) and #1980 (review) above, for a follow-up PR.

shcheklein

Thanks @ClementWalter , @jorgeorpinel , and @skshetry !

…option (#1980)" This reverts commit 23247d8.

jorgeorpinel · 2020-12-22T00:56:30Z

Looks like this is a 2.0 feature so I removed it from the v1 branch in 365b768.

* params: document --targets option * cmd: fux config file paths * Update content/docs/command-reference/params/diff.md Co-authored-by: Jorge Orpinel <[email protected]> * Revert "Update the cmd entry of the dvc.yaml file to add cmd as list option (#1980)" This reverts commit 23247d8. * Restyled by prettier (#2042) Co-authored-by: Restyled.io <[email protected]> * docs: misc copy edits * cmd: roll back wrong changes to repro Co-authored-by: Paweł Redzyński <[email protected]> Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com> Co-authored-by: Restyled.io <[email protected]>

Update dvc-files-and-directories.md

278dab9

Update the dvc.yaml documentation with the list possibility for the `cmd` entry.

restyled-io bot mentioned this pull request Nov 29, 2020

Restyle Update the cmd entry of the dvc.yaml file to add cmd as list option #1981

Closed

ClementWalter mentioned this pull request Nov 29, 2020

Allow user to pass a list of commands for one single stage iterative/dvc#4934

Merged

2 tasks

jorgeorpinel reviewed Nov 29, 2020

View reviewed changes

content/docs/user-guide/dvc-files-and-directories.md Outdated Show resolved Hide resolved

jorgeorpinel suggested changes Nov 29, 2020

View reviewed changes

Clement Walter added 5 commits December 2, 2020 20:06

Fix styling and add more details to new cmd behavior

7b796f5

Fix singular command form in repro doc

6c83768

Update command output in repro doc

e080432

Fix typo

b01aba4

Add an example of a usage of a list of cmd

1849588

jorgeorpinel reviewed Dec 3, 2020

View reviewed changes

content/docs/user-guide/dvc-files-and-directories.md Outdated Show resolved Hide resolved

jorgeorpinel reviewed Dec 3, 2020

View reviewed changes

content/docs/user-guide/dvc-files-and-directories.md Outdated Show resolved Hide resolved

jorgeorpinel reviewed Dec 3, 2020

View reviewed changes

content/docs/command-reference/repro.md Show resolved Hide resolved

jorgeorpinel reviewed Dec 3, 2020

View reviewed changes

content/docs/user-guide/dvc-files-and-directories.md Outdated Show resolved Hide resolved

content/docs/user-guide/dvc-files-and-directories.md Outdated Show resolved Hide resolved

Shorten cmd description and ref to repro

2f937b8

jorgeorpinel self-assigned this Dec 7, 2020

Update content/docs/user-guide/dvc-files-and-directories.md

2384372

jorgeorpinel reviewed Dec 7, 2020

View reviewed changes

content/docs/user-guide/dvc-files-and-directories.md Outdated Show resolved Hide resolved

jorgeorpinel reviewed Dec 7, 2020

View reviewed changes

content/docs/user-guide/dvc-files-and-directories.md Outdated Show resolved Hide resolved

Update content/docs/user-guide/dvc-files-and-directories.md

1df6550

jorgeorpinel reviewed Dec 7, 2020

View reviewed changes

content/docs/user-guide/dvc-files-and-directories.md Show resolved Hide resolved

jorgeorpinel added 2 commits December 6, 2020 23:17

cmd: update repro sample output

abb79ab

per iterative#1980 (review)

cmd: update repro sample outputs

12d730c

per iterative#1980 (review)

jorgeorpinel reviewed Dec 7, 2020

View reviewed changes

skshetry mentioned this pull request Dec 7, 2020

run/repro: show command on dry-run, add ">" to script output iterative/dvc#5041

Merged

2 tasks

jorgeorpinel added 2 commits December 7, 2020 21:43

Merge branch 'patch-1' of github.com:ClementWalter/dvc.org into patch-1

e4ca1bd

cmd: say how multiple cmd's are reproed

56252e3

per iterative#1980 (review)

jorgeorpinel requested review from shcheklein and skshetry December 8, 2020 04:28

skshetry approved these changes Dec 8, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

jorgeorpinel added 2 commits December 8, 2020 13:38

cmd: remove repro --cwd (again)

c4aa493

Merge branch 'master' into patch-1

41ef3ef

jorgeorpinel reviewed Dec 12, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

Update content/docs/command-reference/repro.md

4ee75b0

jorgeorpinel reviewed Dec 12, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

Update content/docs/command-reference/repro.md

ad2dc39

jorgeorpinel reviewed Dec 12, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

Update content/docs/command-reference/repro.md

1ef4203

jorgeorpinel reviewed Dec 12, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

Update content/docs/command-reference/repro.md

0c89f68

jorgeorpinel reviewed Dec 12, 2020

View reviewed changes

content/docs/command-reference/repro.md Outdated Show resolved Hide resolved

Update content/docs/command-reference/repro.md

49faf55

jorgeorpinel approved these changes Dec 12, 2020

View reviewed changes

shcheklein merged commit 23247d8 into iterative:master Dec 12, 2020

shcheklein reviewed Dec 12, 2020

View reviewed changes

jorgeorpinel added the 2.0 release label Dec 22, 2020

jorgeorpinel mentioned this pull request Dec 22, 2020

repro: add multi-cmd example #2045

Closed

jorgeorpinel added a commit that referenced this pull request Dec 22, 2020

Revert "Update the cmd entry of the dvc.yaml file to add cmd as list …

365b768

…option (#1980)" This reverts commit 23247d8.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the cmd entry of the dvc.yaml file to add cmd as list option #1980

Update the cmd entry of the dvc.yaml file to add cmd as list option #1980

ClementWalter commented Nov 29, 2020 •

edited

Loading

jorgeorpinel commented Nov 29, 2020

jorgeorpinel left a comment

jorgeorpinel commented Nov 29, 2020 •

edited

Loading

ClementWalter commented Nov 30, 2020 •

edited by jorgeorpinel

Loading

efiop commented Nov 30, 2020 •

edited by jorgeorpinel

Loading

jorgeorpinel commented Dec 1, 2020

ClementWalter commented Dec 2, 2020

jorgeorpinel Dec 3, 2020

ClementWalter Dec 3, 2020

jorgeorpinel Dec 3, 2020

ClementWalter Dec 3, 2020

shcheklein Dec 4, 2020

skshetry Dec 7, 2020

skshetry Dec 7, 2020

jorgeorpinel Dec 11, 2020

This comment was marked as outdated.

jorgeorpinel Dec 12, 2020

jorgeorpinel left a comment

jorgeorpinel Dec 7, 2020

jorgeorpinel Dec 22, 2020

skshetry left a comment

ClementWalter commented Dec 9, 2020

jorgeorpinel commented Dec 11, 2020

jorgeorpinel Dec 12, 2020 •

edited

Loading

jorgeorpinel left a comment •

edited

Loading

shcheklein left a comment

jorgeorpinel commented Dec 22, 2020

		Running stage 'count':
		$ python process.py numbers.txt > count.txt

Update the cmd entry of the dvc.yaml file to add cmd as list option #1980

Update the cmd entry of the dvc.yaml file to add cmd as list option #1980

Conversation

ClementWalter commented Nov 29, 2020 • edited Loading

jorgeorpinel commented Nov 29, 2020

jorgeorpinel left a comment

Choose a reason for hiding this comment

jorgeorpinel commented Nov 29, 2020 • edited Loading

ClementWalter commented Nov 30, 2020 • edited by jorgeorpinel Loading

efiop commented Nov 30, 2020 • edited by jorgeorpinel Loading

jorgeorpinel commented Dec 1, 2020

ClementWalter commented Dec 2, 2020

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

This comment was marked as outdated.

Choose a reason for hiding this comment

jorgeorpinel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

skshetry left a comment

Choose a reason for hiding this comment

ClementWalter commented Dec 9, 2020

jorgeorpinel commented Dec 11, 2020

jorgeorpinel Dec 12, 2020 • edited Loading

Choose a reason for hiding this comment

jorgeorpinel left a comment • edited Loading

Choose a reason for hiding this comment

shcheklein left a comment

Choose a reason for hiding this comment

jorgeorpinel commented Dec 22, 2020

ClementWalter commented Nov 29, 2020 •

edited

Loading

jorgeorpinel commented Nov 29, 2020 •

edited

Loading

ClementWalter commented Nov 30, 2020 •

edited by jorgeorpinel

Loading

efiop commented Nov 30, 2020 •

edited by jorgeorpinel

Loading

jorgeorpinel Dec 12, 2020 •

edited

Loading

jorgeorpinel left a comment •

edited

Loading