[WIP] cmd: experiment #1926
[WIP] cmd: experiment #1926
Conversation
| A set of commands to generate and compare local _experiments_: | ||
| [show](/doc/command-reference/exp/show), | ||
| [checkout](/doc/command-reference/exp/show), | ||
| [diff](/doc/command-reference/exp/diff), [run](/doc/command-reference/exp/run), |
There was a problem hiding this comment.
Same: run before diff, prob? Maybe also resume before diff, even, since I think it's closely linked to run.
jorgeorpinel
temporarily deployed
to
dvc-landing-experiments-ixzhup
jorgeorpinel
temporarily deployed
to
dvc-landing-experiments-ixzhup
jorgeorpinel
temporarily deployed
to
dvc-landing-experiments-ixzhup
| @@ -0,0 +1,44 @@ | |||
| # exp[eriments] | |||
|
|
|||
| A set of commands to generate and compare local _experiments_: | |||
There was a problem hiding this comment.
How about
| A set of commands to generate and compare local _experiments_: | |
| A set of commands to generate and manage <abbr>experiments</abbr>: |
? (which would also need a glossary entry in content\docs\user-guide\basic-concepts)
There was a problem hiding this comment.
p.s. The keyword/term here being "experiment management".
There was a problem hiding this comment.
BUT this brings up another issue which is: how do we clearly distinguish between regular experiments (e.g. in https://dvc.org/doc/start/experiments) and the special experiments enabled by this feature? Do we need to come up with terminology like "DVC experiment" and use it consistently (along with the tooltip I suggested above)? Cc @shcheklein
There was a problem hiding this comment.
The terminology is definitely an issue, we could maybe use the "ephemeral experiment vs formal/committed experiment" wording that was suggested way back in the original discussion.
There was a problem hiding this comment.
Briefly discussed this with Ivan and it turns out there may not be a need to disambiguate: all experiments are experiments, it's just that some are committed. It's more a distinction in the way they are managed, but we can leave that explanation to an "Experiment Management" guide or use case that we should write (not in this PR of course).
@pmrowla do you know what is missing or outdates in the wiki? Are those things already covered in this PR? Would be great to get help from you to update the wiki or list what else has changed. I can discover some things during QA (doing now) but may miss some things. |
For the command reference everything in this PR should be up-to-date with the wiki, but checkpoints usage is going to be changing again this sprint, so there will be at least one more command reference update coming |
|
OK I'm going to break this PR into several so it's easier to manage. PRs that address my own feedback and other updates coming up finally... See you soon. |
| # exp resume | ||
|
|
||
| Resume (continue) existing checkpoint experiments. |
There was a problem hiding this comment.
This ended up as exp run --resume right? Is there a PR or explanation of the differences? That seems outdated in https://github.com/iterative/dvc/wiki/Experiments too. Feel free to suggest in #2200 (options list of exp run).
See https://github.com/iterative/dvc/wiki/Experiments for ref. (may not be complete)
And https://github.com/iterative/dvc/pulls?q=is%3Apr+label%3Aexperiments+ for related PRs
NOTES (Jorge):