exp save: save workspace state to an experiment ref

[dberenbaum]

dvc exp commit would create an experiment from the current state of the workspace.

Use cases for this command include:

• Not using DVC pipelines but still want to track experiments in a lightweight way. For example, by just having params in params.yaml and metrics in metrics.json (see Default metrics file #6168), DVC can track and compare experiments.
• Code has already been run. The user might have already run code and wants to record it without committing it to their working branch.
• Want to examine the results before deciding whether to save it as an experiment. Users can do dvc repro and do dvc exp commit whenever they want to track the experiment results.
• Enables users to use dvc repro and still use experiments. Some users may prefer this because they are already used to dvc repro, and in other cases dvc exp run might not work quite the same way. This eases the need to have dvc repro and dvc exp run behave exactly the same (see ref: which repro options does exp run support? dvc.org#2861).

[daavoo]

* Enables users to use `dvc repro` and still use experiments. Some users may prefer this because they are already used to `dvc repro`, and in other cases `dvc exp run` might not work quite the same way. This eases the need to have `dvc repro` and `dvc exp run` behave exactly the same (see [ref: which `repro` options does `exp run` support? dvc.org#2861](https://github.com/iterative/dvc.org/issues/2861)).

Should we support checkpoints in dvc repro or would this dvc exp commit workflow not suitable for checkpoints?

[dberenbaum]

Each checkpoint is basically doing the same as dvc exp commit would do, right? I'm not sure what the behavior would be at each checkpoint in dvc repro.

[shcheklein]

One minor thought on this. When I initially read about this I thought about dvc commit + git commit semantics. While this is more like dvc exp stash? or dvc exp save, right? I guess, my concern is that commit is so overloaded that might confuse people.

[daavoo]

Each checkpoint is basically doing the same as dvc exp commit would do, right? I'm not sure what the behavior would be at each checkpoint in dvc repro.

I meant that, currently, you can't run dvc repro if checkpoints are enabled.

[dberenbaum]

One minor thought on this. When I initially read about this I thought about dvc commit + git commit semantics. While this is more like dvc exp stash? or dvc exp save, right? I guess, my concern is that commit is so overloaded that might confuse people.

Hmm, agreed. I like dvc exp save. It's less tangled in git lingo and more obvious to users.

[dberenbaum]

I meant that, currently, you can't run dvc repro if checkpoints are enabled.

What would you expect to happen at each checkpoint with dvc repro?

[dberenbaum]

@daavoo Let's assume for now at least that this workflow does not support checkpoints. However, I do think it should be possible to create a checkpoint within any workflow, including without using pipelines at all. Related: #7381.

[dberenbaum]

• Not using DVC pipelines but still want to track experiments in a lightweight way.

Need to prioritize this feature request to support this workflow as a lightweight way to get started with experiments.