ref: clarifications to exp run

[jorgeorpinel]

• What "based on the latest project version" means.
• --temp isn't useful yet is useful background-ing long experiments (exp run: checks out data dependencies (destructive) [QA] dvc#5593 (comment))
• Other clarifications

[pmrowla]

Maybe we should document that all of the other DVC diff commands and plots commands can also be used with experiments here

[jorgeorpinel]

I'd wait to see if all the show and diff commands get merged 🙂

[pmrowla]

Unification of the show and diff commands is something that probably won't happen for a decent amount of time, and I brought it up because I have seen questions from users in discord asking about how to use plots with experiments.

either way it's not a big deal I guess

[jorgeorpinel]

DVC diff commands and plots commands can also be used with experiments here

Wait. But we mean more like show and compare MANY/ALL experiments here. metrics/params diff only take 2 revs and in those cases exp diff is prob better/easier anyway.

This only really applies to plots diff as of now, I think? Mentioned just that in 29004e6 for now.

[dberenbaum]

I think that approach makes sense @jorgeorpinel.

• Alternatively (or additionally), we could say something like "any dvc command that accepts Git revisions also accepts experiment names."

[dberenbaum]

Agree that "in the background" doesn't seem quite right since temp experiments are blocking once they are run. Otherwise looks good, and I can take one more look once that issue is resolved.

[jorgeorpinel]

Couple more Qs @pmrowla 👇 Thanks!

[jorgeorpinel]

Are exp queues always processed outside the workspace, even with -j 1? And why

[dberenbaum]

To me, "queueing" means setting up an experiment for future execution. To do that, I would expect to add to the queue, then keep making changes to my workspace without impacting the experiment that I already queued. It seems like this workflow would always require processing outside the workspace, right?

[jorgeorpinel]

this workflow would always require processing outside the workspace

At exp run --run-all it could checkout the workspace locally at each queue time, overwriting whatever you have there and forcing you to wait for it to finish.

[dberenbaum]

overwriting whatever you have there

It could do that, but overwriting the workspace isn't ideal, right? I know the current solution isn't perfect either, but avoiding having to overwrite the workspace seems like a big reason why exp queues are always processed outside the workspace (in addition to consistency with parallel execution and possibly remote scenarios in the future).

[jorgeorpinel]

Ah yeah, I agree! I'm just saying it's not necessarily obvious, so we should be explicit about the behavior I think. I guess you were just answering the Q (with a yes). Updated a bit in d8acecb anyway. LGTM!