ref: note that queued exps don't have hash or name yet (in exp show)

[aguschin]

Wanted to share something I struggled a bit while trying out Experiments. For a new user it could be not clear that dvc exp run --queue produces an experiment with a star behind an experiment name, for example *123456, that becomes something like 1618ad3 [exp-c043c] after dvc exp run --run-all. It would be great to add dvc exp show output for clarity and add a few words on why the hashes change (from *123456 to 1618ad3 [exp-c043c]) just after the code snippet with dvc exp run --queue ... here https://dvc.org/doc/start/experiments

[shcheklein]

cc @iesahin

[jorgeorpinel]

For a new user it could be not clear that dvc exp run --queue produces an experiment with a star behind an experiment name, for example *123456

We don't introduce exp show until the next section though. Can you elaborate on why you struggled with that @aguschin ? Maybe you were jumping ahead 🤓

I'm not sure that we want to explain that specific detail about exp show tables here TBH. https://dvc.org/doc/command-reference/exp/show could mention this though, and/or show an example with queued exps.

p.s. I'd shorten that list of queued runs to max 3 though... cc @iesahin 😬 (per #2491)

[dberenbaum]

🤔 Good point, @jorgeorpinel.

@aguschin Keep in mind that a major focus of this section is to keep it as short and simple as possible, so is this important enough for a first impression of experiments?

[jorgeorpinel]

something I struggled a bit while trying out Experiments

BTW this feedback is still very valuable. Please do elaborate so we can think of an alternative way to improve on it.

[aguschin]

@dberenbaum, yes, for the first impression it is enough, I think.

@jorgeorpinel, I was confused at the point I started to use dvc exp locally with the example repo. I expected that the queued experiment that the exp show mentions as lets say *123456 will still be mentioned as something with 123456 after exp run --all-runs in exp show, not something completely different (1618ad3 [exp-c043c]). For example, 123456 [exp-c043c] or 1618ad3 [exp-123456], or something like that.

First I though it was a bug and my experiment was lost somewhere, but after a few tries I got it that this "reference" to experiment in the table changes. Ideally it would be great to change the mechanism this references are assigned to keep it the same, but it's not going to happen now iterative/dvc#6050. I think it's still good to elaborate it somewhere so the user could have an understanding what happens here right after reading the docs :)

[iesahin]

I used -n parameter in the examples more to avoid this confusion in #2497

[jorgeorpinel]

that the queued experiment that the exp show mentions as lets say *123456 will still be mentioned as something with 123456 after exp run --all-runs in exp show
First I though it was a bug and my experiment was lost somewhere...

I see! That's because they don't have actual hashes or ref. names before they run (they're not commits yet).

but it's not going to happen now iterative/dvc#6050. I think it's still good to elaborate it somewhere

Yes I agree we should mention this somewhere. Just probably not here: shouldn't be confusing as exp show hasn't been introduced yet. Let me think 🤔

I used -n parameter in the examples more to avoid this confusion

@iesahin I tried it locally and the name is still not shown by exp show so that wouldn't help 🙁 (for now, see iterative/dvc#6050 (comment))

$ dvc exp run --queue -n myname
$ dvc exp show --no-pager
┏━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━┓
┃ Experiment   ┃ Created      ┃
┡━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━┩
│ workspace    │ -            │
│ master       │ May 22, 2021 │
│ └── *5072852 │ 01:22 PM     │
└──────────────┴──────────────┘

[jorgeorpinel]

I agree we should mention this somewhere

There's a note in https://dvc.org/doc/command-reference/exp/run#queueing-and-parallel-execution that could be updated: "dvc exp show will mark queued experiments with an asterisk *."

And I'd also introduce a note or section about queued/tmp exps in https://dvc.org/doc/command-reference/exp/show

@aguschin please lmk if you're still interested in submitting a draft PR on this (we're happy to take it over after a round of review).

[dberenbaum]

Closing as outdated (queued experiments now get regular exp names)