guide: revisit Exp Sharing

[jorgeorpinel]

Re-do of #2711 and #2719
Per #2654 (comment)

• Links to sharing from other docs and other copy edits around the topic
• Introduce simple diagram for explanation about the necessary remotes
• Improve explanations (more specific and direct)
• Remove repeated or unrelated content (covered in other guides) e.g. about dvc exp list
• Extract short example about looping -> turned to a How to page

PENDING:

• Question on long experiment by the end
  Discussion starting at guide: revisit Exp Sharing #2908 (review). -- REMOVED
• Double check Experiments motivation is clear in more general guide pages?
  See guide: revisit Exp Sharing #2908 (review)

[jorgeorpinel]

explaining --remote and --j doesn't make much sense to me here

It's a good question @shcheklein . I think that @iesahin intended to cover all the possible details in the guide so that we don't have them in the Description of the cmd refs. We probably all agreed to do that here, in fact.

But maybe it's OK to leave out less relevant option explanations (only covered in the Options section of each cmd ref.). This improves the docs focus and maintainability. It's unclear how to define relevant though.

I'm removing them from here and the cmd ref descs. for now (a9988b0). WDYT?

if the page is done right there should be one section around diagram...

The intro currently does that. I'm not trying to completely rewrite the guide. The existing structure also makes sense I think.

We should write about dvc exp list

It's already mentioned here, but mainly covered in Comparing Exps. We have an issue to rewrite that guide though, which structure may be too linear/ disconnected.

examples- up to you... separate subsection "Recipes" or something.

TBH I also just left it because it's already there but didn't find it super relevant either (clever though). I'll remove it for now. And we do have a recipes section, the How To! But there should be a clear need for each step-by-step solution there.

[shcheklein]

I think that @iesahin intended to cover all the possible details in the guide so that we don't have them in the Description of the cmd refs.

To my mind details of the options and their behavior for a specific command should be explained in the cmd ref for that command. UG should be for long sections (remotes) or cross command explanations (experiments overview). Or lengthy explanation how things work.

And that doesn't matter, since -j is still not an essential option, I don't see anything on top of cmd ref options description.

[shcheklein]

Sorry, I'm not sure I understand this

DVC Experiments] are directly woven into this commit history.

Let's try to write is simple and very explicit.

regular Git workflow, namely creating Git commits, pushing them to Github or from Github.

It means that experiments become part of the Git history, they are visible in Git, etc. As we mentioned this is not idea since doesn't scale well, creates mess in Git history etc. DVC experiments still rely on Git and Github, but they are not directly attached ... so won't visible ... etc ..

[jorgeorpinel]

Let's try to write is simple and very explicit.

Simplified in 0fb52cb.

[jorgeorpinel]

I think the rest of your suggestions are great but do we want to elaborate so much on general motivation for DVC Experiments in this guide @shcheklein ? Maybe we should separately review that this is clearly included in the Overview page and/or the Running Exps page. (Added check box to PR desc.)

For now the only mention of motivation is "to avoid cluttering everyone's copies of the repo, by default experiments will only exist in the local environment where they were created."

[shcheklein]

. The existing structure also makes sense I think.

Yes, but it's not ideal. Those small subsections could be removed completely - just one paragraph after the diagram is enough . Hey, you have push/pull/list to share experiments this way. Here are the links to read the details (links to the cmd ref).

Thus the whole purpose of this page is the diagram and text around it - that is the most valuable part. We can write more and more details, caveats that are general for that workflow, write advanced stuff how to see git index, etc.

Also, diagram might include exp pull, exp push?

[jorgeorpinel]

The existing structure also makes sense I think.

Yes, but it's not ideal. Those small subsections could be removed completely

I agree with you @shcheklein. I argued for that from the original PR but we decided to merge something. And this PR is about revisiting that guide so it makes sense to bring it up again. Buuuttt that's actually a task in #2911. I actually intended to only improve the guide with it's current structure here. If you don't think this is valuable enough then I can plan to repurpose the PR, but I think merging the work so far would be a good first step (lots of improvements).

diagram might include exp pull, exp push?

Added pull in d6de4f5.

[iesahin]

I tend to diverge from you about the reading habits of users, so I'm including more headers the readers could skim. Unifying those sections might produce a more beautiful document but the users will spend a longer time finding the information they need.

[shcheklein]

I think we diverge here primarily on the content and the focus in the document, not reading habits. For me removing subsections make sense since I don't see much content in them that should stay - we don't it in the user guide in this form (e.g. -j - why only this option? what is the benefit to repeat it this way, etc).

[jorgeorpinel]

TBH in my answer I was thinking more of https://dvc.org/doc/user-guide/experiment-management/comparing-experiments, not the one (sharing exps), sorry for the confusion.

As for reading habits, FWIW personally I usually try to Ctrl+F key words first. Reading the ToC or headers is a probable next strategy but I expect that kind of structure more in the most technical docs. For a guides I'd assume that to really get it you need to read the whole thing or at least a good chunk at the beginning before skipping ahead.

[jorgeorpinel]

p.s. but I have to admit I'm still not sure when we want guides to be more technical vs. refs (or in what sense).