guide: update experiments guide

[dberenbaum]

Report

Meta-issue to track changes needed for the exp guide:

• Add page/section for params and metrics (close guide: Params, Plots and Metrics #2572)
• Add page/section for artifacts?
• Running (close guide: Running Experiments improvements #2768)
  • Clarify distinction between running in Python and using exp run
  • Explain why to use exp run -- pipelines, run-cache, tuning, queuing, grid search
  • Show grid search
  • Deemphasize checkpoints
  • Show VS Code examples
  • Include cloud experiments
• Comparing (close guide: "Comparing Experiments" improvements #2995)
  • Focus on exp show, VS Code, and Studio
  • Deemphasize exp list
• Sharing
  • High-level workflow for live sharing, pushing, persisting with branch/PR
  • Studio examples
• Add page/section for advanced experiment configuration?
  • Templating, dict unpacking, advanced dvc.yaml features (close ref: exp run --set-param with templating #2313)
  • Hydra composition
• Combined page for managing experiments (sharing/persisting/cleaning)?
  • guide: sharing experiments should include how to manage in studio #4626
• Drop checkpoints guide?
• Incorporate Studio and VS Code more
• Emphasize how experiments save you from managing git commits

[daavoo]

Add page/section for params and metrics (close
#2572)
Add page/section for artifacts?

🤔 Does it need to be tied to the experiments guide?

At this point, I feel like those concepts are not really worth a user guide but rather a good reference (the user guide always ends ups being a reference anyhow) so maybe individual sections of #4381

Add page/section for advanced experiment configuration?

👍

Combined page for managing experiments (sharing/persisting/cleaning)?

👍

Drop checkpoints guide?

👍

[dberenbaum]

🤔 Does it need to be tied to the experiments guide?

At this point, I feel like those concepts are not really worth a user guide but rather a good reference (the user guide always ends ups being a reference anyhow) so maybe individual sections of #4381

For params and metrics, a page is probably too much, but I was thinking we should somewhere introduce the concept of how to do "advanced" params and metrics that don't use DVCLive. That could then point to the reference for details. WDYT?

For artifacts, I think it depends on what @aguschin is working on and how much we want to incorporate that into the experiments workflow.

[daavoo]

I think it would be also nice to explicitly mention and link the run cache in the guide

[dberenbaum]

@daavoo Would you make it part of Explain why to use exp run -- pipelines, tuning, queuing, grid search or a separate item somewhere else?

[daavoo]

@daavoo Would you make it part of Explain why to use exp run -- pipelines, tuning, queuing, grid search or a separate item somewhere else?

An item in that list would make sense

[dberenbaum]

Combined page for managing experiments (sharing/persisting/cleaning)?

@daavoo @omesser We have this in the get started now with https://dvc.org/doc/start/experiments/experiment-collaboration. What do you think we need in the guide? Should we frame it as "collaboration"? Or maybe something like "collaboration with git"? I think a lot of this is about showing how you can use experiments in your git workflows to do more than track and compare them. For example, you can apply/checkout, push/share, or do more advanced git workflows like branch/merge on top of other changes.

[dberenbaum]

* [x]  Add page/section for params and metrics (close [guide: Params, Plots and Metrics #2572](https://github.com/iterative/dvc.org/issues/2572))

* [x]  Add page/section for artifacts?

These are both at least somewhat addressed now in https://dvc.org/doc/user-guide/experiment-management.

[dberenbaum]

• Add page/section for advanced experiment configuration?

I'd like to revisit this after looking at pipelines since it crosses both areas. Opened #4670.