Feature enhacement : pipelines in DVC VS code extension

[SoyGema]

Motivation

• Improve high-level understanding for the user about pipelines and the relationship between code , dvc.yaml, and params.yaml file without leaving the IDE.

• Place the button to create a stage in a place of Run experiments, and not Show experiments : I hold a rational belief due to the extensive use that the stage creation might not be in the table section, as it is a place for analyzing experiments, not creating anything related to them.

• Tangential but maybe important: think about how users might come to the concept of pipeline in DVC to create a coherent user story.

Proposal

I'm sure that you can improve this one , and not fully convinced myself, but I shall put maybe a diagram or animation about the relationship in between the 3 building blocks

Pipeline.mp4

Thanks for the code snippets and stages structure proposal for dvc.yaml file in #4233 Related to #4194

Tasks

Preview Give feedback

No tasks being tracked yet.

[mattseddon]

@SoyGema would a button that could generate the DAG be useful from this screen?

I think if we were going to use an animation like the one shown in the screen grab we would need to swap the positions of params.yaml and load_dataset.py. I think we would also want to stick with a train example as that is the DVCLive onboarding path that we've been focused on up until this point.

[SoyGema]

Hey @mattseddon ! Thanks for the attention and the quick response 🙏

1. Would a button that could generate the DAG be useful from this screen? Yes, it would, once the pipeline is configured. In fact, it's pretty coherent with the DVC done configuration scenario moment in the UI . Once the DAG might be created, would you please consider reinforcing that it is done or give some stamina to the user for running the experiment ?

2. Swaps. Makes sense to me, especially given that the params.yml file will be the file the user will Iterate from. I would consider automating the params.yml creation as well, or even creating it when the user is focusing on the dvc.yaml file. Please note that the animation is an idea, and you might want to test another approaches ... If so, it would be amazing that it still distills the essence : improve high-level understanding for the user about pipelines and the relationship between code , dvc.yaml, and params.yaml file without leaving the IDE.

See you in the Open! 🛰️

[mattseddon]

We will get to this but we don't have capacity right now 🙏🏻. Thank you @SoyGema.

[dberenbaum]

We should consider adding a visual like this to the dvc pipeline docs. If we can include hydra or other advanced config as well, that would be amazing.