Story: Compare other branches and commits in plots and the experiments table

[dberenbaum]

In the Plots and Experiments views, is it possible to compare the workspace or experiments to past commits or branches? Right now, it seems limited to experiments based on the current commit. For example, in plots it would be nice to be able to enter any git ref here:

• First step Multiple commits: show commit information, other improvements #2458
• Next implement Story: Compare other branches and commits in plots and the experiments table #1966 (comment)

[DavidGOrtega]

Im preparing also an issue regarding to branches 😋

[mattseddon]

Close to a duplicate of #456. I think we only need one of these issues.

[dberenbaum]

I don't think we should be constrained by existing CLI flags here like --all-branches etc. VS Code might be able to provide way more flexibility to pick and choose specific refs from the git tree within the UI. I think this type of functionality could be broadly useful not just in this feature, but in other features like removing unwanted experiments or data.

[sroy3]

#2392 Displays the previous two commits. Next step could be have a "show more commits" at the bottom of the table. It could add two (we can easily fine tune that to the number we want) more commits until there are no more to show.

[daveraghav]

Is there a plan to show more than 2 previous commits to vsc-extension experiments table anytime soon?

[shcheklein]

@daveraghav yes! just as a matter of getting feedback - how many commits you would prefer to see? do you need to see branches?

[daveraghav]

Thanks for the update @shcheklein. Ideally the more the better. Either using 'Load More Commits' or something similar as mentioned @sroy3 or by making it configurable in extension settings. Also showing experiments from other branches would be great as multiple data scientists often work across parallel branches on the same project.

[shcheklein]

Also showing experiments from other branches would be great as multiple data scientists often work across parallel branches on the same project.

Thanks, yes. That's exactly why I'm thinking about allowing people to pick and choose in some way what they want to see (vs trying to present all commits and all branches - too expensive, and too much info). I'm thinking what would be the best way to find /add commits to show. We can introduce a control like this:

• Dropdown to pick a branch, tag, commit sha
• A ribbon with picked items from the above ^^
• Each item on the ribbon can have a number on it - how many parent commits to show (1 by default).

By default the ribbon would always have the "current branch" + 2 commits in it.

[dberenbaum]

I guess the idea is to expand in the experiments table, and then those will be automatically shown as options in the plots view?

My take regarding the table:

• Dropdown to pick a branch, tag, commit sha

I think picking from branches would be enough to start and less overwhelming for users.

#2392 Displays the previous two commits. Next step could be have a "show more commits" at the bottom of the table. It could add two (we can easily fine tune that to the number we want) more commits until there are no more to show.

Agree with @sroy3 that "show more" is simpler and more intuitive over having a configurable number of commits per branch/item.

[dberenbaum]

I guess what I'm wondering is whether it's easier/better to have one giant JSON with everything vs. independent JSON for different branches/commits that could be partially updated, shown in separate tables, etc.

[shcheklein]

Good question, Dave. From my past observations/experience (and that was my assumption) - it's always better to minimize the number of commands that we run (less fragile, faster, etc) + I think implementing these flexible, partial updates, or UI that supports that can also be on a different level of complexity. Again, would love to hear other opinions / thoughts on this.

[sroy3]

Would that work for everyone as a first step?

Could you clarify which branch it would be switching to? how can we show multiple branches? Does it mean that we run two dvc exp show commands?

It's actually all branches. Just like the DVC output I've posted.

[shcheklein]

It's actually all branches. Just like the DVC output I've posted.

I see. I think it can become very expensive and noisy tbh. I think we need to have a way to pick a brach(es) and show them. Implementation-wise if it's not a huge effort we can go with multiple commands, eventually migrate to a single one.

[sroy3]

It's actually all branches. Just like the DVC output I've posted.

I see. I think it can become very expensive and noisy tbh. I think we need to have a way to pick a brach(es) and show them. Implementation-wise if it's not a huge effort we can go with multiple commands, eventually migrate to a single one.

There are currently no commands to show only one branch (unless I've misread the docs). I don't think it'd be that expensive. There aren't usually that many branches in a project. There could potentially be an almost infinite number of commits, but the number of branches should stay relatively low.

[shcheklein]

Okay, it can simplify a bit the initial step (no need a mechanism to pick a branch). We still want to show multiple commits per branch + we'll need almost as a next step a way to hide branches that are not relevant (I have repos with 10+ branches, in ML people use branches in some cases extensively for experimentation, etc). So, I'm not sure we are saving anything - it's just a bit different approach, but we'll end up in the same place - ability to pick which branches / tags to show + some history in some cases + show it within a single table.

[daavoo]

There are currently no commands to show only one branch (unless I've misread the docs).

The --rev option can be used to show only experiments derived from one branch. For example dvc exp show --rev try-large-dataset in https://github.com/iterative/example-get-started

[dberenbaum]

@sroy3 Just a note to please not worry too much about what dvc exp show does. If you have other ideas for UI/UX, I'm sure we can either build it into dvc exp show or find some creative solution to show the right rows.

[mattseddon]

@sroy3 Just a note to please not worry too much about what dvc exp show does. If you have other ideas for UI/UX, I'm sure we can either build it into dvc exp show or find some creative solution to show the right rows.

Especially with iterative/dvc#9170 getting integrated soon.

[sroy3]

Branches view was released with 0.7.1. Here is what it looks like:
https://www.loom.com/share/50307b14f4c04f429ee5570f1dcc93ee

Currently working on adding individual branches with previous commits.

[sroy3]

With iterative/dvc#9390 being closed, we'll be able to use one dvc call instead of one per branch.

[sroy3]

#3827 is currently in review.

Since that PR touches a lot of the data, I'll wait until this gets merged before implementing the multiple branches in a single call (iterative/dvc#9391 (comment)).

I'll also do any follow-ups linked to user experience first, as the new call is just a convenience for us and not visible to the user.

Some follow-ups that I currently have on my list are:

• Clean up the table on branch change
• Move the actions in the table around (find the right place for them)

There will probably be more after the review, or feel free to add more.