exp show: Add parallel coordinates plot. #6933

daavoo · 2021-11-05T19:38:05Z

❗ I have followed the Contributing to DVC checklist.
📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.

Closes #4455

Introduces the following new arguments / behavior to dvc exp show:

--html 🤷
--sort-by {foo} Defines the column to create colorscale from (Uses Experiment as default)
--out {foo}
--open

Examples

Using https://github.com/iterative/demo-fashion-mnist

Interactivity

Grabacion.de.pantalla.2021-11-30.a.las.13.28.45.mp4

Default: dvc exp show -T --html

Customize color: dvc exp show -T --html --sort-by accuracy

Interaction with other options: dvc exp show -T --html -sort-by accuracy --exclude-metrics loss

dvc/command/experiments.py

dberenbaum · 2021-11-18T18:51:58Z

My biggest question is whether this belongs in dvc exp show or might be better as a separate command, like dvc exp plot? A dvc exp plot command could also potentially show other things, like dvc plots diff between experiments, or checkpoint trends (like dvclive plots).

dberenbaum

Looks great!

One other comment: behavior seems kind of odd where a value is missing, like the points show up in unpredictable patterns. Would it make sense to have a set value for missing data so it all gets grouped together? Not sure if this is possible for numeric fields.

dvc/command/experiments.py

tests/func/experiments/test_show.py

dvc/command/experiments.py

casperdcl · 2021-12-02T16:52:56Z

just to check - the colourbar missing a label in the video/images :)

daavoo · 2021-12-02T17:44:39Z

just to check - the colourbar missing a label in the video/images :)

Fixed and pushed. Nothing like a good Q/A demo in All Hands

New renderer based on plotly. Not exposed to `dvc plots`. Generate plotly datapoints from `TabularData`. pre-requisite #4455

Uses `ParallelCoordinatesRenderer` and `dvc.render.html.write`. pre-requisite #4455

Uses `TabularData.to_parallel_coordinates`. Adds new arguments: `html`, `out`, `open` Reuses `--sort-by` to define colorscale. Closes #4455

Adds new argument `fill_value` to ParallelCoordinatesRenderer. For scalar columns, prevent converting to categorical by explictly inserting None when value is missing (value == fill_value). For categorical columns, replace `fill_value` with `"Missing"` and ensure it is the last item so it gets rendered on top of the column.

jorgeorpinel · 2021-12-16T18:11:41Z

--html 🤷

Why something as generic and conceptually disconnected as "HTML"? How about --parallel-coordinates-plot/--pcp? Or a separate command as @dberenbaum mentioned

jorgeorpinel · 2021-12-16T18:25:17Z

--out {foo}

I understand the default here is dvc_plots/ but dvc plots (show) uses . by default. Seems inconsistent?

daavoo · 2021-12-16T18:30:12Z

It is the same for plots. At least in current version, DVC pots are saved in dvc_plots

jorgeorpinel · 2021-12-16T18:32:34Z

According to https://dvc.org/doc/command-reference/plots/show#options dvc plots show uses the same path as the source image or data series file.

It would be more consistent to just use the cwd . I think, as we may be in a subdirectory or something. dvc_plots/ seems to assume we're in the root of the repo.

Or are the docs outdated?

jorgeorpinel · 2021-12-16T18:36:45Z

Also, the choice of dvc_plots/ by default reinforces the argument that this feature should be in dvc plots exp --pcp or maybe dvc exp plots --pcp (exp. if we plan to have more exp-specific plots) IMO. But maybe I'm missing a discussion that already happened (lots of collapsed comments above).

shcheklein · 2021-12-17T05:59:51Z

To be honest --html for me would mean an HTML table, not plot. I would consider renaming this to be explicit about its meaning. I think --plot is a bit more explicit. But can limit us later if we decide to introduce more types of plots. Probably --pcp is the best unfortunately.

daavoo · 2021-12-17T10:18:16Z

Or are the docs outdated?

Yes they are iterative/dvc.org#3110

daavoo self-assigned this Nov 5, 2021

daavoo added diff/show Related to the diff/show feature A: experiments Related to dvc exp feature is a feature labels Nov 5, 2021

daavoo force-pushed the parallel-coordinates branch 2 times, most recently from 974ef89 to ffbb688 Compare November 12, 2021 16:03

daavoo force-pushed the parallel-coordinates branch 2 times, most recently from 6560267 to 823a6f7 Compare November 15, 2021 20:24

daavoo changed the title ~~dvc exp: Add parallel coordinates plot~~ exp show: Add parallel coordinates plot. Nov 15, 2021

daavoo force-pushed the parallel-coordinates branch 4 times, most recently from 6c86682 to d8c77da Compare November 16, 2021 13:17

daavoo marked this pull request as ready for review November 16, 2021 16:35

daavoo requested a review from a team as a code owner November 16, 2021 16:35

daavoo requested review from karajan1001 and dberenbaum November 16, 2021 16:35

karajan1001 reviewed Nov 17, 2021

View reviewed changes

dvc/command/experiments.py Outdated Show resolved Hide resolved

dvc/command/experiments.py Outdated Show resolved Hide resolved

dvc/command/experiments.py Outdated Show resolved Hide resolved

daavoo mentioned this pull request Nov 17, 2021

ui: Add open_browser. #6993

Merged

karajan1001 requested a review from pared November 18, 2021 10:51

daavoo force-pushed the parallel-coordinates branch from d8c77da to 7fe01e3 Compare November 18, 2021 18:46

dberenbaum reviewed Nov 18, 2021

View reviewed changes

dvc/command/experiments.py Outdated Show resolved Hide resolved

dvc/command/experiments.py Outdated Show resolved Hide resolved

dvc/command/experiments.py Outdated Show resolved Hide resolved

dvc/command/experiments.py Outdated Show resolved Hide resolved

daavoo force-pushed the parallel-coordinates branch from 7fe01e3 to 9de1e5f Compare November 18, 2021 19:05

This comment has been minimized.

Sign in to view

pared suggested changes Nov 19, 2021

View reviewed changes

tests/func/experiments/test_show.py Show resolved Hide resolved

This comment has been minimized.

Sign in to view

daavoo commented Nov 29, 2021

View reviewed changes

dvc/command/experiments.py Outdated Show resolved Hide resolved

daavoo force-pushed the parallel-coordinates branch 2 times, most recently from cc45fc3 to 140c719 Compare December 1, 2021 16:16

dberenbaum approved these changes Dec 1, 2021

View reviewed changes

daavoo mentioned this pull request Dec 2, 2021

dvc exp show: Add mutually exclusive groups. #7082

Closed

pared approved these changes Dec 2, 2021

View reviewed changes

daavoo force-pushed the parallel-coordinates branch from 43ca84e to e982103 Compare December 2, 2021 17:43

daavoo force-pushed the parallel-coordinates branch from d47cb1a to e1f47bd Compare December 2, 2021 17:46

This was referenced Dec 2, 2021

Story: Parallel coordinates plot iterative/vscode-dvc#1117

Closed

plots: Add constant to handle default output directory. #7088

Closed

daavoo force-pushed the parallel-coordinates branch from e1f47bd to 8a96d24 Compare December 15, 2021 11:04

daavoo added 4 commits December 15, 2021 12:42

dvc render: Add plotly parallel coords renderer

ad6bda6

New renderer based on plotly. Not exposed to `dvc plots`. Generate plotly datapoints from `TabularData`. pre-requisite #4455

TabularData: Add to_parallel_coordinates.

54b842d

Uses `ParallelCoordinatesRenderer` and `dvc.render.html.write`. pre-requisite #4455

exp show: Add parallel coordinates plot.

4b58305

Uses `TabularData.to_parallel_coordinates`. Adds new arguments: `html`, `out`, `open` Reuses `--sort-by` to define colorscale. Closes #4455

daavoo force-pushed the parallel-coordinates branch from 8a96d24 to a12e3ce Compare December 15, 2021 11:44

daavoo mentioned this pull request Dec 16, 2021

Parallel coordinates plot iterative/dvc.org#3105

Merged

1 task

pmrowla merged commit 4d5be46 into main Dec 17, 2021

pmrowla deleted the parallel-coordinates branch December 17, 2021 05:45

daavoo mentioned this pull request Dec 17, 2021

exp show: Rename / Split to separate command the current --html #7165

Closed

dberenbaum mentioned this pull request Apr 7, 2022

UI suggestions iterative/gto#107

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

exp show: Add parallel coordinates plot. #6933

exp show: Add parallel coordinates plot. #6933

daavoo commented Nov 5, 2021 •

edited

Loading

dberenbaum commented Nov 18, 2021

dberenbaum left a comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

casperdcl commented Dec 2, 2021

daavoo commented Dec 2, 2021 •

edited

Loading

jorgeorpinel commented Dec 16, 2021

jorgeorpinel commented Dec 16, 2021 •

edited

Loading

daavoo commented Dec 16, 2021 •

edited by jorgeorpinel

Loading

jorgeorpinel commented Dec 16, 2021 •

edited

Loading

jorgeorpinel commented Dec 16, 2021 •

edited

Loading

shcheklein commented Dec 17, 2021

daavoo commented Dec 17, 2021

exp show: Add parallel coordinates plot. #6933

exp show: Add parallel coordinates plot. #6933

Conversation

daavoo commented Nov 5, 2021 • edited Loading

Examples

dberenbaum commented Nov 18, 2021

dberenbaum left a comment

Choose a reason for hiding this comment

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

casperdcl commented Dec 2, 2021

daavoo commented Dec 2, 2021 • edited Loading

jorgeorpinel commented Dec 16, 2021

jorgeorpinel commented Dec 16, 2021 • edited Loading

daavoo commented Dec 16, 2021 • edited by jorgeorpinel Loading

jorgeorpinel commented Dec 16, 2021 • edited Loading

jorgeorpinel commented Dec 16, 2021 • edited Loading

shcheklein commented Dec 17, 2021

daavoo commented Dec 17, 2021

daavoo commented Nov 5, 2021 •

edited

Loading

daavoo commented Dec 2, 2021 •

edited

Loading

jorgeorpinel commented Dec 16, 2021 •

edited

Loading

daavoo commented Dec 16, 2021 •

edited by jorgeorpinel

Loading

jorgeorpinel commented Dec 16, 2021 •

edited

Loading

jorgeorpinel commented Dec 16, 2021 •

edited

Loading