dvclive 1.0 (#4085)

* initial refactor * Mention API Reference * Fix link * Add redirect * Update content/docs/dvclive/how-to/checkpoints.md Co-authored-by: Dave Berenbaum <[email protected]> * Remove how-to folder * Add outputs content * Reorder sidebar * Add tabs with popular frameworks * Remove outdated `dvclive with DVC` * clean * Add params * Clarify value without DVC * Update outputs to current status * Update content/docs/dvclive/get-started.md Co-authored-by: Dave Berenbaum <[email protected]> * Reduce wording * Add Python API tab * More details * Updates from review * Capitalize pages * Mention studio for sharing experiments * New section for Studio * Add sample output * Move ml-frameworks to api-refeference * Drop resume page * Apply suggestions from code review Co-authored-by: Dave Berenbaum <[email protected]> * Restyled by prettier (#4050) Co-authored-by: Restyled.io <[email protected]> * Suggestions from code review * caps * Add sections * Add example output * Add admon * Share experiment -> Share Results * Update output structure. Per iterative/dvclive#322 * Revisit output names * path -> dir * Add make_report call * Rename log -> log_metric * Update log_sklearn_plot. (#4074) Per iterative/dvclive#339 * Apply suggestions from code review Co-authored-by: Jorge Orpinel <[email protected]> * Replace dvc with cli * Remove tree * Update output dir in frameworks * grammarly * Revisit frameworks naming. * Remplaze `path` with `dir` * Rename `outputs` to `how-it-works` * Rename `Output Folder Structure` to `How it Works` * Rename Scalars to Metrics * Simplify `Live` Attributes description * Renamed folder to directory * Revisit `step` definition * Fix tree * Remove cat * Replace vscode plots with data series * Updates with new summary behavior and step property * Fix links * Consistent framework tabs * Update content/docs/dvclive/api-reference/live/log_metric.md * start: dvclive 1.0 updates to exp viz page * Update content/docs/start/experiment-management/visualization.md Co-authored-by: David de la Iglesia Castro <[email protected]> * Revisit dvclive usage outside doc/dvclive * Focus on context manager usage. * hint about next_step * Update content/docs/dvclive/api-reference/live/next_step.md Co-authored-by: Dave Berenbaum <[email protected]> * Link for `Live.metrics_file` * Restyled by prettier (#4097) Co-authored-by: Restyled.io <[email protected]> * Updates per iterative/example-repos-dev#143 Co-authored-by: Dave Berenbaum <[email protected]> Co-authored-by: restyled-io[bot] <32688539+restyled-io[bot]@users.noreply.github.com> Co-authored-by: Restyled.io <[email protected]> Co-authored-by: Jorge Orpinel <[email protected]> Co-authored-by: rogermparent <[email protected]>
iterative · Nov 5, 2022 · 4c3c1ff · 4c3c1ff
1 parent 9da3511
commit 4c3c1ff
Show file tree

Hide file tree

Showing 34 changed files with 679 additions and 654 deletions.
diff --git a/content/docs/command-reference/exp/init.md b/content/docs/command-reference/exp/init.md
@@ -148,8 +148,8 @@ See the [Pipelines guide] for more on that topic.
   default value (`plots/`).
 
 - `--live` - set the path to the directory where the metrics and plots
-  [produced by DVCLive](https://dvc.org/doc/dvclive/dvclive-with-dvc#outputs)
-  will be found. Overrides the default values for `--metrics` and `--plots`.
+  [produced by DVCLive](https://dvc.org/doc/dvclive/how-it-works) will be found.
+  Overrides the default values for `--metrics` and `--plots`.
 
 - `--explicit` - do not assume default locations of project dependencies and
   outputs. You'll have to provide specific locations via other options or

diff --git a/content/docs/command-reference/plots/index.md b/content/docs/command-reference/plots/index.md
@@ -33,7 +33,7 @@ Plots have to be defined either at the stage level, or at the pipeline level in
 `dvc.yaml`.
 
 [ml pipeline]: /doc/start/data-management/data-pipelines
-[dvclive]: /doc/dvclive/dvclive-with-dvc
+[dvclive]: /doc/dvclive
 
 <admon icon="book">
 

diff --git a/content/docs/dvclive/api-reference/index.md b/content/docs/dvclive/api-reference/index.md
@@ -20,9 +20,7 @@ to its corresponding page.
 ### Initialize DVCLive
 
 ```python
-from dvclive import Live
-
-live = Live()
+with Live() as live:
 ```
 
 See [`Live()`](/doc/dvclive/api-reference/live) for details.
@@ -48,13 +46,13 @@ live.log_params(params)
 See `Live.log_param()` / `Live.log_params()`.
 
 </tab>
-<tab title="Scalars">
+<tab title="Metrics">
 
 ```python
-live.log("acc", 0.9)
+live.log_metric("acc", 0.9)
 ```
 
-See `Live.log()`.
+See `Live.log_metric()`.
 
 </tab>
 <tab title="Images">
@@ -67,15 +65,15 @@ live.log_image("image.png", img)
 See `Live.log_image()`.
 
 </tab>
-<tab title="Plots">
+<tab title="SKLearn Plots">
 
 ```python
 y_true = [0, 0, 1, 1]
 y_pred = [0.2, 0.5, 0.3, 0.8]
-live.log_plot("roc", y_true, y_score)
+live.log_sklearn_plot("roc", y_true, y_score)
 ```
 
-See `Live.log_plot()`.
+See `Live.log_sklearn_plot()`.
 
 </tab>
 </toggle>
@@ -86,7 +84,14 @@ See `Live.log_plot()`.
 live.next_step()
 ```
 
-See `Live.next_step()` and `Live.set_step()` for details.
+See `Live.next_step()`.
+
+Under the hood, `Live.next_step()` calls `Live.make_summary()` and
+`Live.make_report()`.
+
+If you want to decouple the `step` update from the rest of the calls, you can
+manually modify the `Live.step` property and call `Live.make_summary()` /
+`Live.make_report()`.
 
 ## Putting it all together
 
@@ -106,13 +111,13 @@ for epoch in range(NUM_EPOCHS):
     metrics = evaluate_model(...)
 
     for metric_name, value in metrics.items():
-        live.log(metric_name, value)
+        live.log_metric(metric_name, value)
 
     live.next_step()
 ```
 
 ## Outputs
 
 After you run your training code, all the logged data will be stored in the
-`dvclive` folder. Check the [DVCLive outputs](/doc/dvclive/outputs) page for
-more details.
+`dvclive` directory. Check the [DVCLive outputs](/doc/dvclive/how-it-works) page
+for more details.
diff --git a/content/docs/dvclive/api-reference/live/get_step.md b/content/docs/dvclive/api-reference/live/get_step.md
diff --git a/content/docs/dvclive/api-reference/live/index.md b/content/docs/dvclive/api-reference/live/index.md
@@ -7,7 +7,7 @@ class Live:
 
     def __init__(
         self,
-        path: Optional[str] = None,
+        dir: str = "dvclive",
         resume: bool = False,
         report: Optional[str] = "auto",
     ):
@@ -18,7 +18,8 @@ class Live:
 ```py
 from dvclive import Live
 
-live = Live()
+with Live() as live:
+    ...
 ```
 
 ## Description
@@ -28,52 +29,55 @@ metrics and other metadata.
 
 <admon type="warn">
 
-`Live()` will remove all existing DVCLive related files under `path` unless
+`Live()` will remove all existing DVCLive related files under `dir` unless
 `resume=True`.
 
 </admon>
 
-## Attributes
+You can use `Live()` as a context manager. When exiting the context manager,
+`Live.make_summary()` and `Live.make_report()` will be called.
+
+## Properties
+
+- `step` - See `Live.next_step()`.
+
+- `summary` - See `Live.make_summary()`.
 
 - `dir` - Location of the directory to store
-  [outputs](/doc/dvclive/get-started#outputs).
+  [outputs](/doc/dvclive/how-it-works).
 
-- `summary_path` - `{Live.dir}.json`. Location of the
-  [summary](/doc/dvclive/api-reference/live/log#description).
+- `metrics_file` - `{Live.dir}/metrics.json`.
 
-- `report_path` - `{Live.dir}/report.{format}`. Location of the
-  [metrics report](/doc/dvclive/api-reference/live/make_report). The `format`
-  can be HTML or Markdown depending on the value of the `report` parameter.
+- `params_file` - `{Live.dir}/params.yaml`.
 
-## Parameters
+- `plots_dir` - `{Live.dir}/plots`.
 
-- `path` - Where to save DVCLive's outputs. _Default_: `None`. If `None`,
-  `"dvclive"` will be used.
+- `report_file` - `{Live.dir}/report.{format}`. The `format` can be HTML
+  (`.html`) or Markdown (`.md`) depending on the value of the `report`
+  parameter.
 
-- `resume` - If `True`, DVCLive will try to read the previous `step` from the
-  `path` directory and start from that point. _Default_: `False`.
+## Parameters
 
-  <admon type="info">
+- `dir` - Where to save DVCLive's outputs. _Default_: `dvclive`.
 
-  If you are not using steps, don't set `resume=True` since DVCLive will
-  preserve previous run's files and assume that `step` has been enabled.
+- `resume` - If `True`, DVCLive will try to read the previous `step` from the
+  `metrics_file` and start from that point. _Default_: `False`.
 
-  </admon>
+- `report` - Any of `auto`, `html`, `md` or `None`. See `Live.make_report()`.
 
-- `report` - If `auto`, `html`, or `md`, DVCLive will call `Live.make_report()`
-  on each step update. The `auto` mode (default) will use `md` format if a `CI`
-  env var is present, otherwise it will use `html`.
+  The `auto` mode (default) will use `md` format if the `CI` env var is present
+  and [matplotlib](https://matplotlib.org/) is installed, otherwise it will use
+  `html`.
 
-  If `report` is `None`, `Live.make_report()` won't be called automatically.
+  If `report` is `None`, `Live.make_report()` won't generate anything.
 
 ## Methods
 
-- `Live.get_step()`
-- `Live.log()`
 - `Live.log_image()`
+- `Live.log_metric()`
 - `Live.log_param()`
 - `Live.log_params()`
-- `Live.log_plot()`
+- `Live.log_sklearn_plot()`
 - `Live.make_report()`
+- `Live.make_summary()`
 - `Live.next_step()`
-- `Live.set_step()`
diff --git a/content/docs/dvclive/api-reference/live/log.md b/content/docs/dvclive/api-reference/live/log.md
diff --git a/content/docs/dvclive/api-reference/live/log_image.md b/content/docs/dvclive/api-reference/live/log_image.md
@@ -26,48 +26,30 @@ live.log_image("pil.png", img_pil)
 
 Supported values for `val` are:
 
-- A valid numpy array (convertible to image via
+- A valid NumPy array (convertible to image via
   [PIL.Image.fromarray](https://pillow.readthedocs.io/en/stable/reference/Image.html#PIL.Image.fromarray))
 - A `PIL.Image` instance.
 
-The images will be saved in `{Live.dir}/images/{name}`:
+The images will be saved in `{Live.plots_dir}/images/{name}`:
 
-```dvc
-$ tree
-└── dvclive
+```
+dvclive
+└── plots
     └── images
         ├── numpy.png
         └── pil.png
 ```
 
 <admon type="tip">
 
-The logged images can be visualized with `dvc plots`.
-
-</admon>
-
-### Step updates
-
-The first `step` update (with `Live.next_step()` or `Live.set_step()`) will move
-the saved file from `{Live.dir}/images/{name}` to
-`{Live.dir}/images/{step}/{name}`.
+The logged images can be visualized with `dvc plots`:
 
-Each subsequent call to `live.log_image(name, val)` will save the image under
-the folder `{Live.dir}/images/{step}/{name}` corresponding to the current
-`step`.
-
-```dvc
-$ tree
-└── dvclive
-    └── images
-        ├── 0
-        │   ├── numpy.png
-        │   └── pil.png
-        └── 1
-            ├── numpy.png
-            └── pil.png
+```cli
+$ dvc plots diff dvclive/plots
 ```
 
+</admon>
+
 ## Parameters
 
 - `name` - name of the image file that this command will output
@@ -80,4 +62,4 @@ $ tree
   have a supported type.
 
 - `dvclive.error.DataAlreadyLoggedError` - thrown if the provided `name` has
-  already been logged whithin the same `step`.
+  already been logged within the same `step`.