From 54394124db4a9e3c70baaa22297efd7420d86412 Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Mon, 3 Jan 2022 22:12:36 +0100 Subject: [PATCH 01/11] dvclive: Updates for implicit no step. --- .../dvclive/api-reference/live/get_step.md | 2 - .../docs/dvclive/api-reference/live/index.md | 18 ++--- .../docs/dvclive/api-reference/live/log.md | 35 +++++----- .../dvclive/api-reference/live/next_step.md | 6 +- .../dvclive/api-reference/live/set_step.md | 8 +-- content/docs/dvclive/dvclive-with-dvc.md | 6 +- content/docs/dvclive/get-started.md | 65 ++++++++++++------- 7 files changed, 74 insertions(+), 66 deletions(-) diff --git a/content/docs/dvclive/api-reference/live/get_step.md b/content/docs/dvclive/api-reference/live/get_step.md index ab3c5b2f62..8fd704a4be 100644 --- a/content/docs/dvclive/api-reference/live/get_step.md +++ b/content/docs/dvclive/api-reference/live/get_step.md @@ -1,7 +1,5 @@ # Live.get_step() -Returns the current `step` value. - ```py def get_step() -> int: ``` diff --git a/content/docs/dvclive/api-reference/live/index.md b/content/docs/dvclive/api-reference/live/index.md index d7af4d8c08..ea3cb99f38 100644 --- a/content/docs/dvclive/api-reference/live/index.md +++ b/content/docs/dvclive/api-reference/live/index.md @@ -1,7 +1,5 @@ # Live() -Initializes a DVCLive logger. - ```py class Live: @@ -23,25 +21,23 @@ live = Live() ## Description -Its parameters are focused on configuring the behavior of subsequent calls to -[`Live()` methods](#methods). +Initializes a DVCLive logger. ⚠️ `Live()` will remove all existing DVCLive related files under `path` unless `resume=True`. ## Attributes -- `dir` - Location of the - [_metrics logs_](/doc/dvclive/get-started#metrics-logs) directory. +- `dir` - Location of the [linear plots](/doc/dvclive/get-started#linear-plots) + directory. - `summary_path` - Location of the - [_metrics summary_](/doc/dvclive/get-started#metrics-summary). + [summary](/doc/dvclive/get-started#metrics-summary). - `html_path` - Location of the - [_html report_](/doc/dvclive/dvclive-with-dvc#html-report). + [html report](/doc/dvclive/dvclive-with-dvc#html-report). ## Parameters -- `path` - Configure where to save _metrics logs_ and _metrics summary_ - (generated by `Live.log()`). _Default_: `None`. +- `path` - Where to save DVCLive's outputs. _Default_: `None`. If `None` and DVC is enabled (see [DVCLive with DVC](/docs/dvclive/dvclive-with-dvc)), the `path` set by DVC @@ -51,7 +47,7 @@ Its parameters are focused on configuring the behavior of subsequent calls to `path` directory and start from that point. _Default_: `False`. - `summary` - If `True`, upon each `Live.log()` call, DVCLive will generate a - _metrics summary_ (usable by `dvc metrics`). The _summary_ will be located at + summary (usable by `dvc metrics`). The summary will be located at `{path}.json`. _Default_: `True`. ## Exceptions diff --git a/content/docs/dvclive/api-reference/live/log.md b/content/docs/dvclive/api-reference/live/log.md index f1b0dd53ac..9553d41da1 100644 --- a/content/docs/dvclive/api-reference/live/log.md +++ b/content/docs/dvclive/api-reference/live/log.md @@ -1,10 +1,7 @@ # Live.log() -Generates [_metrics logs_](/doc/dvclive/get-started#metrics-logs) (usable by -`dvc plots`) by saving the given `name`: `val` pair to a `.tsv` file. - ```py - def log(name: str, val: float, step: int = None): + def log(name: str, val: float): ``` #### Usage: @@ -19,10 +16,16 @@ live.log("loss", 0.9) ## Description -The first call to `live.log(name, val)` will create a new file in -`{path}/{name}.tsv` including the header and first row. +If `summary` is True, `Live.log()` DVCLive will update the +[summary](/doc/dvclive/get-started#summary) with the latest value logged. + +💡 The summary `{path}.json` is usable by `dvc metrics`. + +### Step updates -For example `live.log("loss", 0.9)` will create `{path}/loss.tsv`: +The first `step` update (with `Live.next_step()` or `Live.set_step()`) will +create a new [linear plot](/doc/dvclive/get-started#linear-plots) in +`{path}/{name}.tsv`: ``` timestamp step loss @@ -32,23 +35,17 @@ timestamp step loss Each subsequent call to `live.log(name, val)` will add a new row to `{path}/{name}.tsv`. -The created file `{path}/{name}.tsv` is usable by `dvc plots`. - -💡 If `name` contains slashes (e.g. `train/loss`), the required subdirectories -will be created and the file will be saved inside the last one (e.g. -`{path}/train/loss.tsv`). - -If `summary` is True, `Live.log()` DVCLive will update the -[_metrics summary_](/doc/dvclive/get-started#metrics-summary) with the latest -value logged. +💡 The linear plot `{path}/{name}.tsv` is usable by `dvc plots`. -The updated summary `{path}.json` is usable by `dvc metrics`. +If `name` contains slashes (e.g. `train/loss`), the required subdirectories will +be created and the file will be saved inside the last one (e.g. +`{path}/train/loss.tsv)`. ## Parameters -- `name` - The _metrics logs_ will be saved in `{path}/{name}.tsv`. +- `name` - Name of the metric being logged. -- `val` - The value to be added in the `name` column of a new row. +- `val` - The value to be logged. ## Exceptions diff --git a/content/docs/dvclive/api-reference/live/next_step.md b/content/docs/dvclive/api-reference/live/next_step.md index 35f4ff9337..59ad66f613 100644 --- a/content/docs/dvclive/api-reference/live/next_step.md +++ b/content/docs/dvclive/api-reference/live/next_step.md @@ -1,7 +1,5 @@ # Live.next_step() -Signals that the current step has ended and increases `step` value by 1 (one). - ```py def next_step() ``` @@ -32,8 +30,8 @@ When `dvclive` is used alongside `DVC`, each `Live.next_step()` call will have additional effects. By default, on each `Live.next_step()` call, `DVC` will prepare an -[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with all the _metrics -logs_ logged in `path`. +[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with all +[linear plots](/doc/dvclive/get-started#linear-plots). In addition, when [checkpoints](/doc/user-guide/experiment-management/checkpoints) are enabled in diff --git a/content/docs/dvclive/api-reference/live/set_step.md b/content/docs/dvclive/api-reference/live/set_step.md index 5fbf439703..a11fbe5e0f 100644 --- a/content/docs/dvclive/api-reference/live/set_step.md +++ b/content/docs/dvclive/api-reference/live/set_step.md @@ -1,7 +1,5 @@ # dvclive.set_step() -Signals that the current step has ended and sets `step` to the given value. - ```py def set_step(step: int): ``` @@ -33,8 +31,8 @@ When `dvclive` is used alongside `DVC`, each `Live.set_step()` call will have additional effects. By default, on each `Live.set_step()` call, `DVC` will prepare an -[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with all the _metrics -logs_ logged in `path`. +[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with all +[linear plots](/doc/dvclive/get-started#linear-plots). In addition, when [checkpoints](/doc/user-guide/experiment-management/checkpoints) are enabled in @@ -45,7 +43,7 @@ the pipeline, `DVC` will ## Example Given the [Usage](#usage) code snippet above, the -[metrics logs](/doc/dvclive/get-started#metrics-logs) generated for `metric_1` +[linear plot](/doc/dvclive/get-started#linear-plots) generated for `metric_1` would be: ```dvc diff --git a/content/docs/dvclive/dvclive-with-dvc.md b/content/docs/dvclive/dvclive-with-dvc.md index 0ea99a1cf0..ab718b85a6 100644 --- a/content/docs/dvclive/dvclive-with-dvc.md +++ b/content/docs/dvclive/dvclive-with-dvc.md @@ -4,7 +4,7 @@ Even though DVCLive does not require DVC, they can integrate in several useful ways: - The [_outputs_](#outputs) DVCLive produces can be fed as - `dvc plots`/`dvc metrics`, making it easier to add metrics logging to DVC + `dvc metrics`/`dvc plots`, making it easier to add metrics logging to DVC stages. Those same outputs can be visualized in [_DVC Studio_](#dvc-studio) - You can monitor model performance in realtime with the @@ -111,6 +111,8 @@ plot for metrics automatically updated during the model training! ![](/img/dvclive-html.gif) +> If you don't update the step number, the HTML report won't be generated. + ### Checkpoints When used alongside DVC, DVCLive can create _checkpoint_ signal files used by @@ -121,3 +123,5 @@ This will save the metrics, plots, models, etc. associated to each You can learn more about how to use them in the [Checkpoints User Guide](/docs/user-guide/experiment-management/checkpoints). + +> If you don't update the step number, checkpoints won't be created. diff --git a/content/docs/dvclive/get-started.md b/content/docs/dvclive/get-started.md index 7452286dcb..0f85f6c69c 100644 --- a/content/docs/dvclive/get-started.md +++ b/content/docs/dvclive/get-started.md @@ -1,12 +1,13 @@ # Get Started -DVCLive is a simple Python library whose interface consists of three main steps. +DVCLive is a simple Python library for logging data in a format usable by DVC. -## Steps +> If you use one of the supported [ML Frameworks](/doc/dvclive/ml-frameworks), +> you can jump directly to it's corresponding page. -To get it up and running you just need to follow these steps: +## Workflow -### 1. Initialize DVCLive +### Initialize DVCLive ```python from dvclive import Live @@ -16,25 +17,37 @@ live = Live() See [`Live()`](/doc/dvclive/api-reference/live) for details. -### 2. Log metrics +### Log data ```python -live.log(metric_name, value) +live.log("acc", 0.9) ``` See `Live.log()` for details. -### 3. Increase the step number +### (Optionally) Update the step number ```python live.next_step() ``` -See `Live.next_step()` for details. +See `Live.next_step()` and `Live.set_step()` for details. + +### No step + +In ML scenarios where there are no iterations (steps) as part of the training +process, there is no need to update the step number. + +When you don't update the step number in your code, DVCLive will **not** +generate: + +- [linear plots](/doc/dvclive/get-started#linear-plots) +- [html report](/doc/dvclive/dvclive-with-dvc#html-report) +- [checkpoints](/doc/dvclive/dvclive-with-dvc#checkpoints) ## Putting all together -Using the above steps, you can easily include DVCLive in your training code: +Joining the above snippets, you can include DVCLive in your training code: ```python # train.py @@ -64,9 +77,25 @@ dvclive train.py dvclive.json ``` -### Metrics Logs +### Summary + +When [`summary`](/doc/dvclive/api-reference/live/#parameters) is enabled (True +by default), DVCLive generates a summary with the latest metrics: + +```dvc +$ cat dvclive.json +{ + "step": 2, + "{metric_name}": 0.8907166719436646, +} +``` + +> If you don't update the step number, the `step` entry won't be present in the +> summary. + +### Linear plots -For each `{metric_name}`, DVCLive produces metrics _logs_ under +In addition, for each `{metric_name}`, DVCLive produces a linear plot under `dvclive/{metric_name}.tsv`: ```dvc @@ -77,19 +106,7 @@ timestamp step {metric_name} 1614129198848 2 0.8907166719436646 ``` -### Metrics Summary - -In addition, when [`summary`](/doc/dvclive/api-reference/live/#parameters) is -enabled (True by default), DVCLive generates a metrics _summary_ with the latest -metrics: - -```dvc -$ cat dvclive.json -{ - "step": 2, - "{metric_name}": 0.8907166719436646 -} -``` +> If you don't update the step number, the Metrics Logs won't be generated. ## What next? From bf9dd60ebc1ec8db9f50cd3b89c6d782e7bdf3d8 Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Fri, 21 Jan 2022 19:02:41 +0100 Subject: [PATCH 02/11] Apply suggestions from code review Co-authored-by: Dave Berenbaum --- content/docs/dvclive/api-reference/live/log.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/content/docs/dvclive/api-reference/live/log.md b/content/docs/dvclive/api-reference/live/log.md index 9553d41da1..c9b4b50754 100644 --- a/content/docs/dvclive/api-reference/live/log.md +++ b/content/docs/dvclive/api-reference/live/log.md @@ -16,7 +16,7 @@ live.log("loss", 0.9) ## Description -If `summary` is True, `Live.log()` DVCLive will update the +If `summary` is True, `live.log(name, val)` will update the [summary](/doc/dvclive/get-started#summary) with the latest value logged. 💡 The summary `{path}.json` is usable by `dvc metrics`. @@ -39,7 +39,7 @@ Each subsequent call to `live.log(name, val)` will add a new row to If `name` contains slashes (e.g. `train/loss`), the required subdirectories will be created and the file will be saved inside the last one (e.g. -`{path}/train/loss.tsv)`. +`{path}/train/loss.tsv`). ## Parameters From 74d1627ba0970253aaa2432bc0d6b71409dd9d3b Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Fri, 21 Jan 2022 19:20:21 +0100 Subject: [PATCH 03/11] remove no-step section --- content/docs/dvclive/get-started.md | 12 ------------ 1 file changed, 12 deletions(-) diff --git a/content/docs/dvclive/get-started.md b/content/docs/dvclive/get-started.md index 0f85f6c69c..0007787cf5 100644 --- a/content/docs/dvclive/get-started.md +++ b/content/docs/dvclive/get-started.md @@ -33,18 +33,6 @@ live.next_step() See `Live.next_step()` and `Live.set_step()` for details. -### No step - -In ML scenarios where there are no iterations (steps) as part of the training -process, there is no need to update the step number. - -When you don't update the step number in your code, DVCLive will **not** -generate: - -- [linear plots](/doc/dvclive/get-started#linear-plots) -- [html report](/doc/dvclive/dvclive-with-dvc#html-report) -- [checkpoints](/doc/dvclive/dvclive-with-dvc#checkpoints) - ## Putting all together Joining the above snippets, you can include DVCLive in your training code: From 43e47a20b9059231834b5d073fdc09380fd14dcd Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Fri, 21 Jan 2022 19:35:49 +0100 Subject: [PATCH 04/11] Update description --- content/docs/dvclive/get-started.md | 3 ++- content/docs/dvclive/index.md | 6 +++--- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/content/docs/dvclive/get-started.md b/content/docs/dvclive/get-started.md index 0007787cf5..b83a44ad31 100644 --- a/content/docs/dvclive/get-started.md +++ b/content/docs/dvclive/get-started.md @@ -1,6 +1,7 @@ # Get Started -DVCLive is a simple Python library for logging data in a format usable by DVC. +DVCLive is a Python library for logging machine learning metrics and other +metadata in simple file formats, which is fully compatible with DVC. > If you use one of the supported [ML Frameworks](/doc/dvclive/ml-frameworks), > you can jump directly to it's corresponding page. diff --git a/content/docs/dvclive/index.md b/content/docs/dvclive/index.md index d37e429cc0..1881adc228 100644 --- a/content/docs/dvclive/index.md +++ b/content/docs/dvclive/index.md @@ -1,8 +1,8 @@ # DVCLive Documentation -[DVCLive](https://github.com/iterative/dvclive) is an open-source Python library -for monitoring the progress of metrics during training of machine learning -models. You can simply +[DVCLive](https://github.com/iterative/dvclive) is a Python library for logging +machine learning metrics and other metadata in simple file formats, which is +fully compatible with DVC. You can [install it](https://github.com/iterative/dvclive#installation) with `pip`. From 3cc8e9c6b2209e09760beec408e815d52c119aa4 Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Fri, 21 Jan 2022 20:19:36 +0100 Subject: [PATCH 05/11] Add no step warning for summary and resume --- content/docs/dvclive/api-reference/live/index.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/content/docs/dvclive/api-reference/live/index.md b/content/docs/dvclive/api-reference/live/index.md index ea3cb99f38..b905cf577b 100644 --- a/content/docs/dvclive/api-reference/live/index.md +++ b/content/docs/dvclive/api-reference/live/index.md @@ -46,10 +46,16 @@ Initializes a DVCLive logger. - `resume` - If `True`, DVCLive will try to read the previous `step` from the `path` directory and start from that point. _Default_: `False`. + ⚠️ Don't change the default value unless you are going to update the `step` + value during training. + - `summary` - If `True`, upon each `Live.log()` call, DVCLive will generate a summary (usable by `dvc metrics`). The summary will be located at `{path}.json`. _Default_: `True`. + ⚠️ Don't change the default value unless you are going to update the `step` + value during training. + ## Exceptions - `dvclive.error.ConfigMismatchError` - thrown if the provided `path` does not From ec6a350fdbe4f9984d01da1ec03bca18ccc0a989 Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Tue, 25 Jan 2022 11:50:39 +0100 Subject: [PATCH 06/11] dvclive: Rename linear plots to metric history --- content/docs/dvclive/api-reference/live/index.md | 2 +- content/docs/dvclive/api-reference/live/log.md | 4 ++-- content/docs/dvclive/api-reference/live/next_step.md | 4 ++-- content/docs/dvclive/api-reference/live/set_step.md | 6 +++--- content/docs/dvclive/dvclive-with-dvc.md | 9 +++++---- content/docs/dvclive/get-started.md | 6 +++--- 6 files changed, 16 insertions(+), 15 deletions(-) diff --git a/content/docs/dvclive/api-reference/live/index.md b/content/docs/dvclive/api-reference/live/index.md index b905cf577b..aa16e6e911 100644 --- a/content/docs/dvclive/api-reference/live/index.md +++ b/content/docs/dvclive/api-reference/live/index.md @@ -28,7 +28,7 @@ Initializes a DVCLive logger. ## Attributes -- `dir` - Location of the [linear plots](/doc/dvclive/get-started#linear-plots) +- `dir` - Location of the [metrics history](/doc/dvclive/get-started#history) directory. - `summary_path` - Location of the [summary](/doc/dvclive/get-started#metrics-summary). diff --git a/content/docs/dvclive/api-reference/live/log.md b/content/docs/dvclive/api-reference/live/log.md index c9b4b50754..7e4071c1c3 100644 --- a/content/docs/dvclive/api-reference/live/log.md +++ b/content/docs/dvclive/api-reference/live/log.md @@ -24,7 +24,7 @@ If `summary` is True, `live.log(name, val)` will update the ### Step updates The first `step` update (with `Live.next_step()` or `Live.set_step()`) will -create a new [linear plot](/doc/dvclive/get-started#linear-plots) in +create a new [metric history](/doc/dvclive/get-started#history) in `{path}/{name}.tsv`: ``` @@ -35,7 +35,7 @@ timestamp step loss Each subsequent call to `live.log(name, val)` will add a new row to `{path}/{name}.tsv`. -💡 The linear plot `{path}/{name}.tsv` is usable by `dvc plots`. +💡 The metric history `{path}/{name}.tsv` is usable by `dvc plots`. If `name` contains slashes (e.g. `train/loss`), the required subdirectories will be created and the file will be saved inside the last one (e.g. diff --git a/content/docs/dvclive/api-reference/live/next_step.md b/content/docs/dvclive/api-reference/live/next_step.md index 59ad66f613..fffd182879 100644 --- a/content/docs/dvclive/api-reference/live/next_step.md +++ b/content/docs/dvclive/api-reference/live/next_step.md @@ -30,8 +30,8 @@ When `dvclive` is used alongside `DVC`, each `Live.next_step()` call will have additional effects. By default, on each `Live.next_step()` call, `DVC` will prepare an -[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with all -[linear plots](/doc/dvclive/get-started#linear-plots). +[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with the [metrics +history(/doc/dvclive/get-started#history). In addition, when [checkpoints](/doc/user-guide/experiment-management/checkpoints) are enabled in diff --git a/content/docs/dvclive/api-reference/live/set_step.md b/content/docs/dvclive/api-reference/live/set_step.md index a11fbe5e0f..43f4941f59 100644 --- a/content/docs/dvclive/api-reference/live/set_step.md +++ b/content/docs/dvclive/api-reference/live/set_step.md @@ -31,8 +31,8 @@ When `dvclive` is used alongside `DVC`, each `Live.set_step()` call will have additional effects. By default, on each `Live.set_step()` call, `DVC` will prepare an -[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with all -[linear plots](/doc/dvclive/get-started#linear-plots). +[HTML report](/doc/dvclive/dvclive-with-dvc#html-report) with the +[metrics history](/doc/dvclive/get-started#lhistory). In addition, when [checkpoints](/doc/user-guide/experiment-management/checkpoints) are enabled in @@ -43,7 +43,7 @@ the pipeline, `DVC` will ## Example Given the [Usage](#usage) code snippet above, the -[linear plot](/doc/dvclive/get-started#linear-plots) generated for `metric_1` +[metric history](/doc/dvclive/get-started#history) generated for `metric_1` would be: ```dvc diff --git a/content/docs/dvclive/dvclive-with-dvc.md b/content/docs/dvclive/dvclive-with-dvc.md index ab718b85a6..952abb48e8 100644 --- a/content/docs/dvclive/dvclive-with-dvc.md +++ b/content/docs/dvclive/dvclive-with-dvc.md @@ -85,11 +85,12 @@ dvc.lock training_metrics train.py dvc.yaml training_metrics.json ``` -The `.tsv` files generated under `training_metrics` can be visualized with -`dvc plots`. +The [metrics history](/doc/dvclive/get-started#history)) generated under +`training_metrics` can be visualized with `dvc plots`. -In addition, `training_metrics.json` can be used by `dvc metrics` and visualized -with `dvc exp show`/`dvc exp diff`. +In addition, the [metrics summary](/doc/dvclive/get-started#summary) +`training_metrics.json` can be used by `dvc metrics` and visualized with +`dvc exp show`/`dvc exp diff`. ### DVC Studio diff --git a/content/docs/dvclive/get-started.md b/content/docs/dvclive/get-started.md index b83a44ad31..ff27ddc195 100644 --- a/content/docs/dvclive/get-started.md +++ b/content/docs/dvclive/get-started.md @@ -82,9 +82,9 @@ $ cat dvclive.json > If you don't update the step number, the `step` entry won't be present in the > summary. -### Linear plots +### History -In addition, for each `{metric_name}`, DVCLive produces a linear plot under +In addition, for each `{metric_name}`, DVCLive stores the metric's history under `dvclive/{metric_name}.tsv`: ```dvc @@ -95,7 +95,7 @@ timestamp step {metric_name} 1614129198848 2 0.8907166719436646 ``` -> If you don't update the step number, the Metrics Logs won't be generated. +> If you don't update the step number, the metrics history won't be generated. ## What next? From 057dc069178841508c4e643ce2f5d24d2786f54e Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Thu, 27 Jan 2022 20:32:45 +0100 Subject: [PATCH 07/11] Bring back short descriptions --- content/docs/dvclive/api-reference/live/get_step.md | 7 ++++++- content/docs/dvclive/api-reference/live/index.md | 5 ++++- content/docs/dvclive/api-reference/live/log.md | 2 ++ content/docs/dvclive/api-reference/live/next_step.md | 4 +++- content/docs/dvclive/api-reference/live/set_step.md | 4 +++- 5 files changed, 18 insertions(+), 4 deletions(-) diff --git a/content/docs/dvclive/api-reference/live/get_step.md b/content/docs/dvclive/api-reference/live/get_step.md index 8fd704a4be..86c3124b5b 100644 --- a/content/docs/dvclive/api-reference/live/get_step.md +++ b/content/docs/dvclive/api-reference/live/get_step.md @@ -1,5 +1,7 @@ # Live.get_step() +Returns the current `step` value. + ```py def get_step() -> int: ``` @@ -21,4 +23,7 @@ while live.get_step() < 3: DVCLive uses `step` to track the progress of each metric logged with `Live.log()`. -The `step` value can be updated with `Live.next_step()` or `Live.set_step()`. +The current `step` value can be retrieved with `Live.get_step()`. + +In order to update the `step` value, you can use `Live.next_step()` or +`Live.set_step()`. diff --git a/content/docs/dvclive/api-reference/live/index.md b/content/docs/dvclive/api-reference/live/index.md index aa16e6e911..a2378958d3 100644 --- a/content/docs/dvclive/api-reference/live/index.md +++ b/content/docs/dvclive/api-reference/live/index.md @@ -1,5 +1,7 @@ # Live() +Initializes a DVCLive logger. + ```py class Live: @@ -21,7 +23,8 @@ live = Live() ## Description -Initializes a DVCLive logger. +A `Live()` instance is required in order to log machine learning metrics and +other metadata. ⚠️ `Live()` will remove all existing DVCLive related files under `path` unless `resume=True`. diff --git a/content/docs/dvclive/api-reference/live/log.md b/content/docs/dvclive/api-reference/live/log.md index 7e4071c1c3..91c9d1e0d7 100644 --- a/content/docs/dvclive/api-reference/live/log.md +++ b/content/docs/dvclive/api-reference/live/log.md @@ -1,5 +1,7 @@ # Live.log() +Logs the given scalar `val` associating it with the given `name`. + ```py def log(name: str, val: float): ``` diff --git a/content/docs/dvclive/api-reference/live/next_step.md b/content/docs/dvclive/api-reference/live/next_step.md index fffd182879..a7c0bc99a2 100644 --- a/content/docs/dvclive/api-reference/live/next_step.md +++ b/content/docs/dvclive/api-reference/live/next_step.md @@ -19,7 +19,9 @@ for step in range(3): ## Description DVCLive uses the `step` value to track the progress of each metric logged with -`Live.log()`. You can use `Live.next_step()` to increase the `step` by 1 (one). +`Live.log()`. + +You can use `Live.next_step()` to increase the `step` by 1 (one). Each metric logged in between `Live.next_step()` (or `Live.set_step()`) calls will be associated to the updated `step` value. diff --git a/content/docs/dvclive/api-reference/live/set_step.md b/content/docs/dvclive/api-reference/live/set_step.md index 43f4941f59..edaefd1635 100644 --- a/content/docs/dvclive/api-reference/live/set_step.md +++ b/content/docs/dvclive/api-reference/live/set_step.md @@ -20,7 +20,9 @@ for step in [0, 10, 20]: ## Description DVCLive uses the `step` value to track the progress of each metric logged with -`Live.log()`. You can use `Live.set_step()` to set `step` to any value. +`Live.log()`. + +You can use `Live.set_step()` to set `step` to any value. Each metric logged in between `Live.set_step()` (or `Live.next_step()`) calls will be associated to the provided `step` value. From 9c9c445193f5cd837d474b9bf666215878ff39d5 Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Thu, 27 Jan 2022 20:34:55 +0100 Subject: [PATCH 08/11] Update content/docs/dvclive/dvclive-with-dvc.md Co-authored-by: Dave Berenbaum --- content/docs/dvclive/dvclive-with-dvc.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/dvclive/dvclive-with-dvc.md b/content/docs/dvclive/dvclive-with-dvc.md index 952abb48e8..8aaaa38f35 100644 --- a/content/docs/dvclive/dvclive-with-dvc.md +++ b/content/docs/dvclive/dvclive-with-dvc.md @@ -85,7 +85,7 @@ dvc.lock training_metrics train.py dvc.yaml training_metrics.json ``` -The [metrics history](/doc/dvclive/get-started#history)) generated under +The [metrics history](/doc/dvclive/get-started#history) generated under `training_metrics` can be visualized with `dvc plots`. In addition, the [metrics summary](/doc/dvclive/get-started#summary) From 6b16f805f0c7af3ebb78736cb1f19ef258f2d91a Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Thu, 27 Jan 2022 20:46:03 +0100 Subject: [PATCH 09/11] Update warnings --- content/docs/dvclive/api-reference/live/index.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/docs/dvclive/api-reference/live/index.md b/content/docs/dvclive/api-reference/live/index.md index a2378958d3..ba8a62fbf5 100644 --- a/content/docs/dvclive/api-reference/live/index.md +++ b/content/docs/dvclive/api-reference/live/index.md @@ -49,15 +49,15 @@ other metadata. - `resume` - If `True`, DVCLive will try to read the previous `step` from the `path` directory and start from that point. _Default_: `False`. - ⚠️ Don't change the default value unless you are going to update the `step` - value during training. + ⚠️ If you are not using steps, don't set to `True` since DVCLive will preserve + existing files from previous run's and assume that `step` has been enabled. - `summary` - If `True`, upon each `Live.log()` call, DVCLive will generate a summary (usable by `dvc metrics`). The summary will be located at `{path}.json`. _Default_: `True`. - ⚠️ Don't change the default value unless you are going to update the `step` - value during training. + ⚠️ If you are not using steps, don't set to `False` since `Live.log()` won't + be generating any output. ## Exceptions From 2a0493378057561c37d1b9ecda29af1173eff665 Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Thu, 27 Jan 2022 21:30:55 +0100 Subject: [PATCH 10/11] Update content/docs/dvclive/api-reference/live/index.md Co-authored-by: Dave Berenbaum --- content/docs/dvclive/api-reference/live/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/content/docs/dvclive/api-reference/live/index.md b/content/docs/dvclive/api-reference/live/index.md index ba8a62fbf5..bb3885533b 100644 --- a/content/docs/dvclive/api-reference/live/index.md +++ b/content/docs/dvclive/api-reference/live/index.md @@ -50,7 +50,7 @@ other metadata. `path` directory and start from that point. _Default_: `False`. ⚠️ If you are not using steps, don't set to `True` since DVCLive will preserve - existing files from previous run's and assume that `step` has been enabled. + previous run's files and assume that `step` has been enabled. - `summary` - If `True`, upon each `Live.log()` call, DVCLive will generate a summary (usable by `dvc metrics`). The summary will be located at From 3bc7a49515a59b2bc14e421739a4211870a7ccd4 Mon Sep 17 00:00:00 2001 From: David de la Iglesia Castro Date: Thu, 27 Jan 2022 21:51:34 +0100 Subject: [PATCH 11/11] Add missing descriptions --- content/docs/dvclive/api-reference/live/next_step.md | 2 ++ content/docs/dvclive/api-reference/live/set_step.md | 2 ++ 2 files changed, 4 insertions(+) diff --git a/content/docs/dvclive/api-reference/live/next_step.md b/content/docs/dvclive/api-reference/live/next_step.md index a7c0bc99a2..402e2e9191 100644 --- a/content/docs/dvclive/api-reference/live/next_step.md +++ b/content/docs/dvclive/api-reference/live/next_step.md @@ -1,5 +1,7 @@ # Live.next_step() +Signals that the current step has ended and increases `step` value by 1 (one). + ```py def next_step() ``` diff --git a/content/docs/dvclive/api-reference/live/set_step.md b/content/docs/dvclive/api-reference/live/set_step.md index edaefd1635..b50003fff4 100644 --- a/content/docs/dvclive/api-reference/live/set_step.md +++ b/content/docs/dvclive/api-reference/live/set_step.md @@ -1,5 +1,7 @@ # dvclive.set_step() +Signals that the current step has ended and sets `step` to the given value. + ```py def set_step(step: int): ```