Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking β€œSign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

live logs initial #4902

Merged
merged 60 commits into from
Jan 4, 2021
Merged
Changes from 1 commit
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
7fcd409
plots: allow dir plots
pared Nov 10, 2020
9986c68
first scenario
pared Nov 10, 2020
39f0591
separate dvclive tests
pared Nov 11, 2020
1779ecc
rename index to step
pared Nov 12, 2020
af5d09a
another test
pared Nov 12, 2020
52254eb
refactor
pared Nov 13, 2020
8f4c481
its ok to have step in data
pared Nov 13, 2020
c69b166
sort plots
pared Nov 13, 2020
6582621
introduce --logs
pared Nov 13, 2020
5daa974
add logs to run commands
pared Nov 16, 2020
86dc9ac
variable name refactoring
pared Nov 16, 2020
014e995
fix logs option
pared Nov 20, 2020
2047c8a
fixup
pared Nov 20, 2020
c7d4b6d
cmd: run: fix unit tests
pared Nov 20, 2020
c4fd9e6
plots: modify: fix tests
pared Nov 20, 2020
b5a78d7
improve dir handling
pared Nov 24, 2020
8cd149d
fixup
pared Nov 24, 2020
4a898c2
dvclive: logs summary
pared Nov 24, 2020
6cdb1da
api: introduce logs summarization method
pared Nov 25, 2020
8f60734
handle no repo case
pared Nov 25, 2020
c922a5a
fix granular dir plots
pared Nov 25, 2020
fc18606
api: dvclive: simplify
pared Nov 27, 2020
7d8e8dc
dvclive: roll back logs flag
pared Nov 27, 2020
0e859a6
cmd: introduce logs
pared Nov 27, 2020
4955bb2
api: adjust to logs
pared Nov 27, 2020
54706d5
test: dvclive fix
pared Nov 30, 2020
6c8f1f4
rename logs to dvclive
pared Dec 1, 2020
3e38509
live: command: add uninitialized
pared Dec 2, 2020
0e50841
outs: dvclive type
pared Dec 4, 2020
cfe54a0
dvclive: support providing config via evn variables
pared Dec 4, 2020
76d8209
dvclive: treat as metrics
pared Dec 10, 2020
62e1bcd
dvclive integration: add support for summary on/off
pared Dec 14, 2020
8272cec
dvc integration: add temporary test for configuration export
pared Dec 14, 2020
adac0fe
remove requirements
pared Dec 16, 2020
f95c11c
cmd: introduce logs
pared Nov 27, 2020
3fbb5a2
rename logs to dvclive
pared Dec 1, 2020
a82b413
cmd: introduce logs
pared Nov 27, 2020
5654244
remove logs leftowvers
pared Dec 16, 2020
4673c9f
dvclive: env: adjust to chekpoints env variables
pared Dec 16, 2020
26cec5a
command: live: remove unnecessary help
pared Dec 16, 2020
d7864fe
fix linter suggestions
pared Dec 16, 2020
90c0c5b
type hinting
pared Dec 17, 2020
56116d8
review refactor
pared Dec 21, 2020
8cf0804
replace dvclive with live
pared Dec 21, 2020
789025a
stage: loading: move live loading to separate method
pared Dec 21, 2020
645654d
stage: serialize: allow only single live output
pared Dec 21, 2020
6b966d2
live: add show/diff
pared Dec 21, 2020
095f926
stage: serialize: roll back single live enforcing
pared Dec 21, 2020
b8096b1
fixup
pared Dec 21, 2020
5bde938
rebase fixups
pared Dec 22, 2020
273eed3
output: live: force single instance
pared Dec 22, 2020
ce1deeb
live: command: add unit tests
pared Dec 22, 2020
1620852
Update dvc/output/base.py
pared Dec 29, 2020
88cf651
Update dvc/stage/__init__.py
pared Dec 29, 2020
47d1c0a
Update dvc/stage/serialize.py
pared Dec 29, 2020
bbe180f
review v2
pared Dec 29, 2020
44c3731
live: fix summary tracking by experiments
pared Dec 30, 2020
3762fc4
naming fixups
pared Dec 30, 2020
14779a2
api: live: add tests
pared Dec 30, 2020
abf92f7
tests: api: separate live tests
pared Dec 30, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
api: live: add tests
  • Loading branch information
pared committed Dec 30, 2020
commit 14779a252a328d5ccfbdbb995a5bd9473b2ad90a
59 changes: 58 additions & 1 deletion tests/func/test_api.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,16 @@
import csv
import io
import json
import logging
import os
from typing import Dict, List

import pytest
from flaky.flaky_decorator import flaky
from funcy import first, get_in
from funcy import first, get_in, last

from dvc import api
from dvc.api.live import summary
from dvc.exceptions import FileMissingError, OutputNotFoundError
from dvc.path_info import URLInfo
from dvc.utils.fs import remove
@@ -271,3 +277,54 @@ def test_get_url_subrepos(tmp_dir, scm, local_cloud):

expected_url = os.path.join(path, "37", "b51d194a7513e45b56f6524f2d51f2")
assert api.get_url("subrepo/bar") == expected_url


def _dumps_sv(metrics: List[Dict], delimiter=","):
stream = io.StringIO()
writer = csv.DictWriter(
stream, fieldnames=list(first(metrics).keys()), delimiter=delimiter
)
writer.writeheader()
writer.writerows(metrics)
stream.seek(0)
return stream.read()


@pytest.fixture
def live_results(tmp_dir):
def make(path="logs"):
datapoints = [{"metric": 0.0, "step": 0}, {"metric": 0.5, "step": 1}]
tmp_dir.gen(
{
(tmp_dir / path).with_suffix(".json"): json.dumps(
last(datapoints)
),
(tmp_dir / path / "metric.tsv"): _dumps_sv(
datapoints, delimiter="\t",
),
}
)

yield make


def test_live_summary_no_repo(tmp_dir, live_results, caplog):
live_results("logs")

with caplog.at_level(logging.INFO, logger="dvc"):
summary("logs")

summary_path = tmp_dir / "logs.html"
assert summary_path.exists()
assert f"file://{str(summary_path)}" in caplog.text


def test_live_summary(tmp_dir, dvc, live_results, caplog):
live_results("logs")

with caplog.at_level(logging.INFO, logger="dvc"):
summary("logs")

summary_path = tmp_dir / "logs.html"
assert summary_path.exists()
assert f"file://{str(summary_path)}" in caplog.text