-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix issue while sending analytics report #6407
Conversation
When sending analytics, we are not flushing the data to the temp file which causes failure at times, especially on Windows
The test in de21ae4 demonstrates the issue with analytics. See https://github.com/iterative/dvc/actions/runs/1116040914. Here's a snippet of the traceback: ------------------------------ Captured log call -------------------------------
ERROR dvc:main.py:86 unexpected error
Traceback (most recent call last):
File "/Users/runner/work/dvc/dvc/dvc/main.py", line 55, in main
ret = cmd.do_run()
File "/Users/runner/work/dvc/dvc/dvc/command/base.py", line 64, in do_run
return self.run()
File "/Users/runner/work/dvc/dvc/dvc/command/daemon.py", line 32, in run
analytics.send(self.args.target)
File "/Users/runner/work/dvc/dvc/dvc/analytics.py", line 67, in send
report = json.load(fobj)
File "/Users/runner/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/json/__init__.py", line 296, in load
parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
File "/Users/runner/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/json/__init__.py", line 348, in loads
return _default_decoder.decode(s)
File "/Users/runner/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/Users/runner/hostedtoolcache/Python/3.7.10/x64/lib/python3.7/json/decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0) This is happening because the file where we are saving the analytics report has been newly created (through Lines 33 to 35 in 38cd7d0
|
This way, the contents bufferred will get a chance to be flushed.
def mock_daemon(mocker): | ||
def func(argv): | ||
return main(["daemon", *argv]) | ||
|
||
m = mocker.patch("dvc.daemon.daemon", mocker.MagicMock(side_effect=func)) | ||
yield m |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Having a conditional daemonizer would be good for testing and QA, rather than mocking it. But didn't want to introduce big change.
@@ -32,7 +32,7 @@ def collect_and_send_report(args=None, return_code=None): | |||
|
|||
with tempfile.NamedTemporaryFile(delete=False, mode="w") as fobj: | |||
json.dump(report, fobj) | |||
daemon(["analytics", fobj.name]) | |||
daemon(["analytics", fobj.name]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good catch!
When sending analytics, we are not flushing the data to the temp file
which causes failure at times, especially on Windows.
β I have followed the Contributing to DVC checklist.
π If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.
Thank you for the contribution - we'll try to review it as soon as possible. π