Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid runtime exception when file doesn't exist #2441

Merged
merged 3 commits into from
Jan 17, 2019

Conversation

sf-wind
Copy link
Contributor

@sf-wind sf-wind commented Jan 16, 2019

When running auto-tuning, autoTVM tries to read from history run logs. However, if the log file doesn't exist, and cannot download from the web, the run would fail. That is because load_from_file(records)
returns a generator, which is guaranteed to be not None. So it would fail in line 279 when iterating the records.

I believe we need to return in case the records does not exist.

@eqy
Copy link
Contributor

eqy commented Jan 16, 2019

I think there is a check for that the tophub package has been downloaded:
https://github.com/dmlc/tvm/blob/a9bd5593a35ce4717f00cc71c2377f745284df5a/python/tvm/autotvm/tophub.py#L172
Can you give an example of a test case where the current implementation fails?

@sf-wind
Copy link
Contributor Author

sf-wind commented Jan 16, 2019

The trace is:

File "/data/users/feisun/tvm/venv/lib/python2.7/site-packages/nnvm-0.8.0-py2.7.egg/nnvm/compiler/build_module.py", line 245, in build [0/1821]
tophub_context = autotvm.tophub.context(target)
File "/data/users/feisun/tvm/venv/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-linux-x86_64.egg/tvm/autotvm/tophub.py", line 83, in context
best_context.load(os.path.join(AUTOTVM_TOPHUB_ROOT_PATH, filename))
File "/data/users/feisun/tvm/venv/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-linux-x86_64.egg/tvm/autotvm/task/dispatcher.py", line 282, in load
for inp, res in records:
File "/data/users/feisun/tvm/venv/lib/python2.7/site-packages/tvm-0.5.dev0-py2.7-linux-x86_64.egg/tvm/autotvm/record.py", line 166, in load_from_file
for row in open(filename):
IOError: [Errno 2] No such file or directory: '/home/feisun/.tvm/tophub/llvm_v0.03.log'

@eqy
Copy link
Contributor

eqy commented Jan 16, 2019

That is interesting; did you see the logging.warning("Failed to download tophub package for %s: %s", backend, e) warning? check_backend(name) should be called before this.

@eqy eqy requested a review from merrymercy January 16, 2019 01:07
@sf-wind
Copy link
Contributor Author

sf-wind commented Jan 16, 2019

Yes, I saw:

INFO:autotvm:Download pre-tuned parameters package llvm_v0.03.log
WARNING:root:Failed to download tophub package for llvm: <urlopen error [Errno 101] Network is unreachable>

@merrymercy
Copy link
Member

We'd better move the error handling to a place within TopHub. Because we want to hide this error for TopHub but we don't want to hide this error for other cases. "cannot find file" is a fatal error that worth an exception.

We can move the error handling to python/tvm/autotvm/tophub.py
in check_backend, return a True if it successes(after L108, L115), returns a False if it fails (after L117)
Then in L80, handle this return value.

@merrymercy
Copy link
Member

Thanks @sf-wind @eqy. It is merged.

@merrymercy merrymercy merged commit d0f8366 into apache:master Jan 17, 2019
Anthony-Mai pushed a commit to Anthony-Mai/tvm that referenced this pull request Jan 20, 2019
* Avoid runtime exception when file doesn't exist

* Update the check based on feedback

* Revert the old fix
Anthony-Mai pushed a commit to Anthony-Mai/tvm that referenced this pull request Jan 20, 2019
* Avoid runtime exception when file doesn't exist

* Update the check based on feedback

* Revert the old fix
wweic pushed a commit to neo-ai/tvm that referenced this pull request Feb 20, 2019
* Avoid runtime exception when file doesn't exist

* Update the check based on feedback

* Revert the old fix
wweic pushed a commit to neo-ai/tvm that referenced this pull request Feb 20, 2019
* Avoid runtime exception when file doesn't exist

* Update the check based on feedback

* Revert the old fix
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants