Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

pytest crashes if raised exception is not hashable #4514

Closed
stuarteberg opened this issue Dec 6, 2018 · 5 comments
Closed

pytest crashes if raised exception is not hashable #4514

stuarteberg opened this issue Dec 6, 2018 · 5 comments
Labels
type: question general question, might be closed after 2 weeks of inactivity

Comments

@stuarteberg
Copy link

stuarteberg commented Dec 6, 2018 •
edited
Loading

If a test fails with a non-hashable exception, pytest crashes.

[Edit: No crash under python 3.7; only under 3.6 (and maybe others).]

Here's a simple example:

# test.py
class MyUnhashableException(Exception):
    def __eq__(self, other):
        return True

def test():
    raise MyUnhashableException()

Click below for the traceback I see when running pytest test.py (using python 3.6).

Click for traceback 
$ pytest test.py
============================================================================== test session starts ===============================================================================
platform darwin -- Python 3.6.2, pytest-3.2.3, py-1.4.34, pluggy-0.4.0
rootdir: /private/tmp, inifile:
plugins: xonsh-0.7.10, cov-2.6.0
collected 1 item

test.py
INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/main.py", line 110, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/main.py", line 146, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/main.py", line 169, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 254, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 254, in _wrapped_call
INTERNALERROR>     return call_outcome.get_result()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/runner.py", line 68, in pytest_runtest_protocol
INTERNALERROR>     runtestprotocol(item, nextitem=nextitem)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/runner.py", line 82, in runtestprotocol
INTERNALERROR>     reports.append(call_and_report(item, "call", log))
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/runner.py", line 164, in call_and_report
INTERNALERROR>     report = hook.pytest_runtest_makereport(item=item, call=call)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 250, in _wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/skipping.py", line 213, in pytest_runtest_makereport
INTERNALERROR>     rep = outcome.get_result()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 279, in get_result
INTERNALERROR>     raise ex[1].with_traceback(ex[2])
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 265, in __init__
INTERNALERROR>     self.result = func()
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/runner.py", line 308, in pytest_runtest_makereport
INTERNALERROR>     longrepr = item.repr_failure(excinfo)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/python.py", line 583, in repr_failure
INTERNALERROR>     return self._repr_failure_py(excinfo, style=style)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/python.py", line 576, in _repr_failure_py
INTERNALERROR>     style=style)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/main.py", line 476, in _repr_failure_py
INTERNALERROR>     style=style, tbfilter=tbfilter)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/_code/code.py", line 435, in getrepr
INTERNALERROR>     return fmt.repr_excinfo(self)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/_code/code.py", line 669, in repr_excinfo
INTERNALERROR>     reprtraceback = self.repr_traceback(excinfo)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/_code/code.py", line 621, in repr_traceback
INTERNALERROR>     reprentry = self.repr_traceback_entry(entry, einfo)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/_code/code.py", line 581, in repr_traceback_entry
INTERNALERROR>     s = self.get_source(source, line_index, excinfo, short=short)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/_code/code.py", line 526, in get_source
INTERNALERROR>     lines.extend(self.get_exconly(excinfo, indent=indent, markall=True))
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/_code/code.py", line 533, in get_exconly
INTERNALERROR>     exlines = excinfo.exconly(tryshort=True).split('\n')
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/site-packages/_pytest/_code/code.py", line 398, in exconly
INTERNALERROR>     lines = format_exception_only(self.type, self.value)
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/traceback.py", line 136, in format_exception_only
INTERNALERROR>     return list(TracebackException(etype, value, None).format_exception_only())
INTERNALERROR>   File "/miniforge/envs/flyem-forge/lib/python3.6/traceback.py", line 462, in __init__
INTERNALERROR>     _seen.add(exc_value)
INTERNALERROR> TypeError: unhashable type: 'MyUnhashableException'

========================================================================== no tests ran in 0.07 seconds ==========================================================================

Admittedly, it's rare for exceptions not to be hashable. But for a real-world example, see this jsonschema issue:

Is this known behavior? Is there any easy change that would permit unhashable exceptions?
@stuarteberg stuarteberg mentioned this issue Dec 6, 2018
Closed
@stuarteberg
Copy link
Author

stuarteberg commented Dec 6, 2018 •
edited
Loading

This is an issue with the latest version of pytest (4.0.1), but only when I test with python 3.6. It is not an issue under python 3.7. (I have not tried other versions of python.)

FWIW, here are my environment details:

# Name                    Version                   Build  Channel
atomicwrites              1.2.1                      py_0    conda-forge
attrs                     18.2.0                     py_0    conda-forge
ca-certificates           2018.11.29           ha4d7672_0    conda-forge
certifi                   2018.11.29            py36_1000    conda-forge
libffi                    3.2.1                hfc679d8_5    conda-forge
more-itertools            4.3.0                 py36_1000    conda-forge
ncurses                   6.1                  hfc679d8_1    conda-forge
openssl                   1.0.2p               h470a237_1    conda-forge
pip                       18.1                  py36_1000    conda-forge
pluggy                    0.8.0                      py_0    conda-forge
py                        1.7.0                      py_0    conda-forge
pytest                    4.0.1                 py36_1000    conda-forge
python                    3.6.7                h5001a0f_1    conda-forge
readline                  7.0                  haf1bffa_1    conda-forge
setuptools                40.6.2                   py36_0    conda-forge
six                       1.11.0                py36_1001    conda-forge
sqlite                    3.26.0               hb1c47c0_0    conda-forge
tk                        8.6.9                ha92aebf_0    conda-forge
wheel                     0.32.3                   py36_0    conda-forge
xz                        5.2.4                h470a237_1    conda-forge
zlib                      1.2.11               h470a237_3    conda-forge

@stuarteberg stuarteberg reopened this Dec 6, 2018
@RonnyPfannschmidt
Copy link
Member

as per traceback this looks like a bug in the python stdlib that seems to be fixed in python 3.7

python/cpython#4014

@Zac-HD Zac-HD added the type: question general question, might be closed after 2 weeks of inactivity label Dec 7, 2018
@Zac-HD
Copy link
Member

Zac-HD commented Dec 7, 2018

The fix has also been backported to 3.6, which is still in the upstream bugfix period, so this is a rare upstream bug that only affects 3.4 and 3.5.

Given that there's nothing sensible we can do to rescue exceptions in the general case, I'd rather not add a best-effort patch that will only obscure the real problem when it fails - especially when it's easy to avoid defining unhashable exceptions!

@Zac-HD Zac-HD closed this as completed Dec 7, 2018
@stuarteberg
Copy link
Author

Ok, good to hear that Python 3.6 will be fixed soon. Thanks very much for looking into it, and for the explanation, @Zac-HD and @RonnyPfannschmidt!

@Zac-HD
Copy link
Member

Zac-HD commented Dec 7, 2018

It's been fixed since October 2017 (python/cpython#4024), meaning 3.6.4 and later are fine 😄

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type: question general question, might be closed after 2 weeks of inactivity
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@RonnyPfannschmidt @stuarteberg @Zac-HD