Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Report ignores decorators when run on Python 3.8 if testing was done on previous Python versions. #941

Closed
abompard opened this issue Feb 22, 2020 · 3 comments
Labels
bug Something isn't working duplicate This issue or pull request already exists

Comments

@abompard
Copy link

Describe the bug
If the coverage run is done on Python < 3.8 and the coverage report is done on Python 3.8, the lines following decorators (the function/method/class definitions) will be ignored and result in lower coverage

To Reproduce

Here's a file to reproduce it:

import unittest

class Dummy:

    @property
    def under_test(self):
        return 1

class TestDummy(unittest.TestCase):
    def test_function(self):
        obj = Dummy()
        self.assertEqual(obj.under_test, 1)

unittest.main()

Here's what I get when I run coverage with different Python versions:

$ .tox/py36-unittest/bin/coverage erase
$ .tox/py36-unittest/bin/coverage run --source=under_test ./under_test.py 
.
----------------------------------------------------------------------
Ran 1 test in 0.000s

OK

The test run has been done with Python 3.6. Now the report:

$ .tox/py36-unittest/bin/coverage report -m
Name            Stmts   Miss  Cover   Missing
---------------------------------------------
under_test.py       9      0   100%

Coverage correctly reports 100% coverage if the reporting is done with Python 3.6 (or 3.7 for that matter)

$ .tox/py38-unittest/bin/coverage report -m
Name            Stmts   Miss  Cover   Missing
---------------------------------------------
under_test.py      10      1    90%   7

When run with Python 3.8, coverage report thinks the method definition was not covered. It would behaved correctly without the decorator.

Here are the versions in both cases:

$ .tox/py36-unittest/bin/coverage --version
Coverage.py, version 5.0.3 with C extension
$ .tox/py38-unittest/bin/coverage --version
Coverage.py, version 5.0.3 with C extension
$ .tox/py36-unittest/bin/python -V
Python 3.6.10
$ .tox/py38-unittest/bin/python -V
Python 3.8.2rc1

I'm available if you need any more info. My use case might not even be supported, but I thought I'd report it in case it helps explain other issues.

@abompard abompard added the bug Something isn't working label Feb 22, 2020
abompard added a commit to abompard/noggin that referenced this issue Feb 22, 2020
Rawhide defaults to Python 3.8 and we get hit by this issue:
nedbat/coveragepy#941

Signed-off-by: Aurélien Bompard <[email protected]>
abompard added a commit to abompard/noggin that referenced this issue Feb 22, 2020
Rawhide defaults to Python 3.8 and we get hit by this issue:
nedbat/coveragepy#941

Signed-off-by: Aurélien Bompard <[email protected]>
abompard added a commit to fedora-infra/noggin that referenced this issue Feb 22, 2020
Rawhide defaults to Python 3.8 and we get hit by this issue:
nedbat/coveragepy#941

Signed-off-by: Aurélien Bompard <[email protected]>
@langmm
Copy link

langmm commented Mar 6, 2020

I just wanted to second that I am also having this issue with my project (e.g. https://codecov.io/gh/cropsinsilico/yggdrasil/src/1a39f11e62cd6924633c14caa613aae31899f9d1/yggdrasil/drivers/tests/test_CMakeModelDriver.py).

langmm added a commit to cropsinsilico/yggdrasil that referenced this issue Mar 6, 2020
@nedbat
Copy link
Owner

nedbat commented Mar 9, 2020

I'm not sure what to do about this. Different versions of Python trace lines slightly differently. Coverage.py "knows" what lines Python 3.7 will trace, and also what lines 3.8 will trace.

When recording execution, coverage.py just records whatever Python reports. During the reporting phase, it takes into account the version of Python to determine what lines could have been traced but were not. So when the measurement is on a different version than the reporting, the "could have" logic is off, and lines get incorrectly marked as missing.

(I feel like I wrote this recently on another issue, but I can't find it.)

@nedbat
Copy link
Owner

nedbat commented Mar 9, 2020

Oh, this is a duplicate of #866

@nedbat nedbat added the duplicate This issue or pull request already exists label Nov 15, 2021
@nedbat nedbat closed this as completed Nov 15, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

3 participants