Using pytest --testmon --cov to see merged coverage results

[brendanator]

When I run pytest coverage along with testmon the coverage reports only the code that was run in the selected tests.

It would be incredible if the coverage reported the merged data from the tests that are selected and the rest of the data in .testmondata.

I'd love to get instantly updated test converage over a whole project. Is this possible?

[tarpas]

That is not unthinkable but quite difficult. Coverage data refers to the source code by line numbers, so it becomes invalid whenever you add or remove a line. I would say it's out of scope of this project for now.

[landeholt]

That is not unthinkable but quite difficult. Coverage data refers to the source code by line numbers, so it becomes invalid whenever you add or remove a line. I would say it's out of scope of this project for now.

Are there no other ways around this? I've just recently found your work @tarpas - so I am quite green when it comes to your code. But I've had a few hours of tinkering with it on my local machine.

What is exactly needed in order to make this work?

I've noticed that you erase the coverage reports for each test in /tests - why is that? Can't we just accumulate that json data?

EDIT:

wouldn't it just be an instance of caching and upserting changed or added "morfs"?

[landeholt]

Seems like commenting out this line

pytest-testmon/testmon/testmon_core.py

Line 346 in 67c8054

self.cov.erase()

doesn't seem to affect testmon, but allows coverage to keep track on the coverage data, and it looks like coverage is upserting their filenames: coveragepy/sqldata.py

meaning that it (should) accomplish what @brendanator wished for.

EDIT:

I did later on notice changes in testmon. So removing self.cov.erase()shouldn't be done. 😞

[tarpas]

As I wrote:

Coverage data refers to the source code by line numbers, so it becomes invalid whenever you add or remove a line.

Also to have complete picture: .testmondata is a "lossy" format. Only method checksums are stored.

Testmon would have to come up with "lossless" .testmondata format which is a challenge by itself (plus keep it performant and not eating too much space)

[tarpas]

This has been implemented and release to the extent it's technically practical/feasible. The results are not perfect, see discussion here: #86