[R-package] [docs] instructions for generating code coverage produce incorrect results #4919
Comments
|
Other things to try while working on this (that I haven't tried yet and won't have time to try for a while):
|
I tried this today, and saw that |
This did not help either. Still got 0.0% coverage for |
|
Ok, getting closer! The Running the following prior to the testing instructions in this issue's description results in coverage being estimated correctly. git rm R-package/tests/testthat/test_lgb.unloader.RThat makes sense to me. That function detaches the LightGBM/R-package/R/lgb.unloader.R Line 49 in af5b40e Notes from the
and a bit more (link)
I'm going to open a PR proposing just skipping those tests when computing coverage. |
…4922) * [R-package] [docs] fix calculation of R test coverage (fixes #4919) * use quotes * use in_covr() instead of env variable * Update R-package/README.md Co-authored-by: Nikita Titov <[email protected]> * check that covr exists * add covr to suggests * use R_COVR environment variable * Update R-package/tests/testthat/test_lgb.unloader.R Co-authored-by: Nikita Titov <[email protected]> Co-authored-by: Nikita Titov <[email protected]>
|
This issue has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this. |
Description
The R package's README contains a section describing how to generate a code coverage report using
{covr}.https://github.com/microsoft/LightGBM/blob/ce486e5b45a6f5e67743e14765ed139ff8d532e5/R-package/README.md#code-coverage
Those instructions run without error and do produce some code coverage, but the coverage results are incorrect.
For example, look at how many files are reported as having 0 coverage.
Seeing 0.00% coverage for
lgb.cv(), for example, is very surprising, given that there are around 30 calls to that function in the R package's tests.LightGBM/R-package/tests/testthat/test_basic.R
Line 363 in ce486e5
It isn't like all coverage results are 0.0, as can sometimes happen with
pytest-covfor example. Other files do show some test coverage.full coverage results (click me)
Reproducible example
Environment info
LightGBM version or commit hash: latest
master(ce486e5)Impact of this issue
Test coverage is a valuable tool in guiding where to invest development time in writing tests, as it answers questions like "which paths through the code are not currently tested?".
As long as the test coverage setup with
{covr}reports incorrect results for{lightgbm}, maintenance work devoted to increasing test coverage requires manually identifying uncovered code paths.Additional Comments
While trying to debug, I used the following code which saves the
covr::coverageobject for further inspection.I suspected that this issue might be related to the used of the now-deprecated
testthat::context(), but I don't think so...I get the same incorrect results on the branch from #4915.The text was updated successfully, but these errors were encountered: