Automated Tests: Code Coverage and Standardized Test Infrastructure

[hayesall]

Currently pytest-cov is reporting 38% code coverage, which is a little on the low side.

It looks like some of the tests are implemented entirely in a __main__ rather than inside the unittest scaffolding (e.g. nnde/differentialequation/examples/tests/test_diff1d_half.py), so this number could be artificially low at the moment.

Recommendation: Implement tests inside a common unittest framework (unittest/nose/pytest) for consistent running and reporting.

---

Reproducing:

I checked against pytest with pytest-cov to check code coverage.

# (Clone repository)

# Install dependencies and run pytest with the coverage extension
pip install pytest pytest-cov
pytest --cov=nnde nnde/
coverage html

# Open up the coverage report (e.g. Ubunut)
xdg-open htmlcov/index.html

[ewinterapl]

I had added more tests a few months ago, and the screen output from the coverage command indicted 72% coverage:

`---------- coverage: platform darwin, python 3.8.11-final-0 ----------
Name Stmts Miss Cover

nnde/init.py 0 0 100%
nnde/differentialequation/init.py 0 0 100%
nnde/differentialequation/differentialequation.py 6 0 100%
nnde/differentialequation/examples/init.py 0 0 100%
nnde/differentialequation/examples/diff1d_half.py 69 0 100%
nnde/differentialequation/examples/diff1d_halfsine.py 87 0 100%
nnde/differentialequation/examples/diff1d_halfsine_increase.py 106 0 100%
nnde/differentialequation/examples/diff1d_one.py 70 70 0%
nnde/differentialequation/examples/diff1d_zero.py 70 70 0%
nnde/differentialequation/examples/diff2d_halfsine.py 142 7 95%
nnde/differentialequation/examples/diff2d_halfsine_increase.py 122 1 99%
nnde/differentialequation/examples/diff3d_halfsine.py 213 118 45%
nnde/differentialequation/examples/diff3d_halfsine_increase.py 188 11 94%
nnde/differentialequation/examples/example_pde1ivp_01.py 53 28 47%
nnde/differentialequation/examples/lagaris_01.py 13 5 62%
nnde/differentialequation/examples/lagaris_02.py 13 5 62%
nnde/differentialequation/examples/lagaris_03_bvp.py 18 18 0%
nnde/differentialequation/examples/lagaris_03_ivp.py 18 18 0%
nnde/differentialequation/examples/lagaris_04.py 10 10 0%
nnde/differentialequation/examples/lagaris_05.py 41 41 0%
nnde/differentialequation/examples/pde_01.py 32 32 0%
nnde/differentialequation/examples/pde_02.py 41 41 0%
nnde/differentialequation/examples/simple_01.py 12 12 0%
nnde/differentialequation/examples/tests/init.py 0 0 100%
nnde/differentialequation/examples/tests/test_diff1d_half.py 263 1 99%
nnde/differentialequation/examples/tests/test_diff1d_halfsine.py 262 1 99%
nnde/differentialequation/examples/tests/test_diff1d_halfsine_increase.py 272 1 99%
nnde/differentialequation/examples/tests/test_diff1d_one.py 263 1 99%
nnde/differentialequation/examples/tests/test_diff1d_zero.py 263 1 99%
nnde/differentialequation/examples/tests/test_diff2d_halfsine.py 493 11 98%
nnde/differentialequation/examples/tests/test_diff2d_halfsine_increase.py 428 1 99%
nnde/differentialequation/examples/tests/test_diff3d_halfsine.py 92 89 3%
nnde/differentialequation/examples/tests/test_diff3d_halfsine_increase.py 710 1 99%
nnde/differentialequation/examples/tests/test_example_pde1ivp_01.py 35 32 9%
nnde/differentialequation/examples/tests/test_lagaris_01.py 8 5 38%
nnde/differentialequation/examples/tests/test_lagaris_02.py 8 5 38%
nnde/differentialequation/ode/init.py 0 0 100%
nnde/differentialequation/ode/ode.py 3 0 100%
nnde/differentialequation/ode/ode1.py 5 0 100%
nnde/differentialequation/ode/ode1ivp.py 27 14 48%
nnde/differentialequation/ode/tests/init.py 0 0 100%
nnde/differentialequation/ode/tests/test_ODE.py 9 1 89%
nnde/differentialequation/ode/tests/test_ODE1.py 12 1 92%
nnde/differentialequation/ode/tests/test_ODE1IVP.py 7 1 86%
nnde/differentialequation/pde/init.py 0 0 100%
nnde/differentialequation/pde/pde.py 3 0 100%
nnde/differentialequation/pde/pde1.py 5 0 100%
nnde/differentialequation/pde/pde1ivp.py 29 16 45%
nnde/differentialequation/pde/pde2.py 5 0 100%
nnde/differentialequation/pde/pde2diff.py 47 28 40%
nnde/differentialequation/pde/tests/init.py 0 0 100%
nnde/differentialequation/pde/tests/test_PDE.py 9 1 89%
nnde/differentialequation/pde/tests/test_PDE1.py 12 1 92%
nnde/differentialequation/pde/tests/test_PDE1IVP.py 8 1 88%
nnde/differentialequation/pde/tests/test_PDE2.py 12 1 92%
nnde/differentialequation/pde/tests/test_PDE2DIFF.py 7 1 86%
nnde/differentialequation/tests/init.py 0 0 100%
nnde/differentialequation/tests/test_DifferentialEquation.py 12 1 92%
nnde/exceptions/init.py 0 0 100%
nnde/exceptions/nndeexception.py 2 0 100%
nnde/exceptions/tests/init.py 0 0 100%
nnde/exceptions/tests/test_NNDEException.py 7 1 86%
nnde/math/init.py 0 0 100%
nnde/math/kdelta.py 2 0 100%
nnde/math/sigma.py 22 0 100%
nnde/math/tests/init.py 0 0 100%
nnde/math/tests/test_kdelta.py 11 1 91%
nnde/math/tests/test_sigma.py 37 1 97%
nnde/math/tests/test_trainingdata.py 27 1 96%
nnde/math/trainingdata.py 41 16 61%
nnde/neuralnetwork/init.py 0 0 100%
nnde/neuralnetwork/neuralnetwork.py 8 0 100%
nnde/neuralnetwork/nnode1ivp.py 250 214 14%
nnde/neuralnetwork/nnpde1ivp.py 474 425 10%
nnde/neuralnetwork/nnpde2diff.py 490 448 9%
nnde/neuralnetwork/slffnn.py 3 0 100%
nnde/neuralnetwork/tests/init.py 0 0 100%
nnde/neuralnetwork/tests/test_NNODE1IVP.py 29 1 97%
nnde/neuralnetwork/tests/test_NNPDE1IVP.py 7 1 86%
nnde/neuralnetwork/tests/test_NeuralNetwork.py 15 1 93%
nnde/neuralnetwork/tests/test_PDE2DIFF.py 27 1 96%
nnde/neuralnetwork/tests/test_SLFFNN.py 9 1 89%
nnde/trialfunction/init.py 0 0 100%
nnde/trialfunction/diff1dtrialfunction.py 73 0 100%
nnde/trialfunction/diff2dtrialfunction.py 79 0 100%
nnde/trialfunction/diff3dtrialfunction.py 85 0 100%
nnde/trialfunction/tests/init.py 0 0 100%
nnde/trialfunction/tests/test_Diff1DTrialFunction.py 57 1 98%
nnde/trialfunction/tests/test_Diff2DTrialFunction.py 57 1 98%
nnde/trialfunction/tests/test_Diff3DTrialFunction.py 57 1 98%
nnde/trialfunction/tests/test_TrialFunction.py 8 1 88%
nnde/trialfunction/trialfunction.py 3 1 67%

TOTAL 6513 1820 72%`

But when I looked at the htmlcov/index.html this morning, I saw that the coverage was only 37%. I did not realize the two outputs were providing different results, but I may be misunderstanding the use of the tool.

If the reviewers indicate more test coverage is desired prior to acceptance, I will do so. Otherwise, I will defer this work until after acceptance.

[diehlpk]

@elwinter I would suggest that you might spend one hour to try to fix that. It would be nice to have the correct coverage in the paper.

[elwinter]

Ok that is what I wanted to know. I will be working on that this weekend. Thanks Eric Get Outlook for Android<https://aka.ms/ghei36>

…

________________________________ From: Patrick Diehl ***@***.***> Sent: Friday, October 15, 2021 10:40:11 AM To: elwinter/nnde ***@***.***> Cc: Eric Winter ***@***.***>; Mention ***@***.***> Subject: Re: [elwinter/nnde] Automated Tests: Code Coverage and Standardized Test Infrastructure (#13) @elwinter<https://github.com/elwinter> I would suggest that you might spend one hour to try to fix that. It would be nice to have the correct coverage in the paper. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub<#13 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/ABSFWCY67CH2X44S4ZTDM2TUHA4MXANCNFSM5AKIOU2A>. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[diehlpk]

@elwinter I think we can close the issue since you fixed the coverage issue.