Fine-tune code coverage measurement for optional builds #208
Labels
Comments
lukpueh
added a commit
to lukpueh/securesystemslib
that referenced
this issue
Apr 6, 2020
But exempt those lines from coverage secure-systems-lab#208
|
Also consider code branches being skipped on certain platform, e.g. PySPX on Windows (#518) |
lukpueh
added a commit
to lukpueh/securesystemslib
that referenced
this issue
Feb 21, 2023
Tests on Windows fail because they don't have 95% coverage (only ~93 %). This commit lowers the requirement to a value that seems reasonable for all platforms. Coverage measurement may be fine tuned with secure-systems-lab#208 and changed back to a higher value if desired. Signed-off-by: Lukas Puehringer <[email protected]>
lukpueh
added a commit
to lukpueh/securesystemslib
that referenced
this issue
Mar 20, 2023
Excluding more tests on Windows (HSM signer) further decreases test coverage to ~88%. This commit makes 85% the lower boundary for all platforms. Coverage measurement should fine tuned with secure-systems-lab#208 and changed back to a higher value! Signed-off-by: Lukas Puehringer <[email protected]>
lukpueh
added a commit
to lukpueh/securesystemslib
that referenced
this issue
Apr 10, 2024
With the upcoming removal of legacy interfaces and related tests, the ratio of tested to untested code changes significantly. This does not mean that the test coverage of the Signer API decreases! Also note that overall test coverage might be higher (see secure-systems-lab#208). Signed-off-by: Lukas Puehringer <[email protected]>
lukpueh
added a commit
to lukpueh/securesystemslib
that referenced
this issue
Apr 12, 2024
With the upcoming removal of legacy interfaces and related tests, the ratio of tested to untested code changes significantly. This does not mean that the test coverage of the Signer API decreases! Also note that overall test coverage might be higher (see secure-systems-lab#208). Signed-off-by: Lukas Puehringer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Description of issue or feature request:
@joshuagl has recently overhauled sslib's extra dependency handling. This included the configuration of different tests for different sets of installed dependencies (see #200 and #206.)
In the course of a code review we discussed the possibility of different coverage profiles that accurately report the tested lines depending on the relevant code branches. See #200 (comment) ff. for some ideas.
Current behavior:
Code branches that are executed due to missing optional dependency are generally excluded from code coverage.
Expected behavior:
Accurately report the tested lines depending on the relevant code branches for a given build.
The text was updated successfully, but these errors were encountered: