CI: Only carry forward full-suite test coverage #60344
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Proposed change
It seems like we are now using the "carry forward" feature of Codecov backwards.
In order to carry forward coverage, flags must be used and the
carryforward
setting must be set for that flag. If that matches, the test coverage can be carried forward to the next coverage result.Right now, we only set flags on a partial build, thus only partial coverage is carried forward, causing coverage to be incomplete in sequential coverage calculation.
This PR flips this around, we still set flags for each integration (as they are nicely identified in the Codecov UI), but we also now set a
full-suite
flag for the full CI run. Adding to that a change to the codecov settings:full-suite
is carried forward, while all other flags are not.This should allow us to carry forward the full suite result to partial builds; hopefully solving the low coverage problems we see on some PRs.
codecode.yaml
validated using:cat codecov.yml | curl --data-binary @- https://codecov.io/validate
Type of change
Additional information
Checklist
black --fast homeassistant tests
)If user exposed functionality or configuration variables are added/changed:
If the code communicates with devices, web services, or third-party tools:
Updated and included derived files by running:
python3 -m script.hassfest
.requirements_all.txt
.Updated by running
python3 -m script.gen_requirements_all
..coveragerc
.The integration reached or maintains the following Integration Quality Scale:
To help with the load of incoming pull requests: