Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CI: Only carry forward full-suite test coverage #60344

Merged
merged 1 commit into from
Nov 25, 2021
Merged

Conversation

frenck
Copy link
Member

@frenck frenck commented Nov 25, 2021

Proposed change

It seems like we are now using the "carry forward" feature of Codecov backwards.

In order to carry forward coverage, flags must be used and the carryforward setting must be set for that flag. If that matches, the test coverage can be carried forward to the next coverage result.

Right now, we only set flags on a partial build, thus only partial coverage is carried forward, causing coverage to be incomplete in sequential coverage calculation.

This PR flips this around, we still set flags for each integration (as they are nicely identified in the Codecov UI), but we also now set a full-suite flag for the full CI run. Adding to that a change to the codecov settings: full-suite is carried forward, while all other flags are not.

This should allow us to carry forward the full suite result to partial builds; hopefully solving the low coverage problems we see on some PRs.

codecode.yaml validated using:

cat codecov.yml | curl --data-binary @- https://codecov.io/validate

Type of change

  • Dependency upgrade
  • Bugfix (non-breaking change which fixes an issue)
  • New integration (thank you!)
  • New feature (which adds functionality to an existing integration)
  • Breaking change (fix/feature causing existing functionality to break)
  • Code quality improvements to existing code or addition of tests

Additional information

  • This PR fixes or closes issue: fixes #
  • This PR is related to issue:
  • Link to documentation pull request:

Checklist

  • The code change is tested and works locally.
  • Local tests pass. Your PR cannot be merged unless tests pass
  • There is no commented out code in this PR.
  • I have followed the development checklist
  • The code has been formatted using Black (black --fast homeassistant tests)
  • Tests have been added to verify that the new code works.

If user exposed functionality or configuration variables are added/changed:

If the code communicates with devices, web services, or third-party tools:

  • The manifest file has all fields filled out correctly.
    Updated and included derived files by running: python3 -m script.hassfest.
  • New or updated dependencies have been added to requirements_all.txt.
    Updated by running python3 -m script.gen_requirements_all.
  • For the updated dependencies - a link to the changelog, or at minimum a diff between library versions is added to the PR description.
  • Untested files have been added to .coveragerc.

The integration reached or maintains the following Integration Quality Scale:

  • No score or internal
  • 🥈 Silver
  • 🥇 Gold
  • 🏆 Platinum

To help with the load of incoming pull requests:

@probot-home-assistant probot-home-assistant bot added the small-pr PRs with less than 30 lines. label Nov 25, 2021
@frenck frenck merged commit f72e9ae into dev Nov 25, 2021
@frenck frenck deleted the frenck-2021-3081 branch November 25, 2021 16:35
@github-actions github-actions bot locked and limited conversation to collaborators Nov 26, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bugfix cla-signed small-pr PRs with less than 30 lines.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants