Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement coverage reports using codecov #143

Merged
merged 1 commit into from
Jan 8, 2024
Merged

Conversation

kdarkhan
Copy link
Collaborator

@kdarkhan kdarkhan commented Jan 5, 2024

cargo-llvm-cov is used for generating coverage reports.

The library supports different report types which all have their own pros and cons:

  • --lcov - reports in lcov format which are supported by Codecov. They support only line level coverage and will not provide any info on regions. Example report can be seen here.
  • --codecov - Codecov's custom format which somewhat supports region coverage but not fully. Example report can be seen here. Notice that some lines are colored in yellow - which means they are partially covered but exact regions within lines are not highlighted.
  • --html - html output which is provided by llvm-cov. This output is the most informative as it highlights covered and uncovered regions. I pushed an example of this to Github Pages here. Unfortunately, Codecov does not support such outputs. We have an option to push these reports manually to some Github Pages repo during PR review. Or we can host these assets in Github Artifacts (GA). GA, however, will not make it easy to preview generated HTML - users will have to download the assets and open locally. More convenient preview in GA is blocked by this issue.

Given the above limitations, I propose using --codecov based approach which this PR implements.

I am not integrating reports provided by cargo fuzz coverage into this yet and planning to do it in a separate PR once we agree on the initial approach. cargo fuzz coverage generates an output which is closer to --lcov mentioned above. There is a hack that I think can be implemented to make it work with --codecov reports.

@codecov-commenter
Copy link

codecov-commenter commented Jan 5, 2024

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@mgeisler
Copy link
Collaborator

mgeisler commented Jan 8, 2024

Given the above limitations, I propose using --codecov based approach which this PR implements.

Excellent, this looks great! I completely agree that we should just use whatever format is easiest 😄

@kdarkhan kdarkhan merged commit 6293b88 into google:main Jan 8, 2024
7 checks passed
@kdarkhan kdarkhan deleted the coverage branch January 8, 2024 11:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants