Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate and HTML reports with the coverage report generated/calculated #19

Open
jorgecoca opened this issue Oct 27, 2020 · 5 comments
Open
Labels
feature A new feature or request p3 Issues that we currently consider unimportant

Comments

@jorgecoca
Copy link
Contributor

As a customer of this Github Action, I would love to be able to generate a visual/HTML report with the coverage of my project, so I can easily inspect what's the status of my testing strategy.

As an example, we can see how genhtml from lcov works:

genhtml ./coverage/lcov.info -o coverage

Screen Shot 2020-10-27 at 8 51 39 AM

Thank you! 💙

@mtwichel
Copy link

mtwichel commented Oct 27, 2020

Could a strategy to tackle this problem be?:

  • Build a template using Handlebars.js
  • Pull data from lcov-parse, as input for the template
  • Write the newly created pages to .html files
  • Either write a comment on the PR with the generated files, or add them as artifacts in github actions

I think this might be an easy and simple way to generate the files we need without involving some big overhead from a framework.

EDIT: This commit shows a proof of concept that might be easier than explaining it, and here's a screenshot of the results:
Screen Shot 2020-10-28 at 12 16 38 AM

@felangel
Copy link
Contributor

Hey @mtwichel thanks so much for the contribution! The only downside to the handlebars approach that I can see is the individual file view where developers can drill down to see which line(s) aren't covered will need to be implemented as well. What do you think?

@jorgecoca
Copy link
Contributor Author

Wow @mtwichel thank you so much for contributing and helping us! I really appreciate the energy! 💙

I am leaning towards @felangel response at the moment: I would love to try to stay as close as possible to genhtml from lcov, or try to see if there's already an npm package that, given an lcov.info file, will generate the HTML report for us. If this does not exist, I think we should then create first that package, and then include it here in this project as a dependency (which I think that's the direction you were taking already).

@mtwichel
Copy link

Thanks for the replies! This is my first toe-dip into really contributing to open source so I apologize in advance if I don't fully get the conventions or standards. 🙂


To @felangel, I think we could make another template file for the line-by-line coverage detail, then read in the files by the path from lcov-parse and display the coverage in that template.


To @jorgecoca,

I would love to try and stay as close to possible to genhtml from lcov, or try to see if there's already an npm package that, given an lcov.info file, will generate the HTML report for us.

I'm curious about your motivation for that? I see that using an existing package that already generates the report file would be much easier and faster; however, I don't think the solution I presented is very challenging either. I'm also not convinced the report generated by the existing genhtml command is the best we can do. I think making the report's design a bit more clear and visually appealing would be a big plus. The template file technique gives us a lot of flexibility down the road.

I'm even curious if a generated report in html is the best solution for a github action? What if we posted a nicely formatted comment on the PR that displays the coverage you attained, as well as lines you missed? That seems even easier than opening a report and clicking through the files that don't have 100%.

I couldn't agree more on splitting off this chunk to a separate package though, maybe very-good-coverage-core? I would propose that package be in charge of:

  • Reading the lcov.info file, and reading the lines of the files under test
  • Returning a nicely structured (using Typescript) data format that's the result of the parse, and also includes the actual lines from the source files
  • Generating a nice set of html files as a report

That way this action can just consume that API and do whatever it wants with it (comment, artifact the reports, etc), and we could also build a CLI for running the tool locally like #20 discusses.

LMK your thoughts

@yeikel16
Copy link

I'd like to recommend coverde, a CLI written in Dart that allows you to manipulate coverage and is actually much easier to use than lcov. It would be great to be able to integrate the use of this.
It would only be necessary to install it with dart pub global activate coverde and use the coverde report -l command to generate the html.

@BeatriceMitchell BeatriceMitchell moved this from Backlog to Blocked in VGV Open Source 🦄 🧙🌟 Dec 19, 2022
@BeatriceMitchell BeatriceMitchell moved this from Blocked to Backlog in VGV Open Source 🦄 🧙🌟 Dec 19, 2022
@github-actions github-actions bot added feature A new feature or request and removed enhancement labels Feb 21, 2024
@tomarra tomarra added the p3 Issues that we currently consider unimportant label May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new feature or request p3 Issues that we currently consider unimportant
Projects
Status: Backlog
Development

No branches or pull requests

6 participants