Skip to content

Latest commit

 

History

History
167 lines (122 loc) · 6.59 KB

hpc_command.md

File metadata and controls

167 lines (122 loc) · 6.59 KB
Raw

The stack hpc commands

stack hpc COMMAND

Available commands:
  report                   Generate unified HPC coverage report from tix files
                           and project targets

Code coverage is a measure of the degree to which the source code of a program is executed when a test suite is run. Haskell Program Coverage (HPC) is a code coverage tool for Haskell that is provided with GHC. Code coverage is enabled by passing the flag --coverage to stack build.

stack hpc provides commands specific to HPC. Command stack hpc for the available commands.

The following refers to the local HPC root directory. Its location can be obtained by command:

stack path --local-hpc-root

The stack hpc report command

stack hpc report [TARGET_OR_TIX] [--all] [--destdir DIR] [--open]

The stack hpc report command generates a report for a selection of targets and .tix files.

Pass the flag --all for a report that uses all stored results.

Pass the flag --open to open the HTML report in your browser.

The extra-tix-files directory

During the execution of the build, you can place additional tix files in the extra-tix-files subdirectory in the local HPC root directory, in order for them to be included in the unified report. A couple caveats:

  1. These tix files must be generated by executables that are built against the exact same library versions. Also note that, on subsequent builds with coverage, the local HPC root directory will be recursively deleted. It just stores the most recent coverage data.

  2. These tix files will not be considered by stack hpc report unless listed explicitly by file name.

Examples

If we have three different packages with test suites, packages A, B, and C, the default unified report will have coverage from all three. If we want a unified report with just two, we can instead command:

stack hpc report A B

This will output to the standard output stream a summary report for the combined coverage from A and B's test suites. It will also log the path to the HTML for the corresponding full report.

This command also supports taking extra .tix files. If you've also built an executable, against exactly the same library versions of A, B, and C, then you could command the following:

stack exec -- an-exe
stack hpc report A B C an-exe.tix

or, equivalently:

stack exec -- an-exe
stack hpc report --all an-exe.tix

This report will consider all test results as well as the newly generated an-exe.tix file.

Usage

stack test --coverage is quite streamlined for the following use-case:

  1. You have test suites which exercise your project packages.

  2. These test suites link against your library, rather than building the library directly. Coverage information is only given for libraries, ignoring the modules which get compiled directly into your executable. A common case where this doesn't happen is when your test suite and library both have something like hs-source-dirs: src/. In this case, when building your test suite you may also be compiling your library, instead of just linking against it.

When your project has these properties, you will get the following:

  1. Summary coverage reports, sent to the standard output stream in the build output, and a log of the paths to the HTML for the corresponding full reports.

  2. A summary unified report, sent to the standard output stream, and a log of the path to the HTML for the corresponding full report. These reports consider the coverage on all local libraries, based on all of the tests that were run.

  3. An index of all generated HTML reports, in index.html in the local HPC root directory, and a log of the path to the HTML for that index.

Implementation details

Most users can get away with just understanding the above documentation. However, advanced users may want to understand exactly how --coverage works:

  1. The GHC option -fhpc gets passed to all project packages. This tells GHC to output executables that track coverage information and output them to .tix files. the-exe-name.tix files will get written to the working directory of the executable.

    When switching on this flag, it will usually cause all project packages to be rebuilt (see issue #1940).

  2. Before the build runs with --coverage, the contents of the local HPC root directory gets deleted. This prevents old reports from getting mixed with new reports. If you want to preserve report information from multiple runs, copy the contents of this path to a new directory.

  3. Before a test run, if a test-name.tix file exists in the package directory, it will be deleted.

  4. After a test run, it will expect a test-name.tix file to exist. This file will then get loaded, modified, and outputted to pkg-name/test-name/test-name.tix in the local HPC root directory.

    The .tix file gets modified to remove coverage file that isn't associated with a library. So, this means that you won't get coverage information for the modules compiled in the executable or test-suite stanza of your Cabal file. This makes it possible to directly union multiple *.tix files from different executables (assuming they are using the exact same versions of the project packages).

    If there is enough popular demand, it may be possible in the future to give coverage information for modules that are compiled directly into the executable. See issue #1359.

  5. Once we have a .tix file for a test, we also generate a summary report and a corresponding full report using HTML. The summary report is sent to the standard output stream. The index of the test-specific HTML report is available at pkg-name/test-name/index.html in the local HPC root directory.

  6. After the build completes, if there are multiple output *.tix files, they get combined into a unified report. The index of this report will be available at combined/all/index.html in the local HPC root directory.

  7. Finally, an index of the resulting coverage reports is generated. It links to the individual coverage reports (one for each test-suite), as well as the unified report. This index is available at index.html in the local HPC root directory.