forge coverage should allow excluding contracts

[varunsrin]

Component

Forge

Describe the feature you would like

Running forge coverage automatically includes scripts and test utils that are contracts, which adds noise to the coverage report. It would be helpful if there was a way to exclude these from the reports

Additional context

No response

[antoncoding]

+1 on this feature, it would be helpful to be able to exclude certain folders too, given that the library coverage is not supported yet, it will make it easier for teams to build robust CICD pipelines if we can avoid the known issues 🙏

[flaskr]

+1, absolutely necessary for custom directory layouts and for CICD scripts to rely on forge coverage

[alexroan]

Looks like @onbjerg suggested that this might be done natively in #1576, but coverage always seems to include .t.sol files.

The build command supports a --skip test option, which excludes .t.sol files, but coverage kicks off a full rebuild. It would be great if that skip flag were able to be used with the coverage command.

[haythemsellami]

Is there any updates on an option to exclude certain contracts from coverage?

[alexroan]

A workaround for this is to retroactively remove unwanted files from the lcov report using the lcov package.

Example CI:

1. forge coverage --report lcov (this generates a ./lcov.info file)
2. sudo apt-get install lcov
3. lcov --remove ./lcov.info -o ./lcov.info.pruned 'exclusion/path/1' 'exclusion/path/2'
4. Run coverage check on ./lcov.info.pruned

[Rubilmax]

This would definitely be handy

[0xTimepunk]

@alexroan how do you re-run the coverage check in step 4 on ./lcov.info.pruned ?

[EdNoepel]

@alexroan how do you re-run the coverage check in step 4 on ./lcov.info.pruned ?

I do not see a way to do this from forge, but you can generate an HTML report using lcov:

genhtml ./lcov.info.pruned -o report --branch-coverage && firefox report/index.html

[distributedstatemachine]

@alexroan how do you re-run the coverage check in step 4 on ./lcov.info.pruned ?

you pass it to the coverage reporter

      - name: Report coverage
        uses: romeovs/[email protected]
        with:
          lcov-file: lcov.info.pruned
          github-token: ${{ secrets.GITHUB_TOKEN }}

[PaulRBerg]

As discussed in #1961, another workaround for this (if you'd prefer not to use @alexroan's solution) if you're using Codecov to track coverage is to ignore the test directory using a Codecov config file codecov.yml:

ignore:
    - "test"

[antoncoding]

One other way to bypass (get excluded by the report) it is to add a function prefixed with test in the contract. That way forge thinks it's a test contract and don't have it on the list.

// add this to be excluded from coverage report
function test() public {}

Works for me for those mocks or tester contracts I don't really care.

[yorhodes]

some of you might find this useful, especially if you are working with hardhat hybrid and node_modules deps https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/main/solidity/coverage.sh

[PaulRBerg]

Permalink: https://github.com/hyperlane-xyz/hyperlane-monorepo/blob/e91c3100c29d2913d175df4b3d1790d6a057d36e/solidity/coverage.sh

(Links with main in them may get outdated)