Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Adds initial support for kspec coverage tool #360

Closed
wants to merge 12 commits into from

Conversation

rajeevgopalakrishna
Copy link
Contributor

This PR adds support for a new kspec coverage tool.

Usage: slither-kspec-coverage contract.sol kspec.md

The tool currently checks which functions in the specified contract.sol have corresponding Klab specifications in the provided kspec.md. It outputs a table illustrating which contract functions have kspec funcs, kspec reached funcs or not covered funcs.

kspec funcs indicates contract functions that have corresponding Klab specifications. kspec reached funcs indicates contract functions that do not have corresponding Klab specifications themselves but are called by functions that do have Klab specifications. not covered funcs highlights contract functions that do not have corresponding Klab specifications. The last two categories of functions need attention because they do not have Klab coverage.

For example: slither-kspec-coverage ./kspec_coverage/test/token/token.sol ./kspec_coverage/test/token/spec.md produces:

+-----------------+---------------------------+-----------------------+---------------------+
| Contract Name   | kspec funcs               | kspec reached funcs   | not covered funcs   |
+=================+===========================+=======================+=====================+
| Token (100.00%) | transfer(address,uint256) | add(uint256,uint256)  |                     |
|                 |                           | sub(uint256,uint256)  |                     |
+-----------------+---------------------------+-----------------------+---------------------+
1/3 (33.33%) functions are directly covered by a kspec
2/3 (66.67%) functions are reached by a kspec
0/3 (0.00%) functions are not reached by a kspec
There is a 0.00% standard deviation among percentage of functions covered per contract

Copy link
Member

@montyly montyly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is probably worth to add tests with inherited contracts, I am not sure the coverage works as expected in these cases.

slither/tools/kspec_coverage/analysis.py Outdated Show resolved Hide resolved
slither/tools/kspec_coverage/analysis.py Outdated Show resolved Hide resolved
slither/tools/kspec_coverage/analysis.py Outdated Show resolved Hide resolved
slither/tools/kspec_coverage/analysis.py Outdated Show resolved Hide resolved
slither/tools/kspec_coverage/analysis.py Outdated Show resolved Hide resolved
slither/tools/kspec_coverage/analysis.py Outdated Show resolved Hide resolved
slither/tools/kspec_coverage/analysis.py Outdated Show resolved Hide resolved
@montyly
Copy link
Member

montyly commented Nov 1, 2019

We should also add unit tests to ensure it works on all the verified Solidity contracts from https://github.com/runtimeverification/verified-smart-contracts

function_arguments = [refactor_type(arg.strip().split(' ')[0]) for arg in function_arguments]
function_full_name = function_full_name[:start] + ','.join(function_arguments) + ')'
covered_functions.add((contract_name, function_full_name))
i += 1
Copy link
Member

@Xenomega Xenomega Nov 1, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would refactor this code, it's weak and too order dependent/specific. I can imagine it causing bugs. It was just kept this way because it worked without issue for the specific codebase it was designed for.

@rajeevgopalakrishna
Copy link
Contributor Author

Refactor complete. Uses json_utils.

@montyly montyly mentioned this pull request Nov 6, 2019
@montyly
Copy link
Member

montyly commented Nov 6, 2019

I am closing this in profit of #364

@montyly montyly closed this Nov 6, 2019
@montyly montyly deleted the dev-kspec-coverage-tool branch November 6, 2019 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants