Static Testing

Static Analysis

A static analyzer looks at code without running it. The point is to find bugs and optimizations by reasoning about how the code would run, e.g. array out of bounds errors, dead pointers. Think compiler warnings ––except the code never has to be fully compiled.

For more info: Wikipedia Page on Static Program Analysis

Cppcheck

The current (as of 13th Jan. 2020) static analyzer used is the Cppcheck static analyzer. It focuses on undefined behaviour and low false positives. It has customizable error suppression and output formating (see below)

For more info: Cppcheck website

And their manual: Cppcheck Manual

Installation

brew install cppcheck on Mac,

sudo apt-get install cppcheck on Linux. (or sudo yum install cppcheck or other equivalent)

See Cppcheck website for Windows users.

Brief Guide

For using on the local repo:

We recommend installing Cppcheck manually before use, otherwise running the analyzer will untar >30M of files in your local repo.

run 'make static' to run analyzer on ./src and ./test/src

pass STATIC_ENABLE={options} to lint with warnings etc.

the options are comma-separated list of: all, warnings, style, performance ... (more on cppcheck site/man page)

Add // cppcheck-suppress {error name} ABOVE offending line to suppress analyzer error. The error name will be reported in square brackets e.g. [constParameter] or [variableScope]

Add the error name itself to ./test/lib/cppcheck-suppress to disable that error completely

Extended Guide

Cppcheck from Command Line

Run cppcheck without arguments for a detailed man page.

Analyze a file with cppcheck some_file_here. Provide a directory instead to analyze all source and header files. e.g. cppcheck some_dir_here

By default, Cppcheck exits without error (return 0) even if there are errors/warnings etc. found in scanned files, as long as the static analysis process succeeds.

Pass --error-exitcode=1 to change this behaviour.

Pass --quiet flag to make Cppcheck only print errors/warnings/etc.

Cppcheck can detect additional errors by passing in the --enable=<id> flag These errors are broken into different levels, they are: all , warning , style , performance , portability , information , unusedFunction , missingInclude

For example, passing --enable=unusedFunction,missingInclude will make Cppcheck complain about, well, unused functions and missing includes, as well as things Cppcheck catches without the flag.

Error Suppression

File names and line numbers can be specified along with error name in the supression file.

e.g. constParameter:../src/utils/math/kalman_multivariate.cpp:102

Suppressions for Cppcheck can be specified from the command-line as well

e.g. cppcheck --suppress=variableScope:src/some_file.cpp:77 ./some_file_or_dir

As stated above, suppressions can be done from source/header files by adding // cppcheck-suppress {error name} ABOVE line where an error occurs. Also, the --inline-suppr flag must be provided to Cppcheck for this to work. This is done automatically in the local repo, so feel free to use inline suppressions.

As an aside, the suppression file can be in XML format, and passed in with the --suppress-xml=<filename>.xml flag.

Until/unless this is used in HYPED later on, please refer to the manual for more info.

Output

Due to (un)ease of installation of older versions, and the osx and Linux images used on Travis, (at time of implementing Cppcheck into our checks) the output format from Cppcheck on Travis may not match with the output format from Cppcheck locally.

Pass in --template=cppcheck1 flag locally in order to use the older format. Other templates are available as well, such as =gcc or =vs (for visual studio)

Alternatively, the command line man page describes how to customize this further. Further details on the Cppcheck Manual.

As a final note, Cppcheck can generate output in XML format with the --xml flag.

Until/unless this is used in HYPED later on, please refer to the manual for more info.