derive_fuzztest

This is not an officially supported Google product.

Description

Proc macros that generates both a fuzz target for use with cargo fuzz, and a property test (via quickcheck or proptest) for use with cargo test.

The reason for having both is that property testing allows for quick iteration to make sure the test works, and can be checked in presubmit CI, while fuzzing can test the input space more exhaustively and run continuously.

Example

#![cfg_attr(fuzzing, no_main)]

#[derive_fuzztest::fuzztest]
fn transitive_ord(a: u32, b: u32, c: u32) {
    if a >= b && b >= c {
        assert!(a >= c);
    }
    if a <= b && b <= c {
        assert!(a <= c);
    }
}

#[test]
fn additional_test_here() {
    /* ... */
}

Result reporting

Test functions report test failures by panicking, similar to how you would write a regular #[test]. If the annotated test function completes without panicking, the test is considered to have passed.

In some cases, you may want to discard some inputs, treating them neither as passes nor failures, just continue onto testing another generated input. This can be done by returning a TestResult from the annotated function. Property testing frameworks will try to generate more test cases to replace the discarded one, up to a certain limit. For fuzzing, test results that are discarded will not be added to the corpus.

use derive_fuzztest::TestResult;

#[derive_fuzztest::fuzztest]
fn increment(a: u8) -> TestResult {
    if a < u8::MAX {
        assert_eq!(a + 1 - 1, a);
        TestResult::Passed
    } else {
        TestResult::Discard
    }
}

Usage

Run the generated property tests

cargo test

Run continuous fuzzing

cargo install cargo-fuzz
cargo +nightly fuzz run <binary name>

Crate structure

If you use #[fuzz] or #[fuzztest], the fuzz target imposes the following requirements:

• The target must be in a separate [[bin]] target that only contains a single fuzz target.
• The crate containing the bin target has [package.metadata] cargo-fuzz = true
• The bin target is annotated with #![cfg_attr(fuzzing, no_main)]

The recommended structure for your crate foo is to put your tests under foo/fuzz/src/bin:

foo
├── fuzz
│   ├── src
│   │   └── bin
│   │       └── fuzz_target_1.rs
│   └── Cargo.toml
├── src
│   └── [project source]
└── Cargo.toml

This is different from the default structure generated by cargo fuzz init or cargo fuzz add so that we can take advantage of target auto-discovery. If you prefer, the default structure generated by cargo fuzz can also work, but make sure you remove test = false from the generated target in Cargo.toml.

You will also need to declare a dependency on the libfuzzer-sys crate, but only if fuzzing is requested:

[target.'cfg(fuzzing)'.dependencies]
libfuzzer-sys = "*"

(The reason for this conditional dependency is that libfuzzer-sys injects a main function to the resulting binary, and there will be linking failures if we link that in without defining a corresponding fuzz_target.)

Features

• quickcheck (default) — Enable generation of quickcheck property tests.
• proptest — Enable generation of proptest property tests.

Relevant reading

• Announcing Better Support for Fuzzing with Structured Inputs in Rust
• Bridging Fuzzing and Property Testing