Testing

Testing#

Rust ships its own test runner. Any function annotated with #[test] becomes a test; cargo test discovers, builds, and runs them. Integration tests live under tests/; doc tests live inside /// comments; bench tests use criterion (nightly built-in benches are still experimental); property tests use proptest or quickcheck; fuzzers use cargo-fuzz or afl.

For assert! family in production, see Errors.

Unit tests#

Drop a #[cfg(test)] module inside the source file; cargo test compiles it only for test builds.

pub fn trim(s: &str) -> &str { s.trim() }

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn trim_removes_whitespace() {
        assert_eq!(trim("  hi  "), "hi");
    }

    #[test]
    #[should_panic(expected = "divide by zero")]
    fn divide_by_zero_panics() {
        divide(10, 0);
    }
}

$ cargo test
running 2 tests
test tests::trim_removes_whitespace ... ok
test tests::divide_by_zero_panics ... ok

Integration tests#

Each file under tests/ is a separate crate that exercises the library through its public API.

my-crate/
  src/lib.rs
  tests/
    integration.rs

// tests/integration.rs
use my_crate::Server;

#[test]
fn handles_ping() {
    let s = Server::new();
    assert_eq!(s.ping(), "pong");
}

Integration tests cannot access pub(crate) items; they see only the public API.

Doc tests#

Code blocks inside /// doc comments are compiled and run. Go-tos for keeping examples honest.

/// Returns the maximum of two values.
///
/// ```
/// assert_eq!(my_crate::max(2, 3), 3);
/// assert_eq!(my_crate::max(5, 1), 5);
/// ```
pub fn max(a: i32, b: i32) -> i32 {
    if a > b { a } else { b }
}

cargo test --doc runs only doc tests.

Assertions#

The standard set.

Macro

Meaning

assert!(cond)

panic if cond is false

assert_eq!(a, b)

a == b; prints both on failure

assert_ne!(a, b)

a != b

assert_matches!(x, pattern)

(nightly / assert_matches crate) x matches a pattern

panic!(msg)

explicit panic

unreachable!()

signal an impossible path

#[should_panic]

this test must panic to pass

Table-driven#

For functions with a small input space, write a single test with a table of cases.

#[test]
fn trim_table() {
    let cases = [
        ("clean",      "hi",      "hi"),
        ("trailing",   "hi   ",   "hi"),
        ("leading",    "   hi",   "hi"),
        ("both",       "  hi  ",  "hi"),
        ("empty",      "",        ""),
        ("whitespace", "   ",     ""),
    ];

    for (name, input, want) in cases {
        assert_eq!(trim(input), want, "case {name}");
    }
}

Helper fixtures#

Common setup goes into a regular function; the operator calls it from each test. cargo test runs tests in parallel by default (--test-threads=1 forces serial; useful when tests share global state).

fn fresh_server() -> Server {
    let s = Server::start(":0").expect("start");
    wait_for_ready(&s);
    s
}

#[test]
fn handles_ping() {
    let s = fresh_server();
    assert_eq!(s.ping(), "pong");
}

Mocking#

Rust’s stdlib has no mock framework; the operator either parameterises behaviour through a trait (and uses a fake impl in tests) or reaches for mockall.

trait Clock { fn now(&self) -> u64; }

struct SystemClock;
impl Clock for SystemClock { fn now(&self) -> u64 { /* … */ 0 } }

#[cfg(test)]
struct FixedClock(u64);
#[cfg(test)]
impl Clock for FixedClock { fn now(&self) -> u64 { self.0 } }

#[cfg(test)]
#[test]
fn uses_clock() {
    let c = FixedClock(1234);
    assert_eq!(timestamp(&c), 1234);
}

proptest (property-based)#

proptest generates random inputs against a property.

[dev-dependencies]
proptest = "1"

use proptest::prelude::*;

proptest! {
    #[test]
    fn rev_twice_is_identity(s in "\\PC*") {
        let twice: String = rev(&rev(&s));
        prop_assert_eq!(s, twice);
    }
}

Run property tests on parsers, codecs, and any function whose contract can be expressed without enumerating inputs.

cargo-fuzz#

LLVM libFuzzer driver. Catches panics, OOM, and sanitizer violations.

$ cargo install cargo-fuzz
$ cargo fuzz init
$ cargo fuzz add parse_target

// fuzz_targets/parse_target.rs
#![no_main]
use libfuzzer_sys::fuzz_target;

fuzz_target!(|data: &[u8]| {
    let _ = my_crate::parse(data);                       // must not panic
});

$ cargo fuzz run parse_target -- -max_total_time=300

Benchmarks (criterion)#

criterion runs statistical benchmarks on stable Rust.

[dev-dependencies]
criterion = "0.5"

[[bench]]
name = "parse"
harness = false

// benches/parse.rs
use criterion::{Criterion, criterion_group, criterion_main};

fn bench(c: &mut Criterion) {
    let data = std::fs::read("samples/big.bin").unwrap();
    c.bench_function("parse", |b| b.iter(|| my_crate::parse(&data)));
}

criterion_group!(benches, bench);
criterion_main!(benches);

$ cargo bench

Coverage#

Use cargo-llvm-cov (LLVM-based) or cargo-tarpaulin.

$ cargo install cargo-llvm-cov
$ cargo llvm-cov --html
$ open target/llvm-cov/html/index.html

References#

Contents