Testing

Testing#

The de-facto stack: pytest as the runner, hypothesis for property-based tests, coverage.py for code-coverage measurement. unittest is in the stdlib but most projects layer pytest on top of it (pytest can run unittest test classes unchanged).

Layout#

A pytest-native layout.

project/
  pyproject.toml
  src/mypkg/
    __init__.py
    widget.py
  tests/
    conftest.py        # shared fixtures
    test_widget.py
    test_api.py

Run:

$ pytest -v
$ pytest -v tests/test_widget.py::test_resize

A test#

# tests/test_widget.py
from mypkg.widget import Widget

def test_resize():
    w = Widget(10, 20)
    w.resize(100)
    assert w.width == 100
    assert w.height == 20

def test_invalid_resize():
    w = Widget(10, 20)
    with pytest.raises(ValueError, match="must be positive"):
        w.resize(-1)

Pytest runs every function that starts with test_ in any file that starts with test_. Failures show a colorized diff of the asserted expression.

Fixtures#

A fixture is a setup helper. Pytest injects fixtures into tests by name; they handle their own teardown via yield.

import pytest, tempfile, pathlib

@pytest.fixture
def tmp_db():
    with tempfile.NamedTemporaryFile(suffix=".db") as f:
        db = open_db(f.name)
        yield db
        db.close()

def test_insert(tmp_db):
    tmp_db.execute("INSERT INTO ...")

Built-in fixtures worth knowing.

Fixture	What it gives you
`tmp_path`	`pathlib.Path` for per-test scratch dir
`tmp_path_factory`	Same, session-scoped
`capsys` / `capfd`	Capture stdout / stderr from the code under test
`monkeypatch`	Patch attributes / env / sys.path; reverts after the test
`caplog`	Capture log records via `logging`
`pytestconfig`	Access CLI flags / config

Scopes (function, class, module, session) control how often the fixture’s setup runs.

Parametrize#

Run the same test against many inputs.

@pytest.mark.parametrize("a,b,want", [
    (1, 2, 3),
    (0, 0, 0),
    (-1, 1, 0),
])
def test_add(a, b, want):
    assert add(a, b) == want

Mocking#

def test_calls_api(monkeypatch):
    calls = []
    monkeypatch.setattr(http, "get", lambda url, **kw: calls.append(url))
    run()
    assert calls == ["https://api/x"]

# or with unittest.mock for the pure-stdlib version
from unittest.mock import patch, MagicMock
with patch("mypkg.client.fetch", return_value={"ok": True}):
    result = run()

Property-based tests#

Hypothesis generates inputs that try to break your code, then shrinks failing cases to the minimal counterexample.

from hypothesis import given, strategies as st

@given(st.lists(st.integers()))
def test_sort_idempotent(xs):
    assert sorted(sorted(xs)) == sorted(xs)

Coverage#

$ pip install pytest-cov
$ pytest --cov=src --cov-report=term-missing
$ pytest --cov=src --cov-report=html        # opens htmlcov/index.html

Coverage tells you which lines weren’t exercised; it doesn’t tell you whether tests are good. Aim for meaningful tests over a target percentage.

Test selection#

$ pytest -k "widget and not slow"       # by name expression
$ pytest -m smoke                       # by mark
$ pytest --lf                           # last-failed
$ pytest --ff                           # failed-first
$ pytest -x                             # stop on first failure
$ pytest --pdb                          # drop into pdb on failure
$ pytest -n auto                        # parallel (with pytest-xdist)

What to test#

Public API of each module, the contract callers depend on.
Boundary conditions, empty inputs, off-by-one, max sizes.
Error paths, pytest.raises(SomeError).
Bug regressions, write the failing test, then fix.
Critical workflows, end-to-end via a small in-process fixture.

What not to test: trivial getters, framework code, third-party libraries themselves.