Testing

AMMM3 uses pytest for automated tests, plus local verification scripts for the broader confidence checks that glue linting, smoke paths, and packaging together. The expected workflow is to run targeted tests while you iterate and then run the wider local verification commands before you finish substantial work.

Test Layout

When you change a specific module seam, add or update tests in the matching test area instead of only asserting through a broad end-to-end test.

Core Commands

Fast targeted runs

pytest tests/<path>/test_*.py -v
pytest tests/mmm/plotting/test_theme.py --no-cov -q
pytest tests/mmm/models/test_panel_serialize.py --no-cov -q

Use targeted runs first. They are faster to interpret and make regressions easier to localise.

Whole-suite pytest

make test

This installs the test extras and runs pytest with the local runtime defaults from the Makefile.

Local verification

make verify_local
make verify_local_all

make verify_local is the preferred baseline for substantial changes because it runs:

• Ruff formatting and lint checks on source, tests and tracked runner scripts
• MyPy on the configured shared modelling and pipeline modules
• the default pytest suite (slow tests remain opt-in)

Run make smoke_mmm separately for the default demo pipeline. This performs real sampling with the demo’s configured draw counts; it is not a fast unit check.

make verify_local_all adds the package build/install smoke path.

Packaging smoke

make verify_package

Run this when any of the following changed:

• packaging metadata in pyproject.toml
• import surfaces or compatibility facades
• bundled assets under src/abacus/
• install-time behaviour or README/package artefacts

Runtime Environment

Some test paths need writable cache directories. The recommended defaults are:

export PYTENSOR_FLAGS="base_compiledir=/tmp/pytensor,linker=py"
export JAX_PLATFORMS=cpu
export XDG_CACHE_HOME=/tmp

The local verification scripts apply these defaults automatically.

Special Cases

Plotting tests

Prefer tests that inspect stable properties such as axes, labels, colours, sizes, rcParams, and return types. Avoid brittle pixel-perfect assertions.

When needed, you can run the top-level plot suite the same way the local verification script does:

NUMBA_DISABLE_JIT=1 pytest tests/mmm/test_plot.py --no-cov -q

Save/load and compatibility work

If you change model serialisation, identity strings, or import compatibility, add tests that prove older saved data or old import paths still work where that compatibility is expected.

Packaging and bundled assets

If you add or move package data, use make verify_package so the change is checked against an installed wheel rather than only the editable repo checkout.

What to Run Before You Finish

Small, localised change

• Targeted pytest
• Targeted ruff check

Moderate code change

• Targeted pytest
• make check_lint
• make smoke_mmm

Broad or risky change

• make verify_local
• make verify_package if packaging or bundled assets changed

Writing Good Tests

• Test observable behaviour, not implementation noise.
• Keep fixtures close to the layer you are testing.
• Prefer additive compatibility tests when preserving old behaviour.
• Use small synthetic data where possible.
• For plotting and serialisation, assert the stable contract rather than fragile internals.