Pipeline Runner

This section covers the structured abacus.pipeline runner: how it loads a config and dataset, executes the retained stage sequence, and writes reproducible run artefacts to disk.

Use the agency workflow to define the evidence and review required before external decision use. Pipeline completion alone does not satisfy that standard.

Pages

• AI diagnostics advisor - Review retained evidence, inspect identification limits and request controlled modelling experiments.

• Runner Overview - How run_pipeline(...) works, which stages run, and when the optimisation stage is skipped.

• YAML Configuration - Which YAML keys the runner consumes and how they map to model build, data loading, holidays, and optimisation.

• Blocked Holdout Validation - What Stage 35 does, how to configure it, and how to read the holdout metrics and plots.

• CLI Reference - The thin python -m abacus.pipeline.runner interface and its supported flags.

• Output Directory Schema - The run directory layout, manifest schema, stage statuses, and main artefacts.

• Extending the Runner - How to add a stage or wire in reporting without bypassing the manifest and artifact helpers.