Quickstart: Pipeline Runner

Use the pipeline runner when you want a full staged run instead of only an in-memory model fit.

The runner writes:

• a run manifest
• copied and resolved config files
• fitted model artefacts
• posterior predictive assessment outputs
• decomposition, diagnostics, and response-curve artefacts

Fastest first run: bundled demo

From the repository root, the quickest way to see a real structured run is the demo launcher:

python3 runme.py --demo timeseries

Other bundled demos are:

• timeseries_controls
• geo_fe
• geo_cre
• geo_panel
• geo_brand_panel

timeseries and timeseries_controls use the released time_series estimator preset. geo_fe and geo_cre use the released one-unit FE and CRE presets. geo_panel and geo_brand_panel use the advanced dimensions.panel surface; they are not named RE, FE, or CRE estimators. The named RE preset remains release-gated. See the demo catalogue for the exact semantics and release status of each recipe.

List them explicitly with:

python3 runme.py --list-demos

runme.py is a convenience wrapper around the structured pipeline. It resolves the demo config under data/demo/<demo_name>/config.yml and runs the pipeline for you.

Run the named panel presets with:

python3 runme.py --demo geo_fe
python3 runme.py --demo geo_cre

Read Choose an Estimator before treating either recipe as the basis for a real model.

The demo YAML files contain evidence-oriented sampling settings. For a quicker workflow check, override the fit from the command line:

python3 runme.py --demo timeseries \
  --prior-samples 10 \
  --draws 500 \
  --tune 500 \
  --chains 2 \
  --cores 2 \
  --curve-samples 25 \
  --curve-points 50

The overrides above reduce the main fit only. The timeseries demo separately sets validation.sampler to four chains with 2,000 tuning and 2,000 retained draws each. Those settings take precedence over the main-fit overrides. For a bounded workflow check, copy the YAML beside the original, set validation.enabled: false or reduce its sampler settings, then pass the copy with --config. Keeping the copy beside the original preserves relative dataset paths. Do not treat a reduced run as final statistical evidence.

Run the pipeline from Python

The direct Python API is:

from pathlib import Path

from abacus.pipeline import PipelineRunConfig, run_pipeline

result = run_pipeline(
    PipelineRunConfig(
        config_path=Path("data/demo/geo_panel/config.yml"),
        output_dir=Path("results"),
        run_name="geo_panel_quickstart",
        prior_samples=10,
        draws=200,
        tune=200,
        chains=2,
        cores=2,
        random_seed=42,
        curve_samples=50,
        curve_points=50,
    )
)

print(result.run_dir)
print(result.manifest_path)

If the YAML config already contains data.dataset_path, you do not need to pass dataset_path again.

Run the thin CLI directly

The pipeline also exposes a thin CLI in abacus.pipeline.runner:

python3 -m abacus.pipeline.runner \
  --config data/demo/geo_panel/config.yml \
  --output-dir results \
  --run-name geo_panel_quickstart \
  --prior-samples 10 \
  --draws 200 \
  --tune 200 \
  --chains 2 \
  --cores 2 \
  --random-seed 42 \
  --curve-samples 50 \
  --curve-points 50

The CLI prints the final run directory and manifest when the pipeline completes. When the run manifest records them, it also prints paths to the estimator summary, diagnostics summary, and interpretation report. These paths are relative to the run directory.

Pipeline completion means that the configured stages finished. It does not mean that the fitted model passed its diagnostic gates or is suitable for interpretation. Review the listed diagnostic and interpretation artefacts before using model outputs.

Override data paths

Use one of these patterns:

Configured relative paths are resolved relative to the YAML config directory.

If you want Stage 50 to use different warn/fail cutoffs, add a runner-only diagnostics.thresholds block to the YAML. See YAML Configuration.

What you get back

run_pipeline(...) returns a PipelineRunResult with:

• run_dir
• manifest_path

The output directory contains stage folders such as:

• 00_run_metadata
• 10_pre_diagnostics
• 20_model_fit
• 30_model_assessment
• 50_diagnostics
• 55_ai_diagnostics_advisor, when ai_advisor is enabled
• 60_response_curves

Named estimator runs also record their resolved contract in 00_run_metadata/estimator_summary.txt and 00_run_metadata/estimator_manifest.yaml. FE writes its transformed within-design screen under 10_pre_diagnostics. CRE writes structural and reference summary-basis screens under 10_pre_diagnostics, a bounded post-fit screen under 20_model_fit, and separate CRE-adjustment decomposition evidence under 40_decomposition.

60_response_curves now includes three complementary curve families:

• saturation-only transformation artefacts
• forward-pass direct contribution artefacts built from scaled observed history
• adstock carryover artefacts

When to use the runner

Choose the runner when you want:

• a reproducible run directory on disk
• structured metadata and manifest files
• staged artefacts for diagnostics and reporting
• a config-driven workflow for repeated runs

If you only need to fit a model interactively in a notebook or script, start with Quickstart: Python API or Quickstart: YAML Builder.