Skip to content

core.pipeline_matrix

CSV-driven matrix generation and batch pipeline execution.

triton.core.pipeline_matrix enables reproducible parameter sweeps across multiple files. It generates cartesian product matrices from files and parameter combinations, then executes each row as an isolated pipeline run. The GUI can also show live progress while rows are executed and can group final outputs by parameter set into clean folders.

Key Functions

parse_parameter_specs(parameter_specs: Iterable[str]) -> dict[str, list[object]]

Parse CLI-style parameter specs into a dictionary of step options and values.

Parameters: - parameter_specs: Iterable of strings like "0.target_peak=0.5,0.8" or "vocode.n_bands=4,8,16"

Returns: Dict mapping step.option to list of values.

Example:

specs = parse_parameter_specs(["0.target_peak=0.5,0.8", "1.target_sr=16000,24000"])
# Result: {"0.target_peak": [0.5, 0.8], "1.target_sr": [16000, 24000]}

generate_matrix_csv(project: Project, pipeline: Pipeline, output_csv: Path, parameter_specs: list[str], files: list[str] | None = None) -> int

Generate a CSV matrix from files and parameter combinations.

Parameters: - project: Loaded project config - pipeline: Pipeline definition - output_csv: Path where matrix CSV will be written - parameter_specs: List of parameter specs (e.g., ["0.target_peak=0.5,0.8"]) - files: List of input file paths (relative to data/raw/ or absolute). If None, all files in data/raw/ are used.

In the GUI workflow, matrix CSVs are typically stored under metadata/matrices/<pipeline_name>/ so they can be browsed per pipeline in the Run Matrix panel.

Returns: Number of rows written to CSV.

Example:

from pathlib import Path
from triton.core import generate_matrix_csv
from triton.core.project import load_project_config, load_project_pipelines

project = load_project_config("my-project")
pipelines = load_project_pipelines("my-project")
pipeline = pipelines[0]

rows = generate_matrix_csv(
    project, 
    pipeline,
    output_csv=Path("matrix.csv"),
    parameter_specs=["0.target_peak=0.5,0.8"],
    files=["audio_001.wav", "audio_002.wav"],
)
print(f"Generated {rows} matrix rows")

run_matrix_csv(project: Project, pipeline: Pipeline, matrix_csv: Path, run_id: str | None = None) -> dict

Execute all rows in a matrix CSV.

Parameters: - project: Loaded project config - pipeline: Pipeline definition - matrix_csv: Path to CSV generated by generate_matrix_csv - run_id: Optional custom run ID. If None, a new one is generated.

In the GUI/CLI workflow, run_matrix_csv can optionally copy final outputs into final_by_params/set_####_* folders grouped by parameter combination, while still preserving the usual row-level run structure.

Returns: Dict with summary of all row execution results.

Example:

from triton.core import run_matrix_csv

result = run_matrix_csv(
    project,
    pipeline,
    matrix_csv=Path("matrix.csv"),
    run_id="sweep_001",
)

print(f"Completed {result['total_rows']} rows")
print(f"Successes: {result['successful']}, Failures: {result['failed']}")

CSV Format

The matrix CSV has this structure:

file 0.target_peak 1.target_sr
audio_001.wav 0.5 16000
audio_001.wav 0.8 16000
audio_002.wav 0.5 16000
... ... ...
  • file column: Input audio path (relative to data/raw/ or absolute)
  • Remaining columns: Step parameter overrides (step.option)
  • Empty cells are treated as "no override"

You can manually edit or filter rows before running.

GUI Matrix Sources

When running a matrix from the GUI, Triton supports three matrix CSV sources:

  • Saved: select from metadata/matrices/<pipeline_name>/
  • Upload: upload a CSV file in-session
  • Path: point to an absolute or project-relative CSV file

Design Notes

  • Cartesian product: all combinations of files × parameter values
  • Isolated outputs: each row gets its own subdirectory to prevent collisions
  • CSV-first design: easy to version control, automate, and audit sweeps
  • Parameter overrides are shallow (step-level only), no nested options
  • File paths are resolved in order: absolute path → relative to data/raw/ → relative to project root