Skip to content

Pipeline Overview

audio-prep has two independent pipelines.

Conversion Pipeline

Raw source audio
  -> discovery
  -> FFmpeg conversion
  -> validation
  -> JSONL manifest
  -> training-ready WAV or FLAC files

Conversion failures do not abort the whole run. Each source file receives a result, and failed files can be inspected through the manifest.

Chunking Pipeline

Raw source audio
  -> discovery
  -> FFmpeg decode and optional resample
  -> VAD speech detection
  -> chunk writing
  -> JSONL chunk manifest

Chunking is separate from conversion. The chunk command scans source audio files directly and writes chunks without first running convert.

Deterministic Output

Discovery returns sorted file paths so sequential and parallel runs produce results in the same order. Output paths mirror source subdirectories under the chosen output directory.

CLI Or Python

Run the pipelines directly from the command line:

audio-prep convert \
    --input-dir data/raw_mp3 \
    --output-dir data/wav16k \
    --manifest data/manifest.jsonl

audio-prep chunk \
    --input-dir data/raw_mp3 \
    --output-dir data/chunks \
    --manifest data/chunk_manifest.jsonl

Or call the same pipeline functions from Python:

from pathlib import Path

from audio_prep import ChunkConfig, ConversionConfig, chunk_batch, convert_batch

converted = convert_batch(
    Path("data/raw_mp3"),
    Path("data/wav16k"),
    ConversionConfig(),
)

chunks = chunk_batch(
    Path("data/raw_mp3"),
    Path("data/chunks"),
    ChunkConfig(),
)