RF Concatenative — Random Forest Corpus Synthesis

A multi-target Random Forest models local feature-state transitions X_t → X_{t+k}. Its prediction is blended with a synthetic target trajectory in standardised corpus space, then matched against a grain pool by L2 distance. The generated output has the requested duration, preserving sample rate and channel count.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 0.3 (2026) License: MIT License Repo: https://github.com/ShaiCohen-ops/Praat-plugin_AudioTools

Contents:

What this does Quick start 5 Presets Pipeline Feature Space Applications

What this does

This script implements Random Forest concatenative synthesis. It reads a source recording, cuts it into overlapping grains, and describes each grain with a 17-dimensional timbre vector (RMS, spectral centroid, spectral flatness, zero-crossing rate, MFCCs 1–13). A multi-target Random Forest models local feature-state transitions X_t → X_{t+k}. Its prediction is blended with a synthetic target trajectory in standardised corpus space, then matched against the grain pool by L2 distance. The generated output has the requested duration and intentionally starts at t=0. Sample rate and source channel count are preserved.

What is concatenative synthesis? Concatenative synthesis builds new sounds by splicing together grains (small fragments) from a source recording. This script adds a machine learning layer: a Random Forest learns how the source's timbral features transition from one grain to the next. This forest prediction is blended with a user-defined trajectory (e.g., crescendo_brightening, spectral_descent, random_walk) to guide the selection of grains. The result is a new sound that follows the requested trajectory while maintaining the natural transitions learned from the source.

Key Features:

5 Presets — Flowing Crescendo, Spectral Descent, Organic Wander, Tight Continuity, Fragmented Mosaic
17-dimensional feature space — RMS, centroid, flatness, ZCR, MFCCs 1–13
Multi-target Random Forest — models transitions X_t → X_{t+k} with OOB diagnostics
3 Trajectory types — crescendo_brightening, spectral_descent, random_walk
Trajectory weight — balance forest prediction vs. target trajectory
Repeat penalty — discourages reusing the same grain
RMS level matching — match output RMS to input (capped at +12 dB)
Multichannel support — cancellation-safe mono analysis, all channels preserved
Visualisation — input/output waveforms, grain map, match distance, optional spectrograms

OOB diagnostics: The Random Forest reports OOB row R² as a diagnostic (optimistic with overlapping grains). More importantly, it reports OOB skill versus persistence — whether the forest beats the trivial predictor Y=X on held-out bootstrap rows. A positive skill value indicates the forest has learned meaningful transitions beyond simply repeating the previous grain.

Quick start

In Praat, select exactly one Sound object.
Run script… → RF_Concatenative.praat.
Choose a preset from the dropdown (5 options, plus Custom).
Set Target_duration_s (output length).
If Custom, adjust grain analysis, trajectory, forest parameters, and output options.
Click OK — the script calls the Python engine, which trains the forest, walks the trajectory, and resynthesises the output.
Result rf_concatenative appears in the Objects window.

Quick tip: Start with Flowing Crescendo for a smooth rise in energy and brightness. Spectral Descent produces a falling spectral register. Organic Wander uses a random walk trajectory with longer grains. Tight Continuity prioritises forest-led transitions (low trajectory weight). Fragmented Mosaic uses short grains and high repeat penalty for a stuttering texture. Enable Show_spectrograms to compare input and output spectra.

Important: Python dependencies required: pip install numpy scipy librosa scikit-learn soundfile. The first run uses an embedded engine; subsequent runs use the installed version if available. The script uses a cancellation-safe mono fold for analysis (mean of channels) but applies the chosen grain indices to all source channels during resynthesis. RMS matching (Match_input_RMS) is applied after overlap-add and before the safety peak ceiling, capped at +12 dB to prevent pathological amplification.

5 Presets

Preset	Grain (ms)	Hop (ms)	Trajectory	Traj Weight	Repeat Penalty	Trees	Depth	k	Character
Flowing Crescendo	120	30	crescendo_brightening	0.65	0.60	100	15	1	Smooth rise in energy/brightness
Spectral Descent	100	25	spectral_descent	0.70	0.65	100	15	1	Falling spectral register
Organic Wander	140	35	random_walk	0.45	0.90	120	18	1	Forest-led random walk, stronger repeat avoidance
Tight Continuity	80	20	random_walk	0.25	0.45	150	20	1	Continuity-first, low target pull
Fragmented Mosaic	60	15	random_walk	0.85	1.60	80	12	2	Short grains, k=2 jumps, target-led

Pipeline — From Source to Synthesis

Stage 1 — Load source audio
Preserve all channels, build a cancellation-safe mono fold for analysis.

If the fold-down is < 10% of the loudest channel's RMS, use the loudest channel instead (cancellation fallback).
Analysis source is reported in the stats.

Stage 2 — Grain features
Window into overlapping grains (frame_len, hop_len) and extract 17 features:

[0] RMS energy
[1] Spectral centroid
[2] Spectral flatness
[3] Zero-crossing rate
[4..16] MFCCs 1–13 (coefficient 0 is gain, so dropped)

Standardise the whole pool with StandardScaler (mean 0, SD 1). Stage 3 — Random Forest transition model
Train a multi-target RandomForestRegressor on transition pairs:

X = Z[:n-lookahead]
Y = Z[lookahead:]
Report OOB row R² and OOB skill vs persistence (Y=X baseline).

Stage 4 — Target trajectory
Synthesise a trajectory of the requested duration in standardised space:

crescendo_brightening: Quiet/dull → loud/bright
spectral_descent: Falling spectral register
random_walk: Ornstein-Uhlenbeck walk (mean-reverting)

Stage 5 — Forest-guided walk
At each output step:

RF predicts the natural continuation of the current state.
Blend prediction with target trajectory: goal = (1 - w) × pred + w × target
Match the blend against the grain pool by L2 distance.
Apply repeat penalty to discourage reusing the same grain.

Stage 6 — Overlap-add resynthesis
Apply chosen grain indices to all source channels:

Hann-window and overlap-add.
Divide by accumulated window envelope.
Optionally match global RMS to the source (capped at +12 dB).
Apply attenuation-only safety ceiling.
Write with soundfile (32-bit float).

Stage 7 — Stats for Praat
Write key=value stats file for the Praat front end.

Multichannel handling: The analysis uses a mono fold (or the loudest channel in case of cancellation) so the RF sees a consistent feature sequence. However, the selected grain indices are applied to all source channels during resynthesis. This preserves the spatial image of multichannel recordings while using a stable analysis source.

Feature Space — 17 Dimensions

RMS energy

Root-mean-square amplitude — loudness proxy.

Spectral centroid

Centre of gravity of the spectrum — brightness.

Spectral flatness

Tonal vs. noise-like (1 = white noise, 0 = pure tone).

Zero-crossing rate

Noisiness / spectral slope proxy.

MFCCs 1–13

Mel-frequency cepstral coefficients — spectral envelope shape. Coefficient 0 (gain) is dropped; coefficients 1–13 are used.

Feature extraction: All spectral features are extracted from a single STFT per grain, avoiding redundant FFT work. MFCCs use a variable number of mel bands (capped at 40, adapted to short FFTs). The feature space is standardised (zero mean, unit variance) before training the Random Forest.

Applications

Trajectory-guided resynthesis (Flowing Crescendo)

Use case: Generate a new sound that follows a smooth rise in energy and brightness from the source material.

Settings: Flowing Crescendo preset. The output has the requested duration and follows the trajectory while maintaining natural transitions learned from the source.

Spectral descent (Spectral Descent)

Use case: Create a falling spectral register — the sound descends from bright to dark.

Settings: Spectral Descent preset. The trajectory pulls the spectral centroid and ZCR downward over time.

Organic variation (Organic Wander)

Use case: Generate a sound that wanders organically through the corpus's feature space.

Settings: Organic Wander preset. The random walk trajectory and higher repeat penalty produce varied, non-repeating textures.

Fragmented / stutter textures (Fragmented Mosaic)

Use case: Create a stuttering, fragmented texture from the source material.

Settings: Fragmented Mosaic preset (short grains, k=2 jumps, high repeat penalty). The output has a granular, mosaic-like character.

Workflow: Voice → Flowing crescendo

Source: Spoken word or singing.
Settings: Flowing Crescendo preset, Target_duration_s = 15.0.
Result: The voice is resynthesised as a smooth crescendo — the output starts quiet and dull, gradually becoming loud and bright over 15 seconds.

Workflow: Instrumental → Spectral descent

Source: Instrumental recording (piano, guitar, synth).
Settings: Spectral Descent preset, Target_duration_s = 10.0.
Result: The instrument's spectral content descends from bright to dark over time — a "melting" texture.

Workflow: Field recording → Organic wander

Source: Long field recording.
Settings: Organic Wander preset, Target_duration_s = 20.0.
Result: The field recording is re-synthesised as an organic, wandering texture — the Random Forest drives the transitions, creating a coherent but unpredictable evolution.

Troubleshooting:
• Python engine not found: The script embeds the engine if not installed. Ensure python3 is on your PATH and dependencies are installed.
• OOB skill vs persistence is negative: The Random Forest is not beating the persistence baseline at the chosen k. Try reducing k (lookahead) or increasing the number of trees.
• Output has clicks at grain boundaries: Increase the overlap (reduce hop_ms relative to frame_ms). The safe default is hop ≤ 25% of frame. The warning in the log will indicate if overlap is too low.
• Output is too quiet: The Match_input_RMS option applies a gain to match the source RMS. If the output is still quiet, the gain may have hit the +12 dB cap (reported in the stats as "RMS level match hit the +12 dB gain cap"). Increase the source volume or reduce the target duration.
• Visualisation grain map is sparse: The grain map shows a subsampled selection of grains (up to 500 points). For long outputs, the map shows the overall trajectory rather than every grain.

Visualisation

When Draw_visualization is enabled, the script generates:

Input / Output waveforms — grey (input) and blue (output).
Grain map — source time vs output time for each selected grain (subsampled). Shows how source material is re-sequenced.
Match distance — L2 distance between the requested state (forest + trajectory blend) and the grain actually selected.
Spectrograms (optional) — input and output spectrograms side by side.
Summary panel — RF parameters, OOB diagnostics, trajectory, distinct grain percentage, RMS gain, safety peak, runtime.

The grain map is the key diagnostic: it shows how the source material is being re-sequenced to follow the trajectory.