IRCAM RAVE Model Runner — Neural Audio Transformation

Loads a TorchScript RAVE .ts model and processes a Sound object using neural audio synthesis. Supports IRCAM RAVE models shipped with nn_tilde for real-time-style offline transformation.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 1.1 (2026) License: MIT License Repo: https://github.com/ShaiCohen-ops/Praat-plugin_AudioTools

Contents:

What this does Quick start RAVE Models Parameters Latent Walk Visualization Applications

What this does

This script implements neural audio transformation using RAVE (Realtime Audio Variational autoEncoder) models from IRCAM. It loads a TorchScript .ts model and processes an input Sound object, writing the result back into Praat as a new Sound. The pipeline uses a Python worker with torch to run the model inference.

What is RAVE? RAVE is a neural audio synthesis model that learns a latent space representation of audio. Unlike traditional DSP, RAVE can perform timbre transfer, style morphing, and creative resynthesis. The model encodes audio into a low-dimensional latent space, then decodes it back. By walking through this latent space, you can create evolving, non-linear transformations. This script integrates RAVE models (break.ts, engine.ts, isis.ts, etc.) into Praat with full visualization including input/output waveforms, spectrograms, and a 2D latent walk trajectory.

Key Features:

8 Built-in RAVE Models — break, darbouka, engine, InstantAlbania, isis, percussion, wheel, plus custom .ts models
Flexible Input/Output Shapes — auto-detect or force BCT, B1T, CT tensor shapes
Output Options — gain control (dB), peak/RMS/none normalization, mono/stereo forcing
Latent Walk Visualization — 2D projection of frame-level features (brightness × tonalness) showing the path through latent space
Rich Praat Graphics — input/output waveforms, side-by-side spectrograms, latent trajectory with time-coloured path (navy → violet → magenta → orange)

Technical Implementation: (1) Export Sound as WAV — temporary 16-bit PCM file. (2) Python inference — load TorchScript model, reshape input tensor according to user settings, run forward pass, apply gain and normalization. (3) Import result — read output WAV back into Praat. (4) Latent walk computation — Praat-native analysis: brightness = log(RMS_hi / RMS_lo) (hi >1500 Hz, lo <1500 Hz), tonalness = pitch_stability × sqrt(HNR+1). Normalized to [-1,1] for display. (5) Visualization — 8-cm canvas with four panels: input waveform, output waveform, input spectrogram, output spectrogram, latent walk trajectory, and summary stats.

Quick start

In Praat, select exactly one Sound object.
Run script… → IRCAM_rave_model.praat.
Select a RAVE model from the dropdown (break, engine, percussion, etc.) or specify a custom .ts file.
Set Models directory — path to folder containing .ts models (e.g., C:\...\nn_tilde\help).
Adjust output options: gain (dB), normalization (peak/RMS/none), output channels (auto/mono/stereo).
Set Walk frames (8–256) for latent trajectory resolution.
Enable Draw_visualization to see waveforms, spectrograms, and latent walk.
Click OK — Python runs inference, result appears as rave_out_originalname.

Quick tip: Start with the break.ts or engine.ts model — these work well on a variety of inputs. For drum loops, try percussion.ts. For voice or monophonic material, isis.ts or InstantAlbania.ts produce interesting textures. Use input_shape = auto for automatic tensor shape detection. Set normalize = peak to prevent clipping. The latent walk visualization shows how the input audio traverses the model's latent space over time.

Important: REQUIRES PyTorch — install with pip install torch. The script expects TorchScript .ts models (exported from RAVE training). Model directory must contain the selected .ts file. If Python or torch is missing, an error log is written and the script exits gracefully. The output duration matches input duration. For stereo input, the script uses channel 1 for latent walk analysis (mono downmix). Latent walk uses Praat-native pitch, harmonicity, and band-filtering — no additional Python calls.

RAVE Models (Built-in)

Model	Filename	Character / Typical use
break	break.ts	Drum breaks, rhythmic material, percussive resynthesis
darbouka	darbouka_onnx.ts	Middle Eastern percussion, sharp transients
engine	engine.ts	General-purpose, industrial textures, mechanical sounds
InstantAlbania	InstantAlbania.ts	Voice, speech, vocal-like transformations
isis	isis.ts	Sustained tones, ambient, drone material
percussion	percussion.ts	Drums, hits, unpitched percussion
wheel	wheel.ts	Cyclic textures, granular-like evolution
Custom	user-provided .ts	Any TorchScript RAVE model from nn_tilde or custom training

Model source: These models are typically located in the nn_tilde package folder. Default path examples: C:\Users\User\Documents\Max 9\Packages\nn_tilde\help (Windows) or /Users/username/Documents/Max 9/Packages/nn_tilde/help (macOS). The script uses the Models_directory field to locate .ts files.

Parameters & Configuration

Model & Paths

Parameter	Description
RAVE_model	Select from 8 presets or "Custom" to enter a filename
Custom_model_name	If Custom selected, enter .ts filename (extension optional)
Models_directory	Full path to folder containing .ts model files
Output_prefix	Prefix for output Sound object (default: "rave_out")

Output Processing

Parameter	Default	Description
Gain_dB	0.0	Output gain in decibels (positive = amplify, negative = attenuate)
Normalize	peak	none = no scaling, peak = scale to 0dBFS, rms = scale to -20dBFS RMS
Walk_frames	64	Number of analysis frames for latent walk trajectory (8–256)
Draw_visualization	yes	Generate Praat picture with waveforms, spectrograms, latent walk
Play_result	yes	Auto-play output after processing

Advanced: Tensor Shapes

Input_shape options: auto — Try BCT, then B1T, then CT (recommended) BCT — [1, channels, samples] batch × channels × time B1T — [1, 1, samples] force mono input to model CT — [channels, samples] no batch dimension Output_channels: auto — Keep model's native output channel count mono — Force mono output (average channels if stereo) stereo — Force stereo output (duplicate mono if needed)

Note on tensor shapes: Different RAVE models expect different input tensor formats. The "auto" setting attempts BCT first (most common), then B1T (mono-forced), then CT. If all fail, the script reports the error. Check model documentation if auto fails.

Latent Walk — 2D Proxy Trajectory

The script computes a perceptual proxy for latent space walking using only Praat-native analysis (no extra Python inference). For each of walk_frames evenly spaced windows, two coordinates are extracted:

Axis X — Brightness

Brightness = ln( RMS_high / RMS_low )

where RMS_low = 0–1500 Hz (low frequency energy), RMS_high = 1500 Hz–Nyquist (high frequency energy). The log ratio captures spectral tilt: brighter sounds have more high-frequency energy → positive values.

Axis Y — Tonalness

Tonalness = pitch_stability × √(HNR + 1)

pitch_stability = 1 – CV(F0) (coefficient of variation of fundamental frequency). Stable pitch → higher stability.
HNR (Harmonics-to-Noise Ratio) measures harmonic content vs. noise. √(HNR+1) scales the range.

High tonalness = clear pitch, low noise (e.g., sustained violin). Low tonalness = noisy, unpitched (e.g., percussion).

The trajectory is then normalized to the range [-1, 1] and drawn as a coloured path:

Navy → violet → magenta → orange — temporal gradient (dark = beginning, light = end)
Green circle = start position
Red circle = end position

Why this proxy? RAVE's true latent space is high-dimensional (e.g., 16–64 dimensions). This 2D projection uses acoustic features known to correlate with latent dimensions in many autoencoders: brightness correlates with spectral centroid dimensions, tonalness correlates with harmonic/noise separation dimensions. The path shows how the input audio moves through the model's perceptual space, which often aligns with meaningful timbral evolution.

Visualization (Praat Picture)

When Draw_visualization is enabled, the script generates a comprehensive 8×8 cm picture with five panels:

Panel	Content
Title bar	Model name, sound name, normalization mode, gain, input shape
Input waveform	Original sound (gray curve, channel 1 if stereo)
Output waveform	Processed sound (blue curve)
Input spectrogram	0–5000 Hz, Gaussian window, left half of canvas
Output spectrogram	0–5000 Hz, right half of canvas — compare spectral changes
Latent walk	2D trajectory (brightness × tonalness), time-coloured path, start/end markers, axes labels
Summary panel	Duration, channels, sample rates, RMS before/after, peak, model name

Interpreting the latent walk: The trajectory reveals how the input's timbral and pitch characteristics evolve. A tight cluster suggests consistent timbre. A large, wandering path indicates diverse spectral/pitch content. The coloured segments (navy→violet→magenta→orange) help track time progression without a separate legend.

Applications

Creative Sound Design

Use case: Resynthesize any sound through a RAVE model to discover unexpected timbres, morphs, and textures.

Technique: Apply break.ts to a vocal — create drum-like vocal percussion. Apply isis.ts to a piano — generate ambient evolving pads.

Timbre Transfer & Morphing

Use case: Imprint the spectral characteristics of one sound onto another via latent space interpolation.

Workflow: Encode two sounds separately, interpolate latent vectors, decode. (Requires custom Python scripting; this script runs a single forward pass per sound).

Experimental Music Composition

Use case: Use RAVE as an "instrument" that transforms input in non-linear, emergent ways.

Example: Feed a short drone into InstantAlbania.ts to generate speech-like formants. Feed field recordings into wheel.ts for cyclic, granular textures.

Teaching Neural Audio

Use case: Demonstrate autoencoder-based audio synthesis in a classroom setting.

Learning outcomes: Understand latent spaces, encoder-decoder architectures, real-time neural audio, and the relationship between acoustic features (brightness/tonalness) and latent trajectories.

Practical workflow: Drum loop → Percussive texture morph

Input: Short drum loop (2–4 bars).
Model: percussion.ts or break.ts.
Settings: gain = 0 dB, normalize = peak, input_shape = auto, output_channels = auto.
Result: The model resynthesizes the drum pattern with altered timbre — maybe tighter transients, different decay, or completely transformed hits.
Visualization: Latent walk shows rhythmic peaks aligning with drum hits.

Workflow: Voice → Instrumental morph

Input: Spoken phrase or sustained vowel.
Model: InstantAlbania.ts.
Settings: gain = +3 dB (boost), normalize = rms (consistent loudness).
Result: Voice becomes instrument-like — preserved pitch contour, transformed spectral envelope.
Visualization: Tonalness stays high (voiced), brightness varies with vowel formants.

Troubleshooting Common Issues:
• PyTorch not installed: Run pip install torch in terminal/command prompt.
• Model file not found: Verify the Models_directory path and filename. On Windows, use backslashes or forward slashes.
• Tensor shape error: Try input_shape = auto. If that fails, inspect model documentation for expected input dimensions.
• Output silent or distorted: Check gain (positive values may clip). Normalize = peak will prevent clipping. Some models produce very low output — increase gain to +6–12 dB.
• Latent walk looks like a dot: Input may have very consistent brightness/tonalness. Increase walk_frames or use a more varied audio source.
• Python crashes with no error log: Ensure torch is compatible with your Python version. Run the Python command manually from terminal to see full traceback.

Python Worker Details

Command line (called by Praat): python run_model_ts.py --input temp_in.wav --output temp_out.wav --model model.ts --error err.txt --gain 0.0 --normalize peak --input_shape auto --out_ch auto The worker: 1. Reads input WAV using Python's wave module (no torchaudio dependency). 2. Converts to torch tensor [channels, samples], range [-1, 1]. 3. Loads TorchScript model with torch.jit.load(). 4. Reshapes tensor according to input_shape and runs model.forward(). 5. Applies gain, normalization (peak/RMS), channel conversion (mono/stereo). 6. Writes output as 16-bit PCM WAV. 7. Writes error log if exception occurs — read by Praat for debugging.

Mathematical Deep Dive: Latent Walk Features

Brightness (Spectral Tilt)

Let x(t) be audio signal. After applying Hann-band filters: x_low(t) = bandpass(0 Hz, 1500 Hz) x_high(t) = bandpass(1500 Hz, Nyquist) RMS_low = sqrt( mean(x_low²) ) RMS_high = sqrt( mean(x_high²) ) Brightness = ln( RMS_high / (RMS_low + ε) ) Range: -∞ (dark) to +∞ (bright). Normalized to [-1,1] across frames.

Tonalness

Pitch contour F0(t) extracted via Praat's autocorrelation method. pitch_mean = mean(F0) over window pitch_std = std(F0) pitch_stability = 1 - min(1, pitch_std / (pitch_mean + ε)) HNR(t) extracted via Praat's harmonicity analysis (dB scale). hnr_linear = 10^(HNR/10) hnr_norm = sqrt(hnr_linear + 1) Tonalness = pitch_stability × hnr_norm Range: 0 (noisy, unpitched) to high (pure tone). Normalized to [-1,1].

Interpretation in RAVE context: These two axes correlate with perceptually relevant latent dimensions. In many trained autoencoders, the first few latent dimensions capture spectral envelope and periodicity — exactly what brightness and tonalness measure. The walk therefore provides an intuitive map of how the input traverses the model's representation space.