Unified Audiovisual Engine — User Guide
Two audiovisual synthesis modes in one script. AV Presets – signal analysis‑driven visuals (RMS, pitch, LFO, swarm, spiral, trochoid, Lissajous). Poly Composition – multi‑voice generative engine with configurable shapes, colours, movements, synthesis types, and pentatonic tuning.
What this does
Unified Audiovisual Engine v5.0 combines two distinct synthesis modes in a single script, sharing a common 3‑phase pipeline (audio synthesis → pre‑slice → frame‑locked playback).
- RMS → Size – amplitude‑driven circle (true analysis).
- Pitch → Y‑Position – pitch‑tracker dot (true analysis).
- Tone → Sharpness – the Morphing Star (LFO → geometry).
- Voices → Swarm – the Particle System (multi‑voice swarm).
- Phase → Rotation – the Hypnotic Spiral (polar dot array).
- FM Index → Geometry – the Trochoid Spirograph.
- Stereo Waveform → Path – Lissajous with phosphor trails.
- Shapes: Circle, Triangle, Square, Pentagon, Hexagon, Star, Plane (filled square), Points, Lines (from centre).
- Colours: Rainbow (per voice), Fire, Ice, Cyan, Magenta, Gold, White.
- Movement: Orbit, Spiral, Bounce (Lissajous paths), Pendulum (coupled XY), Expand/Contract.
- Synthesis: Pure Sine, FM, AM, Additive (harmonics 1–4), Pulse (6 partials).
- Voices: 1–8, each with a pentatonic‑based frequency (A2 root).
Quick start
- In Praat, run the script
Audiovisual_Sonification.praat(no sound selection needed). - Choose Mode:
- AV Presets – select a preset (1–7).
- Poly Composition – configure shape, colour, movement, synthesis, and number of voices.
- Set Total duration, Master volume, Show trails (poly mode), and Randomize seed.
- Click Run. The script synthesises audio, pre‑slices chunks, then plays back with synchronised visuals in the demo window.
demo window for real‑time graphics. Frame rate is fixed at ≈12 fps (step = 0.08 s). For long durations (e.g., 60 s), pre‑computation may take 10–20 seconds.
Mode 1: AV Presets (7 concepts)
1. RMS → Size (Amplitude‑Driven Circle)
Audio: 150 Hz sine with 1.5 Hz tremolo. Visual: circle radius mapped to RMS amplitude (5–125 pixels). Colour: white (quiet) → red (loud).
0.8·sin(2π·150·t)·(0.5 + 0.5·sin(2π·1.5·t))Visual:
radius = 5 + RMS·120
2. Pitch → Y‑Position (Pitch Tracker)
Audio: stepped sequencer (8 steps per second, 200–800 Hz). Visual: cyan dot whose Y‑position follows the detected fundamental frequency (mapped 200–800 Hz → 20–80 % canvas height).
3. Tone → Sharpness (The Morphing Star)
Audio: FM synthesis (300 Hz carrier, 600 Hz modulator with 5 Hz LFO depth). Visual: 10‑point star morphing between sharp star (inner radius 15) and regular decagon (inner radius 40) based on LFO.
4. Voices → Swarm (The Particle System)
Audio: 5 additive voices (200, 400, 600, 800, 1000 Hz, each with 100 Hz LFO). Visual: 5 cyan dots orbiting with different radii and speeds; white lines connect each to the centre.
5. Phase → Rotation (The Hypnotic Spiral)
Audio: stereo phase‑shifted signal (100 Hz and 100.5 Hz, LFO phase modulation). Visual: 40 dots in a spiral rotating at speed modulated by LFO.
6. FM Index → Geometry (The Trochoid Spirograph)
Audio: FM tone (250 Hz carrier, 500 Hz modulator) with time‑varying modulation index. Visual: trochoid parametric curve whose shape changes with FM index.
7. Stereo Waveform → Path (Lissajous + Phosphor Trails)
Audio: stereo perfect fifth (220 Hz left, 330 Hz right, phase offset). Visual: Lissajous figure (3:2 ratio) with 20‑point phosphor trail that fades over time.
L = 0.5·sin(2π·220·t), R = 0.5·sin(2π·330·t + π/4)Visual:
x = 50 + 45·sin(2π·3.0·t), y = 50 + 45·sin(2π·2.0·t + π/4 + phase_drift)
Mode 2: Poly Composition (multi‑voice generative engine)
Pentatonic tuning
Each voice is assigned a frequency from a pentatonic scale based on A2 = 110 Hz. Ratios: 1, 9/8, 5/4, 3/2, 5/3, 2, 9/4, 5/2. Voices cycle through the 8 ratios, so up to 8 voices each have a distinct pitch.
Synthesis types
- Pure Sine – simple sine wave, fundamental only.
- FM – frequency modulation with modulator at 2× carrier frequency.
- AM – amplitude modulation (tremolo) at 4 Hz.
- Additive – harmonics 1–4 with amplitudes 0.55, 0.25, 0.13, 0.07.
- Pulse – harmonics 1–6 with decreasing amplitudes (1, 0.5, 0.33, 0.25, 0.2, 0.17).
Shapes
- Circle – filled circle with radius modulated by voice phase.
- Triangle / Square / Pentagon / Hexagon – regular polygons, rotating.
- Star – 5‑pointed star with alternating inner/outer radii.
- Plane – filled square with rotating outline.
- Points – small dots (radius 2.5).
- Lines – line from centre to each voice’s position.
Movements
- Orbit – circular motion around centre.
- Spiral – radius expands/contracts sinusoidally.
- Bounce – Lissajous paths with different X/Y frequencies per voice.
- Pendulum – coupled XY oscillations.
- Expand / Contract – radial expansion/contraction.
Colour palettes
- Rainbow – hue cycles per voice.
- Fire – red to orange (voice index).
- Ice – blue to white.
- Cyan / Magenta / Gold / White – solid colours.
Trails (when enabled) draw a fading history of each voice’s position (14 points), creating a glowing trace effect.
Parameters & defaults
Mode selection
| Parameter | Options | Default | Description |
|---|---|---|---|
| Mode | AV Presets / Poly Composition | AV Presets | 。
AV Presets mode
Poly Composition mode
| Parameter | Options | Default | Description | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Shape | Circle / Triangle / Square / Pentagon / Hexagon / Star / Plane / Points / Lines | Circle | 。||||||||||||
| Color palette | Rainbow / Fire / Ice / Cyan / Magenta / Gold / White | Rainbow | 。||||||||||||
| Movement | Orbit / Spiral / Bounce / Pendulum / Expand / Contract | Orbit | 。||||||||||||
| Synthesis type | Pure Sine / FM / AM / Additive / Pulse | Pure Sine | 。||||||||||||
| Number of voices | 1–8 | 3 | 。
Shared parameters
3‑phase pipeline
For AV Presets: generates a 2‑channel master sound using a preset‑specific formula. For Poly Composition: additive synthesis of up to 8 voices with pentatonic frequencies and selected synthesis type.
Phase 2 – Pre‑compute analysis & pre‑slice
- For presets 1 and 2: RMS computed by sampling every 10th sample.
- For preset 2: full pitch tracking via
To Pitch(slower, but done once). - All frames are extracted as separate
Soundobjects (pre‑sliced) to avoid extraction during playback.
At each frame (≈12 fps, step = 0.08 s):
- Clear the demo window.
- Draw the visualisation based on pre‑computed data (AV Presets) or per‑voice positions (Poly Composition).
- Play the pre‑sliced audio chunk for that frame.
- Remove the chunk to free memory.
The pre‑computation phase ensures that during playback, only drawing and audio playback occur – no analysis or extraction, which could cause stuttering.
FAQ / troubleshooting
For preset 2 (pitch tracking) and long durations (e.g., 60 s), the To Pitch analysis can take 10–20 seconds. This is normal – the analysis is done once before playback. For faster testing, reduce Total duration to 5–10 seconds.
The script pre‑slices all audio chunks before playback. If stuttering occurs, your system may not be fast enough to play and draw simultaneously. Reduce Total duration or increase frame_step (hardcoded to 0.08 s; you can edit it in the script to 0.1 s for fewer frames).
The script updates the demo window during the pre‑computation phase. If it seems stuck, check the Praat Info window – the script prints progress. For very long durations, the pre‑computation may take a minute; this is normal.
The frequencies are based on A2 = 110 Hz with pentatonic ratios. Voices 1–8 cycle through the ratios: 1, 9/8, 5/4, 3/2, 5/3, 2, 9/4, 5/2. For more than 8 voices, the pattern repeats (not a true octave).
The trail length is 14 points. Trails are drawn oldest‑to‑newest so the brightest dot is on top. The fade is linear from 1.0 (newest) to 0.04 (oldest).
The frame rate is fixed at ≈12 fps (step = 0.08 s). This is a design choice – the visuals are rhythmic, not high‑fps animation. The audio, however, is generated at full 44.1 kHz sample rate and sounds smooth.