Online Latent-Event Relocation — User Guide

Deep Thermodynamic Recomposition — trains a lightweight autoencoder on-the-fly from the input audio, learns a latent space, then relocates events based on latent thermodynamic fields (temperature, affinity, regimes). No external models, no internet, pure numpy ML.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 1.0 (2025) License: MIT License Citation: Cohen, S. (2025). Praat AudioTools Repo: https://github.com/ShaiCohen-ops/Praat-plugin_AudioTools

Contents:

What this does Quick start Latent Relocation Theory Preset Strategies Parameters & Controls Visualization & Analysis Applications

What this does

This script implements an Online Latent-Event Relocation engine — a deep thermodynamic recomposition tool that trains a lightweight autoencoder on-the-fly from the input audio, learns a latent space, then relocates events based on latent thermodynamic fields (temperature, affinity, regimes). All machine learning is pure numpy — no external models, no internet, no PyTorch/TensorFlow.

🧠 What is Latent-Event Relocation?

This approach combines three powerful ideas:

Autoencoder learning: A neural network learns to compress each audio event into a low-dimensional latent vector, capturing its essential acoustic character
Thermodynamic fields: From the latent space, we derive per-event temperature (instability/novelty), affinity (similarity), and thermodynamic regimes (Crystal/Fluid/Gas/Plasma)
Event relocation: Events are reordered based on their latent properties — similar events attract, hot events scatter, cold events anchor, novel events become structural pivots

The result is a new composition that emerges from the latent structure of the original sound.

Key Features:

5 Preset Strategies — Gentle Lattice to Novelty Pivots, plus Custom
On-the-Fly Autoencoder — Pure numpy MLP with Adam optimizer, trained on log-mel patches of each event
Latent Thermodynamic Fields — Temperature (reconstruction error + KNN distance), affinity matrix, regime assignment
Regime-Specific Relocation — Crystal (anchor), Fluid (swap), Gas (scatter), Plasma (pivot)
Stability & Novelty Biases — Control anchoring strength and pivot selection
Event Segmentation — Praat-based segmentation using intensity peaks (200ms–3s)
Click-Free Reconstruction — Equal-power crossfades + post-splice smoothing
Comprehensive Visualization — 5-panel display with waveforms, spectrograms, regime bar, intensity comparison, stats

Technical Implementation: (1) Event Segmentation: Praat segments audio into 200ms–3s events using intensity peaks. (2) Mel Patches: Python extracts 40-band log-mel spectrograms, resized to 32 frames per event. (3) Autoencoder: Trains MLP with hidden layer (input→hidden→latent→hidden→output), ReLU activations, denoising, L2 reg, Adam. (4) Latent Fields: Temperature = reconstruction error + KNN distance; affinity = Gaussian kernel on latent distances; regimes from temperature + pitch stability. (5) Relocation: Regime-specific displacement formulas. (6) Reconstruction: Crossfaded concatenation.

Quick start

In Praat, select exactly one Sound object (any duration, any content).
Run script… → select latent_relocation.praat.
Choose Preset (2-6 for specific strategies, 1 for custom).
Set autoencoder parameters (learning steps, latent size).
Set relocation controls (intensity, stability bias, novelty bias).
Choose whether to preserve original duration.
Set seed for reproducibility.
Enable Draw_visualization for analysis display.
Click OK — engine segments, trains autoencoder, relocates events, reconstructs.

Quick tip: Start with Balanced Flow preset on a 10-20 second recording with varied texture. Enable visualization — you'll see the event boundaries (red lines) on the input waveform, the regime distribution bar, and displacement statistics. Listen to how events are relocated based on their learned latent properties. The output appears as "source_latent" in the Objects window.

Important: PYTHON DEPENDENCIES — This script requires Python with numpy, soundfile, and scipy installed (no scikit-learn needed). The script automatically detects Python installations. AUTOENCODER TRAINING happens on-the-fly and may take 30-60 seconds for 100 steps with many events. EVENT SEGMENTATION uses intensity peaks — if your material has few intensity changes, consider a different source. LATENT SIZE affects representation power — too small may lose detail, too large may overfit.

Latent Relocation Theory

Event Segmentation

Events are segmented using intensity peaks: 1. Extract intensity contour (10ms hop) 2. Find peaks as candidate boundaries 3. Enforce minimum duration (200ms) by merging 4. Enforce maximum duration (3s) by splitting Event boundaries are drawn as red lines on the input waveform in visualization.

Log-Mel Patch Extraction

📊 Per-Event Spectral Representation

For each event, we compute a fixed-size spectrotemporal patch:

STFT with 1024-point FFT, 256-sample hop, Hann window
40-band mel filterbank (20 Hz – Nyquist)
Log-magnitude: log(mel energy + 1e-10)
Resize to 32 frames via padding/truncation (center-crop for longer events, edge-pad for shorter)

Final patch shape: 40 × 32 = 1280 features per event

Numpy Autoencoder Architecture

Encoder: input (1280) → hidden (h) → latent (L) Decoder: latent (L) → hidden (h) → output (1280) Where: h = max(L×2, min(256, √(1280×L))) (geometric mean scaling) Activations: • Hidden layers: leaky ReLU (α=0.01) • Output: linear (reconstruction) Training: • Denoising: add Gaussian noise (σ=0.3, decaying) to input • Loss: MSE against clean target • L2 regularization: 1e-4 • Optimizer: Adam (β₁=0.9, β₂=0.999, ε=1e-8) • Learning rate: 0.003, decaying

Latent Thermodynamic Fields

🔥 From Latent Space to Thermodynamics

Temperature T(i):

T = (1 - stability_bias) × (0.5·err_norm + 0.5·knn_norm) + stability_bias × instability where: err_norm = normalized reconstruction error (novelty) knn_norm = normalized distance to k=3 nearest neighbors in latent space instability = 1 - pitch_stability (from Praat)

Affinity matrix A:

A[i,j] = exp(-||z_i - z_j||² / (2σ²)) where σ = median distance between events

Regime assignment:

Score = T×0.7 + (1-pitch_stab)×0.3	Regime
score < 0.25	Crystal (ordered, stable)
0.25 ≤ score < 0.50	Fluid (flowing, connected)
0.50 ≤ score < 0.75	Gas (dispersed, chaotic)
score ≥ 0.75	Plasma (extreme, novel)

Regime-Specific Relocation Rules

Crystal: target[i] = i + 0.1·intensity·pull (pull toward most similar event) Fluid: target[i] = i + direction·intensity·T·2.0 (direction = -1 if T > mean(T), +1 otherwise) Gas: target[i] = i + intensity·T·direction·0.3·N (direction away from similar events' center of mass) Plasma: if novelty_rank > 0.7: target[i] = N-1 - i·0.2 (move to beginning/end) else: target[i] = i + intensity·T·0.4·N·sin(i·2.3 + T·5.7)

Stability Anchoring & Novelty Pivots

Stability anchoring: target[i] = (1 - α)·target[i] + α·i where α = stability_bias × pitch_stability Novelty pivots: • Find top 20% of events by reconstruction error (true novelty) • Space them evenly as pivot positions • Blend each pivot's target with its pivot position (weight = novelty_bias × 0.5)

Preset Strategies

Preset 2: Gentle Lattice

🌸 Subtle, Ordered

Learning steps: 80 | Latent size: 6

Reloc intensity: 0.3 | Stability bias: 0.6 | Novelty bias: 0.1

Character: Gentle relocation, strong stability anchoring — preserves much of original order

Use on: Delicate material, minimal transformation

Preset 3: Balanced Flow

💧 Balanced Transformation

Learning steps: 100 | Latent size: 8

Reloc intensity: 0.5 | Stability bias: 0.3 | Novelty bias: 0.3

Character: Balanced relocation, moderate anchoring and pivoting

Use on: General purpose, exploration

Preset 4: Volatile Scatter

💨 Chaotic Dispersion

Learning steps: 150 | Latent size: 12

Reloc intensity: 0.8 | Stability bias: 0.1 | Novelty bias: 0.5

Character: Strong relocation, weak anchoring, strong novelty pivots — chaotic, scattered

Use on: Experimental, glitch, chaotic textures

Preset 5: Deep Recomposition

🔮 Deep Structural Change

Learning steps: 200 | Latent size: 16

Reloc intensity: 0.7 | Stability bias: 0.2 | Novelty bias: 0.4

Character: Large latent space, moderate intensity — deep structural reorganization

Use on: Complex material needing fundamental transformation

Preset 6: Novelty Pivots

📍 Novelty-Driven

Learning steps: 150 | Latent size: 10

Reloc intensity: 0.6 | Stability bias: 0.1 | Novelty bias: 0.8

Character: High novelty bias — rare events become structural pivots, reorganizing timeline around them

Use on: Material with rare, distinctive events; creating narrative arcs

Parameters & Controls

Autoencoder Parameters

Parameter	Default	Description
Learning_steps	100	Training iterations for autoencoder (20–500)
Latent_size	8	Bottleneck dimensionality (2–32)

Relocation Controls

Parameter	Default	Description
Relocation_intensity	0.5	Overall strength of event displacement (0–1)
Stability_bias	0.3	Anchoring strength for stable events (0–1)
Novelty_bias	0.3	Structural pivot role for rare events (0–1)

Output Parameters

Parameter	Default	Description
Preserve_duration	1	Maintain original length (pad/trim output)
Seed	42	Random seed for deterministic results
Draw_visualization	1	Generate 5-panel analysis display
Play_result	1	Audition after processing

Visualization & Analysis

5-Panel Display

Online Latent-Event Relocation Visualization: Panel 1: TITLE • Script name, source name, preset, latent size, seed Panel 2: INPUT WAVEFORM • Gray waveform with red dotted lines = event boundaries • Label: events count and duration • Title: "Original (N events)" Panel 3: OUTPUT WAVEFORM • Green waveform = relocated output • Title: "Relocated" • X-axis: Time (s) Panel 4: ORIGINAL SPECTROGRAM • 0-5000 Hz spectrogram of original • Title: "Original spectrogram" Panel 5: OUTPUT SPECTROGRAM • 0-5000 Hz spectrogram of relocated output • Title: "Relocated spectrogram" Panel 6: REGIME DISTRIBUTION BAR • Stacked bar showing percentage of events in each regime • Colors: Blue=Crystal, Green=Fluid, Orange=Gas, Red=Plasma • Title: "Regimes: Blue=Crystal | Green=Fluid | Orange=Gas | Red=Plasma" Panel 7: INTENSITY COMPARISON • X-axis: Time, Y-axis: dB • Gray line = original intensity • Green line = relocated intensity • Title: "Intensity: Grey = original | Green = relocated" Panel 8: SUMMARY PANEL • Relocation stats: events, moved, avg/max displacement • Regime percentages, mean temperature • Autoencoder loss (initial → final) • Parameters: steps, latent size, intensity, biases, seed • RMS comparison • Warnings if any

Reading the Event Boundaries

What the red lines show:

Each red vertical line is a detected event boundary
Events are between 200ms and 3s in duration
The number of events determines the granularity of relocation
Too few events (<10) may not provide enough structure for meaningful relocation
Too many events (>100) may create overly granular results

Interpreting Displacement Statistics

What the numbers mean:

Moved: Number of events that changed position in the order
Avg displacement: Average time shift (ms) of events from original positions
Max displacement: Largest single event shift
Regime percentages: How many events in each thermodynamic state
Mean temperature: Average latent temperature (0=cold, 1=hot)
AE loss reduction: How well the autoencoder learned the data (lower = better)

Applications

Electroacoustic Composition

Use case: Creating deep structural transformations from source material

Technique: Deep Recomposition or Novelty Pivots presets

Workflow:

Select a 20-60 second recording with clear event structure
Run with Deep Recomposition preset (latent=16, 200 steps)
Examine event boundaries and regime distribution
Listen to how events are relocated based on latent properties
Export and use as movement in larger work

Sound Design for Media

Use case: Creating evolving textures, structural variations

Technique: Volatile Scatter or Gentle Lattice on appropriate sources

Applications:

Horror: Volatile Scatter creates chaotic, unpredictable reordering
Ambient: Gentle Lattice preserves structure while subtly rearranging
Transitions: Balanced Flow creates moderate reordering for evolving textures

Music Production

Use case: Creating variations, remixes, or structural reimaginings

Technique: Balanced Flow with different seeds

Examples:

Drum loops: Events may separate hits; relocation creates new rhythmic patterns
Melodic phrases: Events may separate notes; relocation creates new melodies
Field recordings: Events separate sound sources; relocation creates new soundscapes

Research & Education

Use case: Studying autoencoder learning, latent spaces, thermodynamic analogies

Technique: Compare presets on same source, examine regime distributions

Learning outcomes:

Understand how autoencoders learn compressed representations
Explore relationship between latent space and acoustic properties
Observe how different relocation rules affect output structure
Connect thermodynamic concepts to audio organization

Practical Workflow Examples

🎬 Film Scene: Psychological Transformation

Goal: Create 45-second cue representing character's mental breakdown

Settings:

Source: 30-second normal conversation recording
Preset: Volatile Scatter
Custom: learning_steps=200, latent=12, reloc_int=0.9

Result: Speech events scattered chaotically — words disintegrate, reform, creating psychological tension

🎚️ Electronic Music: Remix

Goal: Create new arrangement of 8-bar loop

Settings:

Source: 8-bar electronic loop
Preset: Balanced Flow
Seed=1,2,3 for three variations

Result: Three different reorderings of the loop's events — fresh arrangements from same source

🎙️ Voice Processing: Narrative Restructuring

Goal: Reorganize spoken narrative for surreal effect

Settings:

Source: 20-second spoken story
Preset: Novelty Pivots
novelty_bias=0.9 (strong pivot effect)

Result: Rare words (novel events) become structural anchors — story reorganized around its most distinctive moments

Troubleshooting Common Issues

Problem: Python not found or missing packages
Cause: Python not installed, or packages missing
Solution: Install Python and required packages: pip install numpy soundfile scipy

Problem: Too few events detected
Cause: Source has few intensity peaks, or segmentation parameters inappropriate
Solution: Use source with more dynamic variation, or adjust min/max event duration in script

Problem: Autoencoder loss not decreasing
Cause: Too few steps, too small latent size, or data too complex
Solution: Increase learning_steps, increase latent_size, or use simpler source

Problem: Output has clicks
Cause: Crossfade insufficient at splice points
Solution: Increase XFADE_SEC in Python script (currently 8ms)

Problem: Output very short/long
Cause: preserve_duration=0, and relocation changed total length
Solution: Set preserve_duration=1 to maintain original length

Advanced Techniques

Custom event duration limits:

In the Praat script, modify minEventDur and maxEventDur (currently 0.200 and 3.000) for different segmentation scales.

Autoencoder architecture tuning:

In the Python script, modify the hidden layer size calculation or add more layers for deeper networks.

Regime threshold adjustment:

Modify the score thresholds in _assign_regimes() to change regime boundaries.

Multi-channel audio:

The script preserves multichannel audio throughout — mel patches are extracted from mono mix, but reconstruction maintains original channel count.