Acoustic Identity Separation β€” User Guide

Discovers latent acoustic identities inside a recording via AI clustering, then reorganizes material by identity. Powered by Python (numpy, scipy, scikit-learn, soundfile).

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 1.0 (2025) License: MIT License Citation: Cohen, S. (2025). Praat AudioTools Repo: https://github.com/ShaiCohen-ops/Praat-plugin_AudioTools
Contents:

What this does

This script implements an Acoustic Identity Separation engine β€” an AI-powered tool that discovers latent acoustic identities within a recording via machine learning clustering, then reorganizes the material by identity. The result is a new composition where different "personalities" within the source sound are separated and rearranged.

🧠 What are Acoustic Identities?

Acoustic identities are latent "personalities" within a sound β€” distinct acoustic states that recur throughout a recording. For example:

  • harmonic-stable: Clean, pitched, stable regions
  • sustained-resonant: Long, resonant tones
  • attack-dominant: Percussive onsets, transients
  • noisy-transitional: Breath, noise, transitions
  • chaotic: Complex, turbulent textures

The AI discovers these identities automatically via Gaussian Mixture Models on a rich feature space, then assigns each moment in the sound to one of the discovered identities.

Key Features:

Technical Implementation: (1) Feature Extraction: Praat extracts pitch, HNR, intensity, formants + Python computes spectral features. (2) Identity Discovery: GMM clustering on normalized feature space. (3) Characterization: Per-identity statistics and behavioral typing. (4) Event Segmentation: Group consecutive same-identity frames into events. (5) Layered Separation: Build per-identity audio layers. (6) Resynthesis: Apply selected mode to layers. (7) Output: Stereo or multi-channel.

Quick start

  1. In Praat, select exactly one Sound object (any duration, any content).
  2. Run script… β†’ select identity_separation.praat.
  3. Choose Preset (2-7 for specific strategies, 1 for custom).
  4. Set number of identities (2-8) and resynthesis mode (A-E).
  5. Choose output format (Stereo mix or Multi-channel).
  6. Set seed for reproducibility.
  7. Enable Draw_visualization for analysis display.
  8. Click OK β€” engine extracts features, runs Python analysis, separates identities, resynthesizes.
Quick tip: Start with Gentle preset (3 identities, layered) on a 10-20 second recording with varied texture. Enable visualization β€” you'll see identity profiles with percentages and behavioral types. Listen to the output β€” identities are panned across stereo field, creating spatial separation of acoustic personalities. The output appears as "source_identity" in the Objects window.
Important: PYTHON DEPENDENCIES β€” This script requires Python with numpy, soundfile, scipy, and scikit-learn installed. The script automatically detects Python installations. FIRST RUN may take 1-2 minutes as Python modules are imported. NUMBER OF IDENTITIES affects granularity β€” too few may merge distinct states, too many may over-segment. BEHAVIORAL TYPES are assigned based on acoustic features β€” they may not always match perceptual categories perfectly.

Identity Separation Theory

Feature Space

The feature vector per frame includes: Praat features: β€’ pitch, hnr, intensity, voiced flag β€’ formants f1-f4 and bandwidths b1-b4 Spectral features (computed in Python): β€’ flatness (Wiener entropy) β€’ flux (frame-to-frame spectral change) β€’ centroid, bandwidth, rolloff (85%) β€’ zero-crossing rate (ZCR) β€’ 8-band energy (log-spaced bands) All features are robustly scaled (RobustScaler) and clipped to [-5,5].

Identity Discovery via GMM

πŸ“Š Gaussian Mixture Models

Gaussian Mixture Models (GMM) find clusters in the feature space:

p(x) = Ξ£ Ο€_k Β· 𝒩(x | ΞΌ_k, Ξ£_k) where: Ο€_k = mixing coefficient ΞΌ_k = mean vector Ξ£_k = covariance matrix (full covariance)

For each frame i, the model provides:

  • Z[i] β€” identity label (0..n-1)
  • C[i] β€” confidence (max posterior probability)
  • probs[i,k] β€” full posterior matrix

The model is fitted with n_init=5, max_iter=300 for robust convergence.

Behavioral Types

🏷️ 5 Data-Driven Types

TypeAcoustic SignatureScore Components
harmonic-stableHigh HNR, stable pitch, low flatnessHNR (0.35), pitch stability (0.30), (1-flat) (0.20), voiced ratio (0.15)
sustained-resonantHigh HNR, low flux, low onset densityHNR (0.30), (1-flux) (0.25), (1-onset) (0.25), (1-intensity var) (0.20)
attack-dominantHigh onset density, high flux, high intensity variationonset density (0.35), flux (0.30), intensity var (0.20), (1-pitch stab) (0.15)
noisy-transitionalHigh flatness, low HNR, low voiced ratioflatness (0.35), (1-HNR) (0.30), (1-voiced) (0.20), flux (0.15)
chaoticLow pitch stability, high flatness, high flux, high variation(1-pitch stab) (0.25), flatness (0.25), flux (0.25), intensity var (0.25)

Each identity gets the best-fitting type. Duplicate types are resolved by assigning weaker identities their second-best type.

Resynthesis Modes

ModeDescriptionOutput
A β€” Layered reconstructionSum all identity layers back together (original temporal order), with spatial panningStereo: identities panned evenly across stereo field
B β€” Identity alternationOnly one identity audible at a time β€” alternates as they occur, with crossfadesStereo or mono
C β€” Identity recompositionGroup events by identity, then concatenate identity blocks sequentiallyNew timeline organized by acoustic personality
D β€” Identity morphingAt each sample, blend identity layers weighted by posterior probabilitiesSmooth transitions between identities
E β€” HybridizationUse spectral envelope of most harmonic identity to shape most chaotic oneHybrid timbre + remaining identities

Event Segmentation

Consecutive frames with same identity are grouped into events. Duration constraints: min = 200 ms max = 3 s Short runs are merged into neighbors; long runs are split at local points.

Output Formats

Stereo mix (default): β€’ Each identity layer is panned according to: pan = i / (n-1) β€’ gainL = cos(panΒ·Ο€/2), gainR = sin(panΒ·Ο€/2) β€’ All layers summed to stereo Multi-channel: β€’ One channel per identity (mono mix of that identity's events) β€’ Useful for further spatial processing in DAW

Preset Strategies

Preset 2: Gentle (3 identities, layered)

🌱 Subtle Separation

Identities: 3 | Mode: A (Layered)

Output: Stereo mix

Character: Gentle discovery of 3 broad identities, layered with spatial separation

Use on: General exploration, material with 2-3 clear acoustic states

Preset 3: Detailed (5 identities, layered)

πŸ” Fine-Grained Discovery

Identities: 5 | Mode: A (Layered)

Output: Stereo mix

Character: Finds 5 distinct identities, panned across stereo field

Use on: Complex textures with many acoustic states

Preset 4: Alternating voices (4 identities)

πŸ—£οΈ One at a Time

Identities: 4 | Mode: B (Alternation)

Output: Stereo mix

Character: Only one identity plays at a time β€” voices alternate as they occur

Use on: Dialogue, vocal improvisation, call-and-response material

Preset 5: Recomposition (4 identities, grouped)

πŸ”„ Identity Blocks

Identities: 4 | Mode: C (Recomposition)

Output: Stereo mix

Character: All events of identity 0, then identity 1, etc. β€” new timeline organized by personality

Use on: Creating variations, remixes, structural transformations

Preset 6: Morphing blend (3 identities)

🌈 Smooth Transitions

Identities: 3 | Mode: D (Morphing)

Output: Stereo mix

Character: Continuous blend between identities based on posterior probabilities

Use on: Ambient, evolving textures, gradual transformations

Preset 7: Hybrid filter (3 identities)

πŸŽ›οΈ Spectral Shaping

Identities: 3 | Mode: E (Hybridization)

Output: Stereo mix

Character: Most harmonic identity shapes most chaotic one via spectral envelope transfer

Use on: Creating hybrids, timbral morphing, sound design

Parameters & Controls

Identity Discovery

ParameterDefaultDescription
Number_of_identities4Number of latent identities to discover (2-8)

Resynthesis Mode

ParameterDefaultDescription
ModeAA=Layered, B=Alternation, C=Recomposition, D=Morphing, E=Hybridization

Output

ParameterDefaultDescription
Output_formatStereo mixStereo (panned) or Multi-channel (one channel per identity)
Seed42Random seed for deterministic results
Draw_visualization1Generate 5-panel analysis display
Play_result1Audition after processing

Visualization & Analysis

5-Panel Display

Acoustic Identity Separation Visualization: Panel 1: TITLE β€’ Script name, source name, preset, mode, identity count, seed Panel 2: INPUT WAVEFORM β€’ Gray waveform β€’ Title: "Original" β€’ Duration displayed Panel 3: OUTPUT WAVEFORM β€’ Blue waveform (channel 1 if multi-channel) β€’ Title: "Output" β€’ X-axis: Time (s) Panel 4: ORIGINAL SPECTROGRAM β€’ 0-5000 Hz spectrogram of original β€’ Title: "Original spectrogram" Panel 5: OUTPUT SPECTROGRAM β€’ 0-5000 Hz spectrogram of output β€’ Title: "Output spectrogram (mode X)" Panel 6: IDENTITY PROFILES β€’ Bar chart showing each identity's percentage β€’ Color-coded bars (up to 8 distinct colors) β€’ Labels: ID number, percentage, behavioral type β€’ Right column: HNR, flatness, mean duration stats β€’ Title: "Identity Profiles" Panel 7: INTENSITY COMPARISON β€’ X-axis: Time, Y-axis: dB β€’ Gray line = original intensity β€’ Blue line = output intensity β€’ Title: "Intensity: Grey = original | Blue = output" Panel 8: SUMMARY PANEL β€’ Events count, transitions, mean event duration β€’ RMS original vs output, ratio β€’ Mode, identities, format, seed, channel counts

Reading the Identity Profiles

What the bars mean:
  • Bar width: Percentage of total duration occupied by that identity
  • Bar color: Unique color per identity (consistent across runs with same seed)
  • Label: ID number, percentage, behavioral type (e.g., "harmonic-stable")
  • Stats: HNR, flatness, mean duration for that identity
  • Interpretation: The dominant identities (largest bars) characterize the sound's primary acoustic personalities

Comparing Spectrograms

What to look for:
  • Original: All identities mixed together
  • Output (Mode A): Should look similar but with spatial separation (not visible in spectrogram)
  • Output (Mode C): Should show blocks of similar texture grouped together
  • Output (Mode D): Should show smooth spectral evolution
  • Output (Mode E): May show hybrid timbral characteristics

Applications

Electroacoustic Composition

Use case: Discovering and reorganizing acoustic personalities within a recording

Technique: Detailed preset (5 identities) with Recomposition mode

Workflow:

Sound Design for Media

Use case: Creating evolving textures, hybrid timbres, or spatialized identities

Technique: Morphing or Hybridization presets on appropriate sources

Applications:

Music Production

Use case: Creating variations, remixes, or stems from a single track

Technique: Recomposition mode with different seed values

Examples:

Research & Education

Use case: Studying acoustic feature spaces and unsupervised learning

Technique: Compare different identity counts on same source, examine behavioral types

Learning outcomes:

Practical Workflow Examples

🎬 Film Scene: Multiple Characters

Goal: Create separate "character" tracks from a single recording

Settings:

  • Source: 30-second recording with two distinct sound sources
  • Preset: Gentle (3 identities, layered)
  • Output format: Multi-channel

Result: Each identity in its own channel β€” can be processed independently in DAW

🎚️ Electronic Music: Remix

Goal: Create new arrangement from 4-bar loop

Settings:

  • Source: 4-bar drum/bass loop
  • Preset: Detailed (5 identities, layered)
  • Mode: C (Recomposition) β€” group by identity

Result: New structure: all kick events, then all snare, then all hi-hat, then all bass, then all cymbals

πŸŽ™οΈ Voice Processing: Choral Effect

Goal: Create choral texture from solo voice

Settings:

  • Source: 10-second vocal phrase
  • Preset: Morphing (3 identities)
  • Mode: D (Morphing) β€” smooth blends

Result: Voice morphs between its own identities (e.g., sustained notes, consonants, breaths), creating evolving choral effect

Troubleshooting Common Issues

Problem: Python not found or missing packages
Cause: Python not installed, or packages missing
Solution: Install Python and required packages: pip install numpy soundfile scipy scikit-learn
Problem: All identities have same behavior type
Cause: Insufficient acoustic variety, or too many identities for simple source
Solution: Reduce number_of_identities, or use source with more variation
Problem: Output has clicks
Cause: Crossfade insufficient at event boundaries
Solution: Increase XFADE_SEC in Python script (currently 8ms)
Problem: Multi-channel output has silent channels
Cause: Some identities may have no events
Solution: Reduce number_of_identities, or check identity profiles in visualization
Problem: Processing very slow
Cause: Python startup overhead + GMM fitting
Solution: First run will be slow; subsequent runs faster. For long files, consider shorter duration.

Advanced Techniques

Custom feature weights:

In the Python script, modify the behavioral scoring functions to emphasize different acoustic properties for your specific material.

GMM covariance type:

The script uses "full" covariance. Experiment with "diag" or "tied" for different clustering behavior.

Event duration limits:

Adjust EVENT_MIN_DUR and EVENT_MAX_DUR in Python for different segmentation scales (e.g., shorter for granular, longer for phrases).

Multi-channel output processing:

Use the multi-channel output as input to other spatialization tools (e.g., Spectral Panning Mapper) for further spatial manipulation.