Acoustic Identity Separation β User Guide
Discovers latent acoustic identities inside a recording via AI clustering, then reorganizes material by identity. Powered by Python (numpy, scipy, scikit-learn, soundfile).
What this does
This script implements an Acoustic Identity Separation engine β an AI-powered tool that discovers latent acoustic identities within a recording via machine learning clustering, then reorganizes the material by identity. The result is a new composition where different "personalities" within the source sound are separated and rearranged.
π§ What are Acoustic Identities?
Acoustic identities are latent "personalities" within a sound β distinct acoustic states that recur throughout a recording. For example:
- harmonic-stable: Clean, pitched, stable regions
- sustained-resonant: Long, resonant tones
- attack-dominant: Percussive onsets, transients
- noisy-transitional: Breath, noise, transitions
- chaotic: Complex, turbulent textures
The AI discovers these identities automatically via Gaussian Mixture Models on a rich feature space, then assigns each moment in the sound to one of the discovered identities.
Key Features:
- 6 Preset Strategies β Gentle to Hybrid, plus Custom
- Rich Feature Space β Pitch, HNR, intensity, formants + spectral features (flatness, flux, centroid, bandwidth, rolloff, ZCR, 8-band energy)
- GMM Identity Discovery β Gaussian Mixture Models find latent acoustic identities
- Behavioral Characterization β Each identity gets a descriptive type (harmonic-stable, attack-dominant, etc.)
- 5 Resynthesis Modes β Layered, Alternation, Recomposition, Morphing, Hybridization
- 2 Output Formats β Stereo mix (with spatial separation) or Multi-channel (one channel per identity)
- Event Segmentation β 200ms-3s events grouped by identity
- Click-Free Reconstruction β Crossfaded concatenation with post-splice smoothing
- Comprehensive Visualization β 5-panel display with waveforms, spectrograms, identity profiles, intensity comparison
Technical Implementation: (1) Feature Extraction: Praat extracts pitch, HNR, intensity, formants + Python computes spectral features. (2) Identity Discovery: GMM clustering on normalized feature space. (3) Characterization: Per-identity statistics and behavioral typing. (4) Event Segmentation: Group consecutive same-identity frames into events. (5) Layered Separation: Build per-identity audio layers. (6) Resynthesis: Apply selected mode to layers. (7) Output: Stereo or multi-channel.
Quick start
- In Praat, select exactly one Sound object (any duration, any content).
- Run scriptβ¦ β select
identity_separation.praat. - Choose Preset (2-7 for specific strategies, 1 for custom).
- Set number of identities (2-8) and resynthesis mode (A-E).
- Choose output format (Stereo mix or Multi-channel).
- Set seed for reproducibility.
- Enable Draw_visualization for analysis display.
- Click OK β engine extracts features, runs Python analysis, separates identities, resynthesizes.
Identity Separation Theory
Feature Space
Identity Discovery via GMM
π Gaussian Mixture Models
Gaussian Mixture Models (GMM) find clusters in the feature space:
For each frame i, the model provides:
- Z[i] β identity label (0..n-1)
- C[i] β confidence (max posterior probability)
- probs[i,k] β full posterior matrix
The model is fitted with n_init=5, max_iter=300 for robust convergence.
Behavioral Types
π·οΈ 5 Data-Driven Types
| Type | Acoustic Signature | Score Components |
|---|---|---|
| harmonic-stable | High HNR, stable pitch, low flatness | HNR (0.35), pitch stability (0.30), (1-flat) (0.20), voiced ratio (0.15) |
| sustained-resonant | High HNR, low flux, low onset density | HNR (0.30), (1-flux) (0.25), (1-onset) (0.25), (1-intensity var) (0.20) |
| attack-dominant | High onset density, high flux, high intensity variation | onset density (0.35), flux (0.30), intensity var (0.20), (1-pitch stab) (0.15) |
| noisy-transitional | High flatness, low HNR, low voiced ratio | flatness (0.35), (1-HNR) (0.30), (1-voiced) (0.20), flux (0.15) |
| chaotic | Low pitch stability, high flatness, high flux, high variation | (1-pitch stab) (0.25), flatness (0.25), flux (0.25), intensity var (0.25) |
Each identity gets the best-fitting type. Duplicate types are resolved by assigning weaker identities their second-best type.
Resynthesis Modes
| Mode | Description | Output |
|---|---|---|
| A β Layered reconstruction | Sum all identity layers back together (original temporal order), with spatial panning | Stereo: identities panned evenly across stereo field |
| B β Identity alternation | Only one identity audible at a time β alternates as they occur, with crossfades | Stereo or mono |
| C β Identity recomposition | Group events by identity, then concatenate identity blocks sequentially | New timeline organized by acoustic personality |
| D β Identity morphing | At each sample, blend identity layers weighted by posterior probabilities | Smooth transitions between identities |
| E β Hybridization | Use spectral envelope of most harmonic identity to shape most chaotic one | Hybrid timbre + remaining identities |
Event Segmentation
Output Formats
Preset Strategies
Preset 2: Gentle (3 identities, layered)
π± Subtle Separation
Identities: 3 | Mode: A (Layered)
Output: Stereo mix
Character: Gentle discovery of 3 broad identities, layered with spatial separation
Use on: General exploration, material with 2-3 clear acoustic states
Preset 3: Detailed (5 identities, layered)
π Fine-Grained Discovery
Identities: 5 | Mode: A (Layered)
Output: Stereo mix
Character: Finds 5 distinct identities, panned across stereo field
Use on: Complex textures with many acoustic states
Preset 4: Alternating voices (4 identities)
π£οΈ One at a Time
Identities: 4 | Mode: B (Alternation)
Output: Stereo mix
Character: Only one identity plays at a time β voices alternate as they occur
Use on: Dialogue, vocal improvisation, call-and-response material
Preset 5: Recomposition (4 identities, grouped)
π Identity Blocks
Identities: 4 | Mode: C (Recomposition)
Output: Stereo mix
Character: All events of identity 0, then identity 1, etc. β new timeline organized by personality
Use on: Creating variations, remixes, structural transformations
Preset 6: Morphing blend (3 identities)
π Smooth Transitions
Identities: 3 | Mode: D (Morphing)
Output: Stereo mix
Character: Continuous blend between identities based on posterior probabilities
Use on: Ambient, evolving textures, gradual transformations
Preset 7: Hybrid filter (3 identities)
ποΈ Spectral Shaping
Identities: 3 | Mode: E (Hybridization)
Output: Stereo mix
Character: Most harmonic identity shapes most chaotic one via spectral envelope transfer
Use on: Creating hybrids, timbral morphing, sound design
Parameters & Controls
Identity Discovery
| Parameter | Default | Description |
|---|---|---|
| Number_of_identities | 4 | Number of latent identities to discover (2-8) |
Resynthesis Mode
| Parameter | Default | Description |
|---|---|---|
| Mode | A | A=Layered, B=Alternation, C=Recomposition, D=Morphing, E=Hybridization |
Output
| Parameter | Default | Description |
|---|---|---|
| Output_format | Stereo mix | Stereo (panned) or Multi-channel (one channel per identity) |
| Seed | 42 | Random seed for deterministic results |
| Draw_visualization | 1 | Generate 5-panel analysis display |
| Play_result | 1 | Audition after processing |
Visualization & Analysis
5-Panel Display
Reading the Identity Profiles
- Bar width: Percentage of total duration occupied by that identity
- Bar color: Unique color per identity (consistent across runs with same seed)
- Label: ID number, percentage, behavioral type (e.g., "harmonic-stable")
- Stats: HNR, flatness, mean duration for that identity
- Interpretation: The dominant identities (largest bars) characterize the sound's primary acoustic personalities
Comparing Spectrograms
- Original: All identities mixed together
- Output (Mode A): Should look similar but with spatial separation (not visible in spectrogram)
- Output (Mode C): Should show blocks of similar texture grouped together
- Output (Mode D): Should show smooth spectral evolution
- Output (Mode E): May show hybrid timbral characteristics
Applications
Electroacoustic Composition
Use case: Discovering and reorganizing acoustic personalities within a recording
Technique: Detailed preset (5 identities) with Recomposition mode
Workflow:
- Select a 20-60 second recording with diverse textures
- Run with Detailed preset, Mode C (Recomposition)
- Examine identity profiles to understand discovered personalities
- Listen to the reorganized version β a new structure based on acoustic states
- Export and use as movement in larger work
Sound Design for Media
Use case: Creating evolving textures, hybrid timbres, or spatialized identities
Technique: Morphing or Hybridization presets on appropriate sources
Applications:
- Ambient: Morphing mode creates smooth identity transitions
- Horror: Hybridization can blend stable and chaotic identities for unsettling timbres
- Sci-fi: Alternating voices creates dialogue-like textures from non-speech material
Music Production
Use case: Creating variations, remixes, or stems from a single track
Technique: Recomposition mode with different seed values
Examples:
- Drum loops: Identities may separate kick, snare, hi-hat, cymbals
- Vocal samples: Identities may separate sustained notes, consonants, breaths
- Instrumental phrases: Identities may separate attacks, sustains, releases
Research & Education
Use case: Studying acoustic feature spaces and unsupervised learning
Technique: Compare different identity counts on same source, examine behavioral types
Learning outcomes:
- Understand how GMMs discover latent structure in data
- Explore relationships between acoustic features and perceptual categories
- Observe how identity counts affect segmentation
- Hear how different resynthesis modes interpret the discovered identities
Practical Workflow Examples
π¬ Film Scene: Multiple Characters
Goal: Create separate "character" tracks from a single recording
Settings:
- Source: 30-second recording with two distinct sound sources
- Preset: Gentle (3 identities, layered)
- Output format: Multi-channel
Result: Each identity in its own channel β can be processed independently in DAW
ποΈ Electronic Music: Remix
Goal: Create new arrangement from 4-bar loop
Settings:
- Source: 4-bar drum/bass loop
- Preset: Detailed (5 identities, layered)
- Mode: C (Recomposition) β group by identity
Result: New structure: all kick events, then all snare, then all hi-hat, then all bass, then all cymbals
ποΈ Voice Processing: Choral Effect
Goal: Create choral texture from solo voice
Settings:
- Source: 10-second vocal phrase
- Preset: Morphing (3 identities)
- Mode: D (Morphing) β smooth blends
Result: Voice morphs between its own identities (e.g., sustained notes, consonants, breaths), creating evolving choral effect
Troubleshooting Common Issues
Cause: Python not installed, or packages missing
Solution: Install Python and required packages: pip install numpy soundfile scipy scikit-learn
Cause: Insufficient acoustic variety, or too many identities for simple source
Solution: Reduce number_of_identities, or use source with more variation
Cause: Crossfade insufficient at event boundaries
Solution: Increase XFADE_SEC in Python script (currently 8ms)
Cause: Some identities may have no events
Solution: Reduce number_of_identities, or check identity profiles in visualization
Cause: Python startup overhead + GMM fitting
Solution: First run will be slow; subsequent runs faster. For long files, consider shorter duration.
Advanced Techniques
In the Python script, modify the behavioral scoring functions to emphasize different acoustic properties for your specific material.
The script uses "full" covariance. Experiment with "diag" or "tied" for different clustering behavior.
Adjust EVENT_MIN_DUR and EVENT_MAX_DUR in Python for different segmentation scales (e.g., shorter for granular, longer for phrases).
Use the multi-channel output as input to other spatialization tools (e.g., Spectral Panning Mapper) for further spatial manipulation.