Perceptual Graph Explorer: Audio Texture Segmentation

A Praat script that analyzes audio through perceptual feature extraction and k-means clustering to separate different texture types into distinct audio layers.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 1.0 (2025) Concept: Perceptual audio segmentation via feature clustering Repo: https://github.com/ShaiCohen-ops/Praat-plugin_AudioTools
Contents:
What this does Quick Start Perceptual Feature Theory Clustering Algorithm Parameters Explained Output & Interpretation Applications Visualization Guide

What this does

The Perceptual Graph Explorer analyzes audio by extracting three perceptual features from sliding windows, creating a 3D feature space, clustering similar windows together, and reconstructing separate audio streams for each cluster. Think of it as "color separation for audio" - different texture types are extracted into separate layers.

πŸ” Core Process Pipeline

  1. Windowing: Divide audio into overlapping frames (50ms windows by default)
  2. Feature Extraction: Calculate three perceptual features for each frame
  3. Normalization: Scale features to 0-1 range for clustering
  4. Clustering: Group similar frames using k-means algorithm
  5. Reconstruction: Create separate audio files for each cluster
  6. Visualization: Display feature space and cluster distributions

Key Innovation: Instead of traditional frequency-based analysis, this uses perceptual features that correspond to how humans hear textures. The result is intuitive separation based on "what sounds similar" rather than mathematical similarity.

Quick Start

  1. In Praat, select exactly one Sound object (mono or stereo).
  2. Open the script editor and load Perceptual_Graph_Explorer.praat.
  3. Choose a Preset or use Custom settings:
    • 2 Clusters - Basic Split: Good starting point
    • 3 Clusters - Detailed: More nuanced separation
    • 4 Clusters - Fine Analysis: For complex audio
  4. Adjust parameters if needed (see guide below).
  5. Enable Draw_visualization for graphical output.
  6. Click Run β†’ Run (or Ctrl+R).
  7. Listen to the resulting cluster sounds individually.
Best Practices:
  • Start with short audio (10-30 seconds) for testing
  • Use 3 Clusters - Detailed preset for most material
  • Enable visualization to understand the clustering
  • Listen to each cluster sound separately first
  • For speech: Try 2-3 clusters (voice, silence, noise)
  • For music: Try 3-4 clusters (melody, harmony, percussion, background)
Limitations & Requirements:
  • Audio length: Must have enough frames for clustering (at least 2Γ— number_of_clusters)
  • Processing time: Scales with audio length and number of clusters
  • Feature simplicity: Uses only 3 features - may miss subtle differences
  • Fixed clustering: k-means requires pre-specified cluster count
  • Overlap artifacts: Crossfade helps but may not eliminate all artifacts

Perceptual Feature Theory

The Three Perceptual Dimensions

πŸŽ›οΈ Dimension A: Energy (Intensity)

What it measures: Loudness/amplitude envelope of the audio frame

Praat Implementation: To Intensity: 100, 0, "yes" Normalized: 0 = minimum energy in file, 1 = maximum energy Perceptual Correlate: "How loud is this moment?"

Separation examples: Loud vs. soft sections, foreground vs. background, speech vs. silence

🎡 Dimension B: Stability (Harmonicity)

What it measures: Degree of periodicity/pitchedness in the frame

Praat Implementation: To Harmonicity (cc): 0.01, 75, 0.1, 1.0 Values: 0 = noisy/unpitched, 1 = perfectly periodic Normalized: Across entire file range Perceptual Correlate: "How pitched/tonal is this moment?"

Separation examples: Voice vs. noise, melody vs. percussion, harmonic vs. inharmonic

🌟 Dimension C: Brightness (Spectral Centroid)

What it measures: Center of gravity of the frequency spectrum

Praat Implementation: To Spectrum β†’ Get centre of gravity: 2 Units: Hertz (converted to normalized 0-1) Normalized: Across file frequency range Perceptual Correlate: "How bright/high-frequency is this sound?"

Separation examples: Bass vs. treble, dark vs. bright textures, low vs. high instruments

Why These Three Features?

Perceptual Foundations: These three features correspond to fundamental auditory dimensions:

  • Energy: Related to auditory attention (loud sounds grab attention)
  • Stability: Related to source identification (pitched = likely voice/instrument)
  • Brightness: Related to timbre perception and auditory stream formation

Computational Efficiency: Three features provide enough discrimination while remaining computationally tractable. More features could lead to overfitting or the "curse of dimensionality."

Interpretability: Each feature has clear perceptual meaning, making results interpretable to human listeners.

Feature Space Construction

FOR each window i at time t: 1. Extract window: [t, t + window_length] from audio 2. Calculate energy: intensity at window center 3. Calculate stability: harmonicity at window center 4. Calculate brightness: spectral centroid of window 5. Create 3D point: (energy_i, stability_i, brightness_i) 6. Normalize each dimension to [0, 1] range 7. Store in feature table with timestamp RESULT: N points in 3D unit cube, where N = number of windows

Clustering Algorithm: K-Means Implementation

🎯 Custom K-Means Clustering in Praat

Algorithm Steps:

1. INITIALIZATION: For k = 1 to number_of_clusters: Randomly select a data point as initial centroid centroid_k = (energy_random, stability_random, brightness_random) 2. ASSIGNMENT PHASE: For each data point i: Calculate Euclidean distance to each centroid distance_k = sqrt((e_i - e_k)Β² + (s_i - s_k)Β² + (b_i - b_k)Β²) Assign point to nearest centroid (cluster_k) 3. UPDATE PHASE: For each cluster k: Recalculate centroid as mean of all assigned points new_centroid_k = (mean_energy, mean_stability, mean_brightness) 4. CONVERGENCE CHECK: If centroid movement < threshold (0.001) β†’ converged Else repeat steps 2-3 (max 20 iterations)

Distance Metric: Euclidean in 3D Space

distance(point_i, centroid_k) = √[(energy_i - energy_k)² + (stability_i - stability_k)² + (brightness_i - brightness_k)²]

Why Euclidean distance? Simple, fast, and works well for normalized features. In the unit cube [0,1]Β³, Euclidean distance has clear geometric interpretation.

Cluster Characterization

🏷️ Automatic Cluster Labeling

Label generation based on feature averages:

Feature RangeLabelInterpretation
Energy > 0.66_HighEnergyLoud sections
Energy < 0.33_LowEnergyQuiet sections
Stability > 0.66_StablePitched/tonal
Stability < 0.33_UnstableNoisy/unpitched
Brightness > 0.66_BrightHigh-frequency content
Brightness < 0.33_DarkLow-frequency content

Example labels: Cluster_1_HighEnergy_Stable_Bright would be loud, pitched, high-frequency content (e.g., trumpet melody).

Audio Reconstruction from Clusters

FOR each cluster k: 1. Get all windows assigned to cluster k 2. Sort windows by time (already in temporal order) 3. Extract first window from original audio 4. For each subsequent window: - Extract window from original audio - Concatenate with previous using crossfade - Overlap_time controls crossfade duration 5. Result: Continuous audio containing only windows from cluster k

Crossfade importance: Without crossfade, concatenated windows would create clicks at boundaries. 5ms crossfade (default) provides smooth transitions.

Parameters Explained

Preset Configurations

PresetClustersWindow (ms)Step (ms)Overlap (ms)Best For
2 Clusters - Basic Split260305Simple separation (speech/background)
3 Clusters - Detailed350255Most audio (default)
4 Clusters - Fine Analysis440205Complex music, detailed analysis
CustomUserUserUserUserExperimental, specific needs

Analysis Parameters

πŸ”§ Window_length_ms (grain size)

Range: 20-200 ms (typical: 40-80 ms)

Effect:

  • Too small (＜30ms): Poor frequency resolution, noisy features
  • Optimal (40-80ms): Good tradeoff for most audio
  • Too large (＞100ms): Poor temporal resolution, misses rapid changes

Musical correlation: ~50ms β‰ˆ sixteenth note at 120 BPM

↔️ Step_size_ms (hop size)

Range: 10-50 ms (typically Β½ to ΒΌ of window length)

Effect:

  • Small step (high overlap): More frames, smoother reconstruction, slower processing
  • Large step (low overlap): Fewer frames, may miss transitions, faster processing

Default: 25ms provides 50% overlap with 50ms window (good balance)

🎯 Number_of_clusters

Range: 2-8 (practical limit based on audio complexity)

Choosing the right number:

  • 2 clusters: Binary separation (foreground/background)
  • 3 clusters: Most natural audio (voice, music, noise)
  • 4 clusters: Complex music (melody, harmony, percussion, effects)
  • ＞4 clusters: May create artificial splits

Rule of thumb: Start with 3, adjust based on results

Smoothing & Output Parameters

ParameterDefaultRangeEffect
Overlap_time_ms5 ms1-20 msCrossfade between concatenated windows
Scale_peak0.950.1-1.0Output amplitude normalization
Draw_visualizationOnOn/OffDisplay feature space and clusters
Play_resultOnOn/OffPlay first cluster after processing

Output & Interpretation

Generated Sound Objects

Output naming convention: Cluster_X_Description

Examples:

  • Cluster_1_HighEnergy_Stable_Bright - Loud, pitched, high-frequency
  • Cluster_2_MidEnergy_Unstable_Dark - Medium volume, noisy, low-frequency
  • Cluster_3_LowEnergy_Stable_MidBright - Quiet, pitched, mid-frequency

What each cluster contains:

  • Temporal structure: Windows appear in original time order
  • Gaps: Between windows from same cluster may have silence
  • Continuity: Crossfade provides smooth transitions within cluster
  • Duration: Same as original audio (with silences where other clusters were)

Information Window Output

=== PERCEPTUAL GRAPH EXPLORER === Source: myAudio (12.34 s) Preset: Detailed_3 1. Extracting Feature Nodes... 2. Constructing Graph Space... 3. Detecting Communities (k-means)... Converged after 8 iterations Cluster Characteristics: Cluster_1_HighEnergy_Stable_Bright: 245 frames Energy=0.82 Stability=0.75 Brightness=0.68 Cluster_2_MidEnergy_Unstable_Dark: 167 frames Energy=0.45 Stability=0.22 Brightness=0.31 Cluster_3_LowEnergy_Stable_MidBright: 89 frames Energy=0.18 Stability=0.81 Brightness=0.52 4. Generating Smoothed Audio Layers... === Done === Created 3 cluster sound(s) - Cluster_1_HighEnergy_Stable_Bright - Cluster_2_MidEnergy_Unstable_Dark - Cluster_3_LowEnergy_Stable_MidBright

Interpreting Cluster Statistics

Frame count: How many windows assigned to each cluster. Larger clusters = more prevalent texture.

Energy values (0-1):

  • 0.8-1.0: Very loud sections
  • 0.4-0.7: Medium volume
  • 0.0-0.3: Quiet sections

Stability values (0-1):

  • 0.8-1.0: Highly periodic (voice, sustained instruments)
  • 0.4-0.7: Moderately periodic
  • 0.0-0.3: Noise-like (percussion, fricatives, wind)

Brightness values (0-1): Normalized frequency position. Higher = brighter timbre.

Applications & Use Cases

Audio Analysis & Research

πŸ”¬ Texture Analysis in Soundscapes

Use case: Analyzing environmental recordings

Method:

  1. Record or load soundscape (forest, city, underwater)
  2. Use 4 Clusters - Fine Analysis preset
  3. Examine cluster characteristics
  4. Interpret: Which clusters correspond to which sources?

Example results:

  • Cluster 1: Bird songs (HighEnergy, Stable, Bright)
  • Cluster 2: Wind/leaves (MidEnergy, Unstable, MidBright)
  • Cluster 3: Distant traffic (LowEnergy, Unstable, Dark)
  • Cluster 4: Occasional events (variable)

Music Production & Sound Design

🎡 Isolating Musical Elements

Use case: Extracting drum groove from full mix

Method:

  1. Load music track with drums, bass, melody
  2. Use 3 Clusters preset
  3. Listen to each cluster separately
  4. Identify which cluster contains drums (typically Unstable, MidBright)
  5. Use that cluster as starting point for drum processing

Creative applications:

  • Create "stems" from finished mixes
  • Isolate vocal breaths/artifacts for removal
  • Extract specific instrument characteristics
  • Create texture-based effects

Speech & Voice Processing

πŸ—£οΈ Voice/Noise Separation

Use case: Cleaning noisy speech recordings

Method:

  1. Load noisy speech recording
  2. Use 2 Clusters - Basic Split preset
  3. Cluster 1 will likely be voice (Stable, MidEnergy)
  4. Cluster 2 will likely be noise (Unstable, variable Energy)
  5. Save Cluster 1 as cleaned speech
  6. Optional: Mix small amount of Cluster 2 back for naturalness

Limitations: Works best when voice and noise have different feature profiles. May struggle with similar noise types.

Educational & Cognitive Studies

🧠 Perceptual Grouping Demonstrations

Use case: Teaching auditory scene analysis principles

Method:

  1. Create or use audio with multiple concurrent streams
  2. Run perceptual graph analysis
  3. Show how clusters correspond to perceptual groups
  4. Discuss which features drive segregation
  5. Compare with Bregman's ASA principles

Example: Bach fugue - Show how different voices separate into clusters based on pitch range and timing.

Visualization Guide

πŸ“Š Understanding the Graphical Output

Six visualization panels:

PanelContentPurpose
TitleScript name, filename, presetContext
Original WaveformComplete input waveform (gray)Reference
Cluster WaveformsSeparate waveforms for each cluster (colored)Show temporal distribution
Feature Space2D plot: Energy vs Brightness, colored by clusterShow clustering in feature space
Cluster CentroidsBlack crosses in feature spaceShow cluster centers
Cluster DistributionBar chart of frame counts per clusterShow cluster sizes
ParametersSettings usedDocumentation

Color Coding

Cluster Colors: Cluster 1: RGB(0.7, 0.5, 0.6) - Pinkish purple Cluster 2: RGB(0.5, 0.6, 0.7) - Bluish Cluster 3: RGB(0.6, 0.7, 0.5) - Greenish Cluster 4: RGB(0.7, 0.6, 0.5) - Orangish Additional: Continue similar pastel variations Original Waveform: Gray (RGB(0.6, 0.6, 0.6))

Interpreting the Feature Space Plot

Axes:

  • X-axis (Energy): 0 = minimum energy, 1 = maximum energy
  • Y-axis (Brightness): 0 = darkest/lowest frequencies, 1 = brightest/highest frequencies

Points: Each dot represents one 50ms window of audio.

Clustering patterns:

  • Tight clusters: Distinct texture types
  • Spread along diagonal: Correlation between energy and brightness
  • Vertical spread at certain energies: Same loudness but different timbres
  • Horizontal spread at certain brightness: Same timbre but different volumes

Ideal clustering: Well-separated clusters with minimal overlap.

Troubleshooting Visualization Issues

Problem: All points in one corner of feature space
Causes: Poor feature range, all windows similar, normalization issue
Solutions: Check audio diversity, try different window size, examine raw features

Problem: Clusters overlap heavily in visualization
Causes: Too many clusters for audio complexity, poor feature discrimination
Solutions: Reduce number_of_clusters, examine which features are not discriminating

Problem: Visualization missing or incomplete
Causes: Audio too short, not enough frames, Praat viewport issues
Solutions: Use longer audio, reduce step_size for more frames, check Praat version
```