Internal Polyphony — v1.0 User Guide

Reveals hidden simultaneous voices inside a sound using NMF-based functional decomposition. Support, body, accent, halo, residue, shimmer – each with its own voice engine and compositional staging. Not a denoiser, not a latent gimmick: genuine chamber music already latent in the source.

Author: Shai Cohen Affiliation: Department of Music, Bar‑Ilan University, Israel Version: 1.0 (2025) License: MIT License Repo: GitHub

Contents:

What it does Quick start The six voices Pipeline (9 stages) Compositional modes Presets Parameters Metrics & visualisation

What this does

Internal Polyphony uncovers the latent voices that are already present, but masked, inside a single audio recording. It does not synthesise new material – it reveals what is acoustically implied.

The core idea: a sound is not a single line; it contains multiple simultaneous “functions” – a low drone (support), a midrange melodic core (body), short punctuations (accent), a sustained glow (halo), noise seepage (residue), and a rapid upper flicker (shimmer). NMF separates the spectrogram into components; acoustic descriptors (centroid, flatness, sparsity, onset affinity, etc.) assign each component to one of these six functional roles. Then, for each role, a specialised voice engine re‑assembles fragments into a continuous stream that behaves according to its role: support drones with slow breath, accent punctuates with enforced separation, halo uses smooth arc envelopes, etc.

Five compositional modes apply formal laws to the voice streams:

Reveal – each voice emerges from silence at a different point.
Counterpoint – staggered independent entries.
Canon – true canonic delay chain (support→body→halo→shimmer).
PedalHalo – pedal and halo foregrounded, body recedes.
FracturedChoir – all voices dense, independently modulated.

Important: This is not source separation in the usual sense. No instrument is isolated. Instead, the six functions are revealed, each behaving according to its musical role. If a function is truly absent (e.g. a sound with no low drone), its voice remains silent – you get a trio, not a forced sextet.

Quick start

In Praat, select exactly one Sound object.
Run script… → InternalPolyphony.praat.
Choose a Preset (Reveal / Counterpoint / Canon / PedalHalo / FracturedChoir) or adjust parameters manually.
Set Num_components (NMF rank, default 10).
Select Polyphony_mode (matches preset usually).
Adjust Voice_density, Dry_wet, Accent_prominence, Halo_amount, Stereo_width.
Enable Draw_visualization to see the role activity bars and summary panel.
Click OK – Python runs NMF, infers roles, builds voices, and renders stereo output.

Quick tip: Start with “Reveal” to hear each voice fade in one by one. For a dense, choral effect, try “FracturedChoir”. The visualisation shows coloured activity bars for each of the six roles – you immediately see which voices are active.

Important: Python dependencies: numpy, scipy, soundfile. NMF can be slow for long files – reduce Num_components (min 3) or increase Hop_size. The output is stereo, panned according to fixed pan positions (support centre, body left‑centre, accent right‑centre, etc.).

The six voices – functional roles

Role	Acoustic profile	Voice engine behaviour	Pan	Base gain
support	Low centroid (≤700 Hz), high sustain, low sparsity	Drone layer: longest fragments, heavy overlap, slow sine‑wave breath modulation	0.50 (C)	0.70
body	Midrange energy (300–2000 Hz), moderate sustain	Phrased core: natural gaps, moderate overlap, crossfade	0.42 (L‑C)	0.82
accent	High sparsity, high onset affinity, short fragments	Punctuation: no overlap, minimum separation enforced	0.58 (R‑C)	0.78
halo	High centroid (2.8 kHz+), high sustain, low onset	Glow: arc envelopes (rise‑sustain‑fall), enters after body	0.25 (L)	0.52
residue	High flatness, high roughness, noise‑like	Seep: irregular low‑amplitude scatter, background texture	0.72 (R)	0.42
shimmer	Very high centroid (>5.5 kHz), high recurrence	Flicker: rapid short fragments, tiny gaps, upper counterline	0.50 (C)	0.48

Each role’s voice engine is a separate algorithm (build_support, build_body, …) that implements these behaviours. Absence is permitted – if NMF finds no component matching a role, that voice remains silent.

Pipeline — nine stages

Stage 1 – Audio load (any channel count, mono conversion)
Stage 2 – STFT (FFT/hop, optional HPSS or transient pre‑emphasis)
Stage 3 – NMF (KL‑divergence, K = num_components)
Stage 4 – Component descriptors (14 acoustic features per component)
Stage 5 – Role inference (rule‑based, no ML – uses thresholds on centroid, flatness, sparsity, onset_aff, sustain, recurrence, etc.)
Stage 6 – Fragment harvesting (dominance threshold → intervals → extract fragments with fade in/out)
Stage 7 – Per‑role voice engines (diversity‑aware pool, overlap/add, specialised behaviours)
Stage 8 – Mode staging (reveal, counterpoint, canon, pedalhalo, fracturedchoir)
Stage 9 – Stereo rendering + metrics (pan, mid/side width, dry/wet mix, RMS‑match)

All stages are pure Python (numpy/scipy) – no external ML.

Compositional modes – formal logics

🎭 Reveal

Each role emerges from silence at a different time: support (0%), body (5%), halo (10%), shimmer (25%), accent (20%), residue (30%). Linear fade‑in with exponent 1.5. Accent and residue are additionally subdued. Listener hears one voice at a time unveiling itself.

🎼 Counterpoint

Staggered independent entry points: support 0s, accent 0.05, body 0.08, shimmer 0.15, halo 0.20, residue 0.35 (fraction of total duration). Voices are time‑shifted so they are never all simultaneous – true independence.

🔁 Canon

True canonic delay chain: support (dux) → body (12% later) → halo (24%) → shimmer (36%). Accent and residue free but delayed by 6% and 18% respectively. Later entries are slightly attenuated (decay proportional to delay).

🎹 PedalHalo

Support and halo are foregrounded (×1.35, ×(1+halo×0.7)). Body recedes (×0.55), accent very sparse (×0.2+0.2×prom), residue and shimmer background. Creates a pedal‑plus‑glow texture.

🧩 FracturedChoir

All voices at high density, each independently amplitude‑modulated (sine, rate 0.3–1.5 Hz random). No single voice foregrounded – dense overlapping choral texture.

Built‑in presets (6)

Preset	Mode	Density	Dry/wet	Accent	Halo	Width	Description
Reveal (subtle)	reveal	0.45	0.65	0.4	0.5	0.9	Gentle emergence
Counterpoint	counterpoint	0.70	0.90	0.8	0.55	1.3	Strong independence
Canon	canon	0.60	0.85	0.6	0.65	1.4	Staggered entries
PedalHalo	pedalhalo	0.55	0.80	0.3	0.9	1.5	Sustain + glow
FracturedChoir	fracturedchoir	0.90	0.95	0.75	0.5	1.6	Dense overlapping fragments

Other parameters (components, FFT, min_fragment, expansion) vary per preset – see Praat form.

Parameters & defaults

Decomposition

Parameter	Default	Description
Num_components	10	NMF rank (3–24). Higher = more potential roles
Fft_window	2048	STFT window size (samples)
Hop_size	512	STFT hop (samples)
Analysis_mode	full	full / hpss / transient – pre‑emphasises parts

Voice & staging

Parameter	Default	Description
Polyphony_mode	canon (3)	reveal / counterpoint / canon / pedalhalo / fracturedchoir
Voice_density	0.65	Global density multiplier (0–1)
Min_fragment_sec	0.15	Minimum fragment duration (seconds)
Max_overlap	0.75	Maximum allowed overlap (reserved)
Expansion_factor	1.0	Time expansion factor (0.5–4.0) – changes target duration

Mix / character

Parameter	Default	Description
Dry_wet	0.85	Dry/wet mix (0 = dry only, 1 = polyphony only)
Accent_prominence	0.7	Boosts accent volume and behaviour
Halo_amount	0.6	Boosts halo volume and envelope
Stereo_width	1.2	Mid/side width multiplier (0 = mono, >1 = widened)

Options

Flag	Default	Effect
Random_seed	42	Reproducibility
Draw_visualization	1	Praat picture with waveforms, spectrograms, role activity bars, summary
Play_result	1	Auto‑play after processing

Metrics & visualisation

Role activity bars

The visualisation (if enabled) shows a coloured bar for each role, with length proportional to the fraction of time that role is active. On the right you see:

Duration (seconds active)
Fragments (number of harvested fragments)
AvgF (s) – average fragment duration

Numerical metrics (from JSON report)

Metric	Description
effective_components	Number of components explaining 90% of energy
decomp_error	Relative NMF reconstruction error (lower = better fit)
novelty_ratio	RMS(output – dry) / RMS(dry) – distance from original
overlap_density	Fraction of frames with ≥2 voices active
voice_independence	Mean 1−\|correlation\| across all voice pairs (1 = fully independent)
stereo_spread	Mean absolute L‑R difference
roles_present	List of roles that produced fragments

Troubleshooting:

No output / all roles absent: NMF components may not match any role. Try increasing Num_components or lowering Min_fragment_sec. Use Draw_visualization to see which roles (if any) have fragments.
Python fails: ensure numpy, scipy, soundfile installed. Check internal_polyphony.py is in plugin_AudioTools/py/.
Output clicks / pops: increase Min_fragment_sec or reduce Voice_density.
NMF too slow: reduce Num_components or increase Hop_size (e.g. 1024).

Technical note: role inference rules

shimmer: centroid > 5500 Hz AND sustain > 0.15 AND recurrence > 0.08
halo: centroid > 2800 Hz AND sustain > 0.30 AND onset_aff < 0.12
accent: sparsity > 0.50 AND onset_aff > 0.08
residue: flatness > 0.28 AND roughness > 0.5
support: lo_e > 0.40 AND sustain > 0.35 AND sparsity < 0.55
body: mid_e > 0.25 AND sustain > 0.25
Fallbacks: centroid < 700 Hz → support; centroid < 2500 Hz → body; centroid < 5500 Hz → halo; else shimmer.

All thresholds are hard‑coded and hand‑tuned on a wide variety of sounds. No clustering, no machine learning.