Hierarchical Neural Recomposition — v1.0 User Guide

Multi‑scale neural recomposition engine. Segments audio into events → phrases → sections, builds hierarchical embeddings via PyTorch, and generates a recomposition plan guided by compositional parameters. Eight presets implement formal archetypes (Counterpoint, Spiral, Litany, Braiding, Refrain, Echo, Choir, Sonata).

Author: Shai Cohen Affiliation: Department of Music, Bar‑Ilan University, Israel Version: 1.0 (2025) License: MIT License Repo: GitHub

Contents:

What it does Quick start Presets (8 archetypes) Pipeline (6 stages) Hierarchical model Parameters The 16‑dim plan vector Visualization FAQ / troubleshooting

What this does

Hierarchical Neural Recomposition learns the latent structure of a sound at three timescales: events (grains), phrases (groups of events), and sections (groups of phrases). A PyTorch model encodes each level and produces a recomposition plan – a 16‑dimensional control vector per phrase that determines repetition, fragmentation, overlap, memory, contrast, inversion, braiding, and more. The engine then renders a new audio stream from the original source material, following the plan.

Why hierarchical? Musical form operates at multiple timescales. The event level handles micro‑articulation; the phrase level shapes gestures; the section level governs large‑scale contrast and memory. By learning embeddings for each level, the model can generate plans that respect the source’s inner grammar while imposing a new formal logic. Eight presets implement distinct compositional archetypes (Latent Counterpoint, Memory Spiral, Fragmented Litany, …). If PyTorch is unavailable, a numpy fallback model provides a reasonable approximation.

Quick start

In Praat, select exactly one Sound object (mono or stereo).
Run script… → HierarchicalRecomposition.praat.
Choose a Preset (Latent Counterpoint, Memory Spiral, … or Custom).
Set Target_duration (relative to input, e.g. 1.2 = 20% longer).
Adjust high‑level parameters: Density, Phrase_coherence, Section_contrast, Memory_strength, Repetition, Fragmentation.
Fine‑tune rendering: Overlap_amount, Source_trace (now deprecated, kept for compatibility), Formal_surprise (randomness in phrase ordering).
Click OK. Praat exports WAV, Python runs the full pipeline (segmentation, feature extraction, phrase grouping, neural planning, rendering). Result imported as originalname_hnr_preset.

Quick tip: Start with Latent Counterpoint to hear two phrase classes braided. For a slowly evolving, recursive texture, try Memory Spiral. Fragmented Litany creates ritualistic micro‑event repetitions. The visualisation shows event/phrase counts and intensity curves.

Important: Python dependencies: torch, numpy, scipy, soundfile. The model runs in evaluation mode – no training is performed on your audio. If PyTorch is not installed, a pure‑numpy fallback is used (slightly less structured). The Source_trace parameter is now disabled (it caused tremolo); it remains in the form for compatibility.

The eight presets (compositional archetypes)

Preset	Description	Key ops
LatentCounterpoint	Two or more phrase classes are extracted and braided into a polyphonic texture. Each phrase forms an independent voice defined by its timbral identity. Voices interweave, creating multi‑strand counterpoint from a monophonic source.	`braid=True, polyphony=2`
MemorySpiral	Material accumulates in spiralling layers of recall. Earlier events return at increasing distances, folded into later phrases. The form curves back on itself while continuously elongating — a spiral, not a loop.	`memory_depth=4, recurrence=True`
FragmentedLitany	Short micro‑events are extracted and arranged into a slow, incantatory repetition. Each litany cycle slightly mutates the ordering or duration of its fragments. The form accumulates ritual weight through obsessive variation.	`litany_cycles=3, micro_events=True`
FormalBraiding	Three phrase classes — attack‑heavy, sustained, and noisy — are braided into a single continuous strand. The braid periodically unravels and re‑forms with different phase relationships, producing a dynamic formal weave.	`braid=True, n_strands=3`
CollapsingRefrain	A strong opening phrase acts as a refrain. It returns three times, each time more fragmented and harmonically eroded. The final statement collapses into isolated residual events. Form is built entirely by progressive dissolution.	`refrain=True, collapse_rate=0.4`
EchoArchitecture	Each event generates one or more echoes — slightly delayed and reduced in amplitude. Phrases are constructed by the layering of events with their echo trails. The result is a space‑like architecture built from the source’s own resonance.	`echo_depth=3, echo_decay=0.55`
RecursiveSpeechChoir	The source is treated as a single utterance. Events corresponding to syllable‑like onsets are extracted and redistributed into a multi‑voice “choir” — the same material sung back to itself from multiple temporal positions simultaneously.	`choir_voices=4, phase_offset=True`
HiddenSonata	A classical three‑part sonata form is projected onto the event/phrase structure: an exposition introduces two contrasting phrase groups, a development section fragments and combines them, and a recapitulation restates the opening material, transformed. The hidden sonata is latent inside the source.	`sonata_form=True, recapitulation=True`

Pipeline — six stages

Stage 1 – Segmentation (onset strength + peak picking → events)
Stage 2 – Feature extraction (duration, RMS, centroid, flatness, harmonicity, onset sharpness → 7‑dim vector)
Stage 3 – Phrase grouping (greedy clustering based on similarity + coherence parameter)
Stage 4 – Section descriptors (density, brightness, harmonicity, flatness per temporal window)
Stage 5 – Hierarchical neural model (EventEncoder → PhraseEncoder → SectionPlanner → 16‑dim plan vector per phrase)
Stage 6 – Recomposition & rendering (event ordering with repetition, fragmentation, memory recalls, braiding, echoes, inversion; overlap‑add crossfade; soft‑limiter)

Feature vector (7 dims)

duration (normalised to 0‑1 by 4 s max)
RMS (clipped ×4)
centroid (normalised 0‑1)
flatness (0‑1)
harmonicity (0‑1, via autocorrelation)
onset_sharpness (ratio of first 64 samples RMS to full RMS)
log_rms (clipped and scaled)

Hierarchical neural model (PyTorch)

EventEncoder

7 → 32 (LayerNorm, GELU) → 64 (LayerNorm, GELU) → 32 (phrase embedding dimension)

PhraseEncoder

Positional encoding + 2‑layer transformer encoder (4 heads) → mean pool → linear to 64 (section embedding)

SectionPlanner

Bidirectional GRU (2 layers, hidden 64) → linear → GELU → linear → sigmoid → 16‑dim plan vector per phrase

If PyTorch is not installed, a numpy fallback uses random projections and simple softmax attention – still produces usable recompositions.

Parameters & defaults

Target

Parameter	Default	Description
Target_duration	1.0	Output duration relative to input (0.5‑5.0).

Event / Phrase level

Parameter	Default	Description
Density	0.5	Influences min event duration (higher = shorter events).
Phrase_coherence	0.5	0 = large heterogeneous phrases, 1 = tight homogeneous phrases.
Fragmentation	0.3	Tendency to split events into micro‑fragments.

Section level

Parameter	Default	Description
Section_contrast	0.5	Encourages dissimilar phrases to be juxtaposed.
Memory_strength	0.5	Probability of recalling earlier phrases.
Repetition	0.5	Tendency to repeat phrases/events.

Rendering

Parameter	Default	Description
Overlap_amount	0.3	Crossfade length between overlapping events.
Source_trace	0.85	Deprecated – kept for compatibility. No effect.
Formal_surprise	0.2	Random swaps in phrase order (0–1).

The 16‑dimensional plan vector (slots)

Each phrase receives a 16‑dim vector from the SectionPlanner (values in [0,1] after sigmoid). The slots are interpreted as follows:

Index	Slot name	Interpretation
0	repetition	tendency to repeat this phrase
1	fragmentation	break phrase into sub‑events
2	overlap	overlap with next phrase
3	stretch	time‑stretch amount
4	foreground	prominence / loudness weight
5	memory	connect back to earlier material
6	contrast	seek dissimilar phrase for juxtaposition
7	inversion	time‑reverse the phrase
8	density_up	increase local event density
9	density_down	decrease local event density
10	braid	interleave with another phrase
11	call_response	set up a call (odd) / response (even) pair
12	collapse	sudden density drop → silence
13	restatement	verbatim recall of an earlier phrase
14	surprise	random structural disruption
15	formal_weight	weight in final formal arc

These raw values are modulated by user parameters (e.g. plan[i, memory] * (0.3 + 0.7*memory_strength)) before being used in the event ordering and rendering stages.

Visualization (Praat picture)

When Draw_visualization = 1, the script draws a multi‑panel plot:

Original and recomposed waveforms – for comparison.
Original and recomposed spectrograms (0–5 kHz).
Intensity comparison – grey = original, blue = recomposed (dB scale).
Summary panel with:
- Preset name, event/phrase/section counts.
- Target duration ratio, input/output durations.
- All parameter settings (coherence, memory, contrast, repetition, fragmentation, overlap, surprise).
- RMS values and PyTorch availability.

Tip: The intensity overlay shows how the recomposition’s dynamic profile differs from the source. A large divergence suggests the plan has introduced significant structural reshaping.

FAQ / troubleshooting

“Python not found” or “torch not installed”

Install: pip install torch numpy scipy soundfile. The script falls back to a numpy model if PyTorch is missing.

Output is silent / very quiet

Check the RMS values in the Info window. If the output RMS is near zero, the plan may have placed very few events. Increase Density, Repetition, or reduce Fragmentation.

Source_trace parameter – why is it deprecated?

Originally it mixed the original (time‑stretched) audio back into the output. This caused comb filtering and amplitude beating (tremolo) because the stretched signal correlated with the summed events at unpredictable phases. It has been removed. The parameter remains in the Praat form for backward compatibility; it does nothing.

Recomposition sounds like random grains, no structure

The numpy fallback model can be less structured than the full PyTorch version. Install PyTorch for the full hierarchical embeddings. Also try presets with high Phrase_coherence and Memory_strength.

Choir mode (RecursiveSpeechChoir)

This preset generates multiple voices (offset copies) of the entire plan. The output is the sum of several slightly varied recompositions, creating a “choir” effect. Voice count is set by ops['choir_voices'] (4 in the preset).

Command‑line usage of hierarchical_recomposition.py

The Python engine can be run independently (batch processing):

python hierarchical_recomposition.py input.wav output.wav stats.txt target_dur density coherence contrast memory repetition fragmentation overlap source_trace surprise seed preset_name

All arguments are positional. preset_name can be “Custom” or one of the preset strings.