NMF Spectral Resynthesizer — User Guide

Dual‑mode NMF + pitch‑locked resynthesis for spectral transformation, texture design, and sound morphing.

Author: Shai Cohen Affiliation: Department of Music, Bar‑Ilan University, Israel Version: 1.0 (2025) License: MIT License Repo: https://github.com/ShaiCohen-ops/Praat-plugin_AudioTools

Contents:

What this does Quick start NMF Theory Presets & Modes Parameters Applications

What this does

This script implements Non‑negative Matrix Factorization (NMF) with dual‑mode smoothing and pitch‑locked resynthesis. It decomposes a sound’s spectrogram into spectral bases (W) and temporal activations (H), then reconstructs with perceptual smoothing and original pitch contour.

Key Features:

Dual‑Mode Smoothing — Transient‑preserving + texture‑blurring
Pitch‑Locked Resynthesis — Preserves original pitch contour
4 Built‑in Presets — Manual, Smooth Gliss, Clicks, Texture
Full Manual Control — Adjust window, step, decay, blur, NMF components
Diagnostic Output — Real‑time parameter feedback
Efficient NMF Loop — Multiplicative update with configurable iterations

Why NMF for sound? NMF factorizes a spectrogram into interpretable parts: spectral templates (W) and their time‑varying gains (H). This allows for:
(1) Structured sound decomposition — separate noise, harmonics, transients.
(2) Controllable resynthesis — modify H for smoothing, W for spectral shaping.
(3) Pitch preservation — re‑impose original pitch contour for naturalness.
(4) Creative sound design — glissandi, clicks, micro‑textures, morphing.

Technical Pipeline:
1. Spectrogram — Gaussian window, configurable length/step.
2. NMF Decomposition — W (spectral bases) × H (activations) ≈ V (spectrogram).
3. Dual‑Mode Smoothing — Transient decay (first 2 components) + texture blur (remaining).
4. Reconstruction — W × smoothed H → modified spectrogram.
5. Pitch Locking — Extract original pitch contour, impose on resynthesized sound.
6. Output — “originalname_NMF_preset” with preserved pitch, transformed texture.

Quick start

In Praat, select exactly one Sound object.
Run script… → nmf_spectral_resynthesizer.praat.
Choose preset: Manual, Smooth Gliss, Clicks, or Texture.
If Manual, adjust window, step, decay, blur, pitch range, NMF parameters.
Click OK — processing runs with diagnostic output.
Result named “originalname_NMF_preset” appears in Objects window.

Quick tip: Start with Smooth Gliss for flowing, pitch‑gliding textures. Use Clicks for stuttered, granular effects. Try Texture for micro‑detail blurring. Enable diagnostic output to see parameter values. Processing time depends on NMF iterations and window length.

Important: Pitch tracking assumes voiced sound. If input is noisy/unpitched, pitch‑locking may produce artifacts. Adjust min_pitch and max_pitch to match source. Large window_ms values blur transients; small values increase time‑frequency trade‑off. NMF components control complexity — too few = over‑smooth, too many = noisy.

NMF Theory

Non‑negative Matrix Factorization

V ≈ W × H Where: V = spectrogram (frequency × time, non‑negative) W = spectral bases (frequency × components) H = temporal activations (components × time) Goal: Find W, H ≥ 0 that minimize reconstruction error. Multiplicative update rule (used in script): H ← H * (Wᵀ V) / (Wᵀ W H + ε) W ← W * (V Hᵀ) / (W H Hᵀ + ε) Iterate until convergence (n_iterations).

Dual‑Mode Smoothing

Mode 1 — Transient Decay (first 2 components):

H[row≤2, col] = (1‑decay) * H[row, col] + decay * H[row, col‑1]

Preserves sharp onsets while smoothing continuations.

Mode 2 — Texture Blur (remaining components):

H[row>2, col] = 0.25*H[row, col‑1] + 0.5*H[row, col] + 0.25*H[row, col+1]

Applies multiple passes to blur micro‑temporal details.

Pitch‑Locked Resynthesis

Steps:

Extract original pitch contour (Praat’s Pitch object).
Smooth with pitch_smoothing_hz.
Convert to PitchTier.
Apply to resynthesized sound via Manipulation object.
Overlap‑add resynthesis preserves timing, imposes original pitch.

🧠 Engine Overview

Dual‑Mode NMF: Separates transient and texture components for independent control.

Pitch‑Locking: Keeps melodic/harmonic identity while transforming spectral texture.

Preset Logic: Pre‑configured parameter sets for specific sonic effects.

Presets & Modes

Preset 1: Manual

⚙️ Full Control

Use when: You know exactly what you want.

Parameters: All settings available in form.

Character: Totally configurable.

Preset 2: Smooth Gliss

🎵 Flowing, Evolving Texture

Window: 2.0 ms (narrow, detailed spectrogram)

Step: 50.0 ms (large, creates time‑stretched glides)

Decay: 0.9 (strong smoothing, blurred transients)

Blur: 1 pass (light texture smoothing)

Result: Smooth, pitch‑gliding, ethereal textures.

Preset 3: Clicks

🔊 Stuttered, Granular

Window: 60.0 ms (wide, blurs harmonics)

Step: 15.0 ms (short, creates repetition)

Decay: 0.4 (weak smoothing, preserves transients)

Blur: 5 passes (heavy temporal blurring)

Result: Click‑like, granular, stuttered effects.

Preset 4: Texture

🌫️ Micro‑Detail Blur

Window: 1.0 ms (very narrow, extreme TF resolution)

Step: 2.0 ms (small, preserves timing)

Decay: 0.5 (moderate transient smoothing)

Blur: 2 passes (moderate texture blur)

Result: Micro‑textural blurring, granular smearing, spectral diffusion.

Parameters

Form Parameters

Parameter	Type	Default	Description
preset	option	Manual	Preset selection
window_ms	positive	10	Analysis window length (ms)
step_ms	positive	4.0	Analysis time step (ms)
trans_decay	positive	0.6	Transient smoothing strength (0–1)
texture_blur_passes	integer	4	Texture blur iterations
min_pitch	positive	75	Minimum pitch (Hz) for tracking
max_pitch	positive	600	Maximum pitch (Hz) for tracking
pitch_smoothing_hz	positive	10	Pitch contour smoothing (Hz)
max_freq_hz	natural	4000	Maximum frequency in analysis
n_components	integer	4	Number of NMF components
n_iterations	integer	8	NMF update iterations

Parameter Details

window_ms & step_ms:
Control time‑frequency resolution. Small window = good time resolution (preserves transients), poor frequency resolution. Large window = good frequency resolution (clear harmonics), poor time resolution. Step controls overlap; smaller step = smoother temporal evolution.

trans_decay:
How much temporal smoothing to apply to the first 2 NMF components (assumed to carry transient energy). 0 = no smoothing, 1 = complete smoothing (transients become sustains).

texture_blur_passes:
Number of temporal blurring passes applied to remaining NMF components (assumed to carry texture/noise). More passes = more smearing, less micro‑detail.

Pitch parameters (min_pitch, max_pitch, pitch_smoothing_hz):
Define the pitch range for contour extraction. Smoothing reduces jitter; higher values = smoother but potentially laggy pitch tracking.

NMF parameters (n_components, n_iterations):
Components = number of spectral bases. More components = more detailed decomposition but slower. Iterations = NMF convergence steps; 8–20 typical for audio.

Applications

Spectral Morphing

Use case: Transform one sound’s texture onto another’s pitch contour.

Technique: Apply NMF to source A, reconstruct with source B’s pitch contour.

Example: Speech texture + singing pitch = whispered singing.

Texture Design

Use case: Create evolving, granular, or blurred textures.

Technique: Use "Texture" preset with high blur passes.

Example: Piano note → cloud‑like sustained pad.

Transient Design

Use case: Shape attack/decay characteristics independently.

Technique: Adjust trans_decay to smooth or sharpen onsets.

Example: Percussive hit → swelling bow‑like attack.

Pitch‑Preserving Sound Design

Use case: Alter spectral content while keeping melody intact.

Technique: NMF decomposition + pitch‑locked resynthesis.

Example: Vocal → robotic or whispered version, same pitch.

Creative Preset Workflows

🎹 Glissando Pad

Goal: Create smooth, evolving pad from monophonic instrument.

Settings:

Preset: Smooth Gliss
Input: Piano or voice
Adjust decay to taste

Result: Washed‑out, pitch‑gliding texture suitable for ambient music.

🥁 Granular Percussion

Goal: Turn sustained sounds into granular, click‑based rhythms.

Settings:

Preset: Clicks
Input: String sustain or noise burst
Increase blur passes for more smearing

Result: Stuttered, rhythmic texture with preserved pitch contour.

🗣️ Whispered Speech

Goal: Remove vocal cord vibration while keeping articulation.

Settings:

Preset: Manual
window_ms: 5.0, step_ms: 2.0
trans_decay: 0.7, blur_passes: 3
n_components: 6, n_iterations: 12

Result: Breath‑only version of speech with intact pitch contour.

Troubleshooting

Problem: Output sounds choppy or artifact‑laden
Cause: Window too small, step too large, or NMF components too few.
Solution: Increase window_ms, decrease step_ms, or add more n_components.

Problem: Pitch contour sounds wrong or wobbly
Cause: Pitch tracking range mismatched or smoothing too high.
Solution: Adjust min_pitch/max_pitch to match source, reduce pitch_smoothing_hz.

Problem: Processing very slow
Cause: Too many NMF components/iterations, or very small step.
Solution: Reduce n_components, n_iterations, or increase step_ms.

Problem: Output too similar to input
Cause: Smoothing too weak, or NMF not converging.
Solution: Increase trans_decay, blur_passes, or n_iterations.

Advanced Techniques

Custom NMF Initialization

Modify the script to initialize W and H with custom matrices (e.g., from another sound) for cross‑sound morphing.

Component‑Selective Processing

Process only specific NMF components (e.g., only component 1 for transient design, components 3‑4 for texture).

Real‑time Parameter Modulation

Use Praat’s scripting to vary parameters over time (e.g., increasing blur_passes during a phrase).

Hybrid Presets

Combine preset logic: e.g., Clicks‑like window with Smooth‑like decay for novel textures.

Pro tip: Use the diagnostic output to track parameter values. The script prints window, step, decay, blur, and preset info before processing. This helps reproduce results and debug.