NMF Granular Spatial Composer
Composing new timbres and spatial gestures from one sound through NMF-derived materials, time-local granulation, and the source's own amplitude envelope.
The musical idea: decomposition becomes composition
A recorded sound contains more than one possible musical identity. Its spectral content, movement in time, and changing intensity can be separated and recombined to suggest new materials. NMF Granular Spatial Composer treats a selected sound not as something to reconstruct faithfully, but as a starting point for an offline compositional process.
Non-negative Matrix Factorization (NMF) provides a small collection of spectral patterns and their changing activations. The instrument uses that analysis to generate eight related but different audio materials. It then draws short grains from those materials, keeping each grain near the corresponding moment of the original recording. Finally, the original amplitude envelope shapes the new texture, while individual grains may be distributed independently in stereo or quadraphonic space.
The resulting music may preserve the dynamic contour of speech, an instrumental phrase, or a noisy gesture without retaining the original sound's recognizable timbre. NMF components are algorithmic spectral patterns, not necessarily separate physical sources or instruments.
From one Sound to a spatial composition
Select a single Sound in Praat. The Praat script manages the interface, temporary exchange, output objects, and Picture visualization; a companion Python engine carries out NMF, spectral transformations, grain scheduling, audio rendering, and spatial distribution. No corpus preparation is required.
Sound becomes a corpus
Three isolated NMF component signals are joined by five resynthesized transformations, including phase reconstruction, cross-routing, spectral sculpture, and a stationary portrait. The eight materials are generated from the same input.
Grains remain locally connected
Each grain is selected around its corresponding moment in the input, rather than from an unrestricted position in the whole corpus. That local relationship, together with the imposed amplitude envelope, helps retain the larger gesture even when the timbre changes.
The source envelope is applied after granular accumulation and spatial distribution. The instrument does not simply widen a finished mono mix: spatial positions are assigned to individual grains as part of the composition.
The eight-source NMF corpus
The reference analysis operates at 16 kHz with a 1024-sample STFT, 256-sample hop, and 12 NMF components. Three of these are selected by their contribution to the decomposition. The other five corpus slots are produced by distinct spectral rendering procedures. The corpus exists internally during processing; it is not a set of files the user must prepare.
| Slot | Corpus material | How it is obtained | Listening perspective |
|---|---|---|---|
| 1–3 | Component solos | Three high-energy NMF components extracted using soft spectral masks and the source's complex STFT. | Partial spectral identities and features obscured by the complete source. |
| 4 | Neutral Griffin-Lim | Reconstruction from the NMF magnitude approximation with estimated phase. | An alternative synthetic rendering of the analysis. |
| 5 | Cross-route / original phase | Temporal activations are reassigned across spectral bases; synthesis uses the source phase. | New relationships between spectral identity and temporal behavior. |
| 6 | Cross-route / Griffin-Lim | The same routed magnitude representation is synthesized using estimated phase. | A contrasting reconstruction of the same transformed spectrum. |
| 7 | Spectral Sculpture | Frequency profiles are stretched and activations are smoothed and shifted. | Greater timbral and temporal departure from the original. |
| 8 | Spectral Portrait | A leading spectral basis is held at constant activation and rendered with randomized phase. | A relatively stationary spectral color available for granular recomposition. |
How the granular cloud is formed
Grain onsets follow a Poisson process. Each event chooses one of the eight corpus slots, a duration, a playback speed, and a read position near the corresponding time in the original recording. Samples are interpolated, windowed with a Hann function, and accumulated with overlap normalization.
For the Reference preset, the mean density is 93 grains per second; durations range from 65 to 140 milliseconds; speeds range from 0.92 to 1.08 times the original; and the source-read position is displaced by at most approximately ±0.26 seconds. These are statistical choices rather than a metrical pattern.
The source envelope is derived from a combination of fast and slow RMS measurements. It controls the finished granular texture's dynamics, so the new spectral material may retain the rise, fall, and pauses of its parent recording.
Mono, stereo, and quadraphonic composition
Rendering determines the output channel layout and spatial behavior. It is independent of Preset, which determines how grains are scheduled and selected.
Mono Local Granular
All grains accumulate in one channel. The best starting point for evaluating the relationship between NMF material, local granulation, and the source envelope without spatial variation.
Stereo Source Islands
Each of the eight corpus sources occupies a characteristic position in the stereo field. Its grains drift gently around that location, creating spatially distinguishable spectral identities.
Stereo Grain Orbit
A shared center moves across the stereo field while each grain is scattered independently around it. The sound field becomes an evolving constellation rather than a fixed left-right mixture.
Quad Constellation
Grains occupy regions around the listener and move locally between adjacent speakers using equal-power panning. The four channels are Front Left, Front Right, Back Left, and Back Right.
Corpus slots tend toward their own stereo locations, with small per-grain variation.
Four independent channels around the listener; the speaker layout must be respected during playback.
Quad_stereo_preview enabled, it also imports a stereo downmix of Quad. The preview is not binaural and does not reproduce four loudspeakers over headphones.Getting started
NMF_Granular_Spatial_Composer.praat and nmf_granular_spatial_composer.py in the same folder inside Praat AudioTools. Install Python 3.10 or later and the required packages: numpy, scipy, librosa, scikit-learn, and soundfile. The script checks Python and its packages automatically. If automatic discovery fails, an optional nmf_gsc_python.txt file containing the Python executable path can be placed beside the scripts.- Select exactly one Sound in Praat. It may contain speech, instrumental music, noisy material, or a composite sound. The selected original is preserved.
- Run
NMF_Granular_Spatial_Composer.praat. LeavePreseton Reference for the validated starting configuration. - Choose a Rendering mode. Start with Stereo Source Islands for an immediately spatial result, or choose All Outputs to compare all formats from the same grain events.
- Keep Original envelope enabled. It carries the source's dynamic gesture into the granular texture. Try disabling it later to hear what that gesture contributes.
- Render, listen, and inspect the Picture. With
Draw_visualizationenabled, the tool draws the temporal gesture, source activity, and spatial organization. New Sound objects appear in Praat's Objects window. - Repeat with one change. Compare a different preset, spatial mode, or grain seed while keeping the source fixed. Choose Custom to specify individual granular settings.
Compositional presets
Presets determine the temporal and textural character of the granular cloud. They do not determine the number of output channels. All five preset options work with every Rendering mode.
| Preset | Density (grains/s) | Grain duration | Locality | Playback speed | Envelope |
|---|---|---|---|---|---|
| Reference | 93 | 65–140 ms | ±0.26 s | 0.92–1.08 | 100% |
| Suspended | 75 | 140–260 ms | ±0.35 s | 0.92–1.08 | 80% |
| Fragmented | 125 | 40–85 ms | ±0.12 s | 0.85–1.15 | 100% |
| Diffuse Cloud | 150 | 90–210 ms | ±0.55 s | 0.78–1.22 | 65% |
| Custom | Opens a second dialog for density, minimum/maximum duration, locality, speed range, envelope strength, and random seed. Starts from Reference values. | ||||
Reference: the musical baseline
The Reference preset preserves the settings used to establish the successful NMF + local granulation + original-envelope process. Use it to compare spatial renderings and to establish a familiar listening point before changing the grain organization.
Suspended
Longer grains and a slightly wider local search create a starting point for sustained, overlapping textures. Try it with a resonant instrument or an extended vocal sound.
Fragmented
Shorter and more numerous grains offer a starting point for fragmented gestures and rapid micro-events. Try it with consonants, attacks, or percussive material.
Diffuse Cloud
A denser cloud, a wider source search, and a broader playback-speed range invite a less literal relationship with the source's temporal order.
Custom
Use manual values to isolate a particular compositional variable. A fixed nonnegative random seed makes controlled comparisons possible.
Listening status: Reference reflects the previously approved experiment. Suspended, Fragmented, and Diffuse Cloud are proposed exploratory starting points. Their musical effects are intentions, not guaranteed results for every recording, and they have not been validated by the same reference listening experiment.
Controls and output options
| Control | What it does |
|---|---|
Preset | Reference, Suspended, Fragmented, Diffuse Cloud, or Custom. Controls the statistical organization of grains. |
Rendering | Mono Local Granular, Stereo Source Islands, Stereo Grain Orbit, Quad Constellation, or All Outputs. Default: All Outputs. |
Original_envelope | Enabled by default. When unchecked, envelope strength is set to zero regardless of the preset. |
Quad_stereo_preview | Imports a two-channel downmix when Quad is rendered; it is not a binaural simulation. |
Draw_visualization | Creates the compositional Picture using real render data and measured RMS curves. |
Show_processing_report | Reports input, preset, parameters, NMF diagnostics, corpus slot identities, grain counts, and imported outputs in Praat's Info window. |
Retain_work_files | Keeps a clearly marked, user-owned folder with generated files and diagnostic data instead of removing them after import. |
Play | Auto, a specific mono/stereo result, Quad Preview, or None. Auto prefers Stereo Islands, then Orbit, Quad Preview, and Mono among available outputs. |
The Custom dialog
When Custom is selected, a second dialog offers Density (1–500 grains/s), Grain_min_ms and Grain_max_ms (5–1000 ms), Locality_s (0–5 s), Speed_min and Speed_max (0.25–4.0), Envelope_strength (0–1), and Random_seed (nonnegative integer). The minimum values must not exceed the corresponding maximum values.
Reference and the three exploratory presets use the grain seed 20261029. Within one execution the same grain event list is reused for the requested spatial modes; this makes comparisons between modes more meaningful.
Result objects
| Praat Sound name suffix | Channels | Purpose |
|---|---|---|
_NMF_Local_Granular | 1 | Local granular composition with optional source-envelope shaping. |
_NMF_Stereo_Islands | 2 | Distinct corpus-source islands across the stereo field. |
_NMF_Stereo_Orbit | 2 | Granular cloud around a moving stereo center. |
_NMF_Quad | 4 | Four-channel constellation in FL, FR, BL, BR order. |
_NMF_Quad_Preview | 2 | Stereo downmix of the quad result, when requested. |
All results are returned as Praat Sound objects. Quad stems are also written by the engine as intermediate WAV files, but are not automatically imported as separate Praat Sounds. To preserve these files for routing in a DAW, enable Retain_work_files.
Reading the compositional visualization
When Draw_visualization is enabled, Praat draws a compositional score in its Picture window. Python supplies time-stamped grain events, the positions actually used for spatial rendering, and measured amplitude curves. The picture describes what was produced; it does not modify the synthesis.
Temporal Gesture
Displays the input RMS, the output RMS, and the imposed envelope. Each curve is separately scaled to its own maximum, so the panel compares shape, not absolute loudness.
Granular Source Map
Shows grain events across eight corpus lanes. Horizontal position marks onset time, line length represents grain duration, and the small vertical shift within each lane reflects the difference between source-read time and output time. Labels show the actual component identities and event counts.
Spatial Composition
For Stereo Islands or Orbit, displays the actual grain pan positions through time. Orbit adds the moving center; Islands shows source anchors. For Quad, four speaker-energy curves appear together with the speaker arrangement.
The information strip identifies the corpus size, grain-event count, preset, envelope setting, output choice, and random seed. This makes the visualization a useful record for comparing successive renders.
Compositional possibilities
The instrument becomes most informative when one variable is changed at a time. The following are listening experiments rather than guaranteed presets or recipes.
Speech without words
Choose a spoken phrase containing pauses, breaths, and contrasting consonants. Use Reference with Stereo Islands. Listen for the original utterance's dynamic shape emerging as an unfamiliar spectral texture.
The same gesture, different space
Keep Reference and the same source, then compare Islands with Orbit and Quad. Because the grain events are shared, differences can be heard primarily as changes in spatial organization.
Stretching an instrumental resonance
Try Suspended on a sustained instrumental sound, bowed resonance, or long decay. Listen for whether the longer grains expose stable timbral colors or smear recognizable attacks.
A cloud of micro-attacks
Try Fragmented with pitched percussion or short transient-rich phrases. Compare the rhythmic impression with the source's own envelope retained and switched off.
Preserve or dissolve the phrase
With Custom, keep the grain duration fixed while expanding the locality range. Listen for the point at which short-term source relationships become less apparent.
Composition as a family of renders
Keep one random seed, save several preset/space combinations, and arrange them later as contrasting materials. The offline nature of the tool encourages comparison, selection, and revision.
Technical and listening notes
- Offline hybrid system. Praat is the user interface and final Sound environment; Python performs the NMF, granular synthesis, and spatial audio computation. No real-time audio or external DAW is needed for rendering.
- Single Sound input. The engine converts the input to a mono analysis signal. It does not preserve the input's existing stereo image. The new spatial image is generated from granular events.
- Reference-rate output. The engine currently works at 16 kHz. A source recorded at a higher rate is resampled for analysis; the generated output is also 16 kHz. This limits the representable frequency range to 8 kHz.
- NMF is not source separation. An individual component is a computational spectral pattern rather than a guaranteed voice, note, or physical sound source. Component identities can change with the input.
- Reproducibility. A fixed seed, identical source, and consistent software versions make controlled comparisons possible. Changed granular parameters or library versions can alter the output.
- Temporary files. Normal processing uses a uniquely marked working folder outside the plugin directory. Imported results remain in Praat; the working files are removed on successful completion. The retained-work option protects a deliberately kept folder from automatic cleanup.
- Quad playback. The four-channel Sound uses
FL, FR, BL, BRchannel order. Verify the channel routing on the actual playback or DAW system. A stereo preview is only a downmix, not a simulation of quad loudspeaker listening. - Limits and performance. Inputs are accepted from 1 second to 10 minutes. NMF and multiple spatial renders can require substantial time and memory on long sources. The selected input is not modified.
- Interpret the visualization musically. A high number of grains or strong correlation between amplitude envelopes is not a measure of artistic quality. The display makes decisions and structures visible; listening remains decisive.
The instrument builds on Python reference experiments combining NMF decomposition, transformed spectral reconstructions, local granular synthesis, source-envelope shaping, and per-grain spatialization. The Reference preset preserves that established processing design; other presets remain exploratory configurations.