MSE Feature-Constrained Variation — User Guide

Feature-constrained audio variation using a normalized bag-of-frames distance. Each iteration generates a new transformed candidate, measures its mean-and-variance feature distance from the source, and adjusts transformation intensity toward a user-defined target band.

Author: Shai Cohen Version: 1.6 (2026) Technique: Feature Extraction + Stochastic Search Category: Composition

Contents:

What this does Quick start Feature Distance Preset Strategies Parameters & Controls Transformation Types Visualization Applications

What this does

This script generates controlled audio variations by comparing summary statistics extracted from the source and from newly transformed candidates. The source is analyzed as a mono, peak-normalized signal. Nineteen acoustic feature dimensions are sampled across time and summarized by their mean and variance, producing up to 38 statistics for the distance calculation.

Each iteration starts again from the same normalized source, applies a fresh stochastic transformation realization at the current intensity, measures its normalized feature distance, and then adjusts the intensity upward or downward. The first candidate inside the requested target band is accepted. If no candidate enters the band, the candidate whose distance is closest to the target is returned.

Key Features:

19 acoustic feature dimensions — 12 MFCCs, harmonicity, four band-energy measures, a four-band energy centroid, and frame-to-frame total-band-energy change.
Bag-of-frames representation — each dimension is summarized by mean and variance across the analysis frames.
Normalized distance — mean and variance differences are scaled by statistics of the original sound before squaring and averaging.
Dead-dimension exclusion — dimensions with near-zero source mean and variance are excluded from the metric for numerical stability.
Stochastic candidate search — every iteration redraws the enabled transformations.
Intensity feedback — too-similar candidates raise intensity; too-different candidates reduce it.
Maximum_intensity — explicit upper bound for the search controller.
Five transform families — granular restructuring, spectral mutation, intensity-contrast shaping, noise bursts, and pitch jitter.
Duration-preserving processing — the transformation pipeline is designed to retain the source duration.
Reproducible stochastic runs — positive Random_seed values reproduce the search sequence.

Metric scope: the distance measures changes in bag-of-frames acoustic statistics. It is not a calibrated perceptual distance and does not model frame order, rhythm, musical syntax, or listener discrimination thresholds.

Quick start

Select exactly one Sound object in Praat.
Run MSE_Feature_Constrained_Variation.praat.
Choose a preset. Standard is the default.
For Custom, set Target_distance, Tolerance, Max_iterations, Initial_intensity, Intensity_step, Maximum_intensity, N_analysis_frames, N_mut_segments, Noise_level, Tilt_scale, and Seg_emph_scale.
Set Random_seed if you need reproducible stochastic output.
Enable Show_visualization to inspect the source/output waveforms, spectrograms, stochastic search trace, feature changes, and summary.
Click OK. The result is named <source>_experimental.

Reading the target: target values are meaningful only within this feature-distance definition and its current normalization. They should not be interpreted as universal “subtle/moderate/extreme” perceptual scales.

Channel behavior: multichannel input is converted to mono for the complete transformation and output path. The delivered result is mono.

Feature Distance

What is MSE?

MSE (Mean Squared Error) is a measure of average squared difference between corresponding values. In this tool, the compared values are not audio samples themselves. Instead, the source and each transformed candidate are represented by summary statistics derived from acoustic features. The differences between corresponding feature statistics are normalized, squared, and averaged to produce a single distance value.

A distance of 0 means that the included feature statistics are identical. Larger values indicate greater displacement in the measured feature space. Because the metric summarizes feature distributions rather than waveform samples or listener responses, it should be interpreted as an acoustic feature distance, not as a direct measure of perceptual similarity.

19-Dimensional Feature Layout

Index	Feature	Implementation
1–12	MFCC coefficients	12 coefficients from a MelSpectrogram/MFCC analysis
13	Harmonicity	Praat Harmonicity (cc)
14	Band 1 energy	0–500 Hz, subject to Nyquist
15	Band 2 energy	500–2000 Hz, subject to Nyquist
16	Band 3 energy	2000–5000 Hz, subject to Nyquist
17	Band 4 energy	upper band from the remaining spectrum to Nyquist
18	Four-band energy centroid	energy-weighted average of the four band-center frequencies
19	Total-band-energy change	absolute frame-to-frame change in the sum of the four band energies

The MFCC count is fixed at 12. It is not a user control because the 19-slot feature layout reserves positions 13–19 for the remaining descriptors.

Bag-of-Frames Summary

For each feature dimension d: mean_d = average over N_analysis_frames variance_d = average(x²) - mean_d² The sound is therefore represented by: 19 means + 19 variances = up to 38 statistics

The individual frame values are used only to compute these per-dimension summaries. Temporal frame order is discarded by the metric.

Dead Dimensions

A source dimension is excluded if: variance < 1e-10 AND abs(mean) < 1e-6

This is a numerical-stability rule, not a statement that the feature is inherently uninformative. A dimension excluded because it is effectively zero in the source will not contribute to the distance even if a transformation later changes it strongly.

Normalized Mean-and-Variance Distance

For each active feature dimension d: varianceFloor = (0.02 × abs(originalMean_d))² + 1e-12 normalized mean difference = (originalMean_d - candidateMean_d) / sqrt(originalVariance_d + varianceFloor) normalized variance difference = (originalVariance_d - candidateVariance_d) / (originalVariance_d + varianceFloor) Each normalized term is clipped to: -10 ... +10 distance = mean of all squared normalized terms across active dimensions and both statistics

The resulting value is scale-normalized across heterogeneous features. A value near 1 can be read only loosely as an average one-standardized-unit displacement across the included statistics; it is not a perceptual unit.

Gain Normalization

analysis peak = 0.95 source: convert to mono scale peak to 0.95 extract original statistics every candidate: normalize to the same peak extract candidate statistics measure distance final output: delivered at the same measured gain

This ensures that the distance is measured on the exact gain-normalized candidate that can become the final output. There is no additional final peak change after selection.

Stochastic Search Controller

At each iteration: 1. Start again from the normalized source. 2. Draw a fresh realization of every enabled transform. 3. Measure normalized feature distance. If distance < target - tolerance: intensity += Intensity_step If distance > target + tolerance: intensity -= Intensity_step / 2 with a minimum of 0.05 If target - tolerance <= distance <= target + tolerance: accept and stop Intensity cannot exceed Maximum_intensity.

Because each iteration redraws the stochastic transformations, the distance trace need not move monotonically with intensity. The controller changes expected transformation strength, not a deterministic state variable.

Best Result

If a candidate enters the target band: return that candidate Otherwise: return the candidate minimizing abs(candidate_distance - Target_distance)

“Best” therefore means closest to the requested target, not minimum distance from the source.

Preset Strategies

Preset	Target	Tolerance	Initial	Step	Iterations	Main transform emphasis
Subtle	0.40	0.12	0.4	0.10	8	Spectral mutation + mild intensity contrast
Standard	1.00	0.25	0.8	0.20	15	Balanced multi-transform search
Granular	1.50	0.35	1.0	0.25	15	Granular reordering/reversal/gating
Spectral	1.80	0.30	0.9	0.20	15	Spectral mutation + noise + pitch
Extreme	2.50	0.50	1.5	0.30	20	All transforms at high multipliers

Preset Details

Subtle

Spectral mutation and mild intensity-contrast processing only. Granular, noise, and pitch-jitter stages are disabled.

Standard

All transform families enabled with restrained granular, noise, and pitch multipliers.

Granular

Heavy granular restructuring. Spectral mutation and intensity-contrast processing are disabled; noise and pitch remain at low multipliers.

Spectral

No granular restructuring. Spectral mutation, noise, intensity contrast, and pitch jitter operate at stronger settings.

Extreme

All transform families enabled with the largest built-in multipliers and the widest target band.

Parameters & Controls

Parameter	Default	Behavior
Preset	Standard	Subtle / Standard / Granular / Spectral / Extreme / Custom
Target_distance	1.0	Target normalized feature distance; used directly in Custom
Tolerance	0.25	Half-width of the accepted distance band
Max_iterations	15	Maximum number of stochastic candidates
Initial_intensity	0.8	Starting transformation strength
Intensity_step	0.2	Increase step; decreases use half this amount
Maximum_intensity	3.0	Upper bound for the controller; never allowed below Initial_intensity
N_analysis_frames	50	Number of evenly spaced bag-of-frames samples
N_mut_segments	6	Number of segments used by spectral mutation
Noise_level	0.02	Base amplitude for noise bursts
Tilt_scale	2.0	Base spectral-tilt scaling
Seg_emph_scale	1.5	Amplitude of sinusoidal spectral emphasis
Random_seed	0	0 = unpredictable; positive integer = reproducible
Show_visualization	On	Draw v1.6 diagnostic page
Play_result	On	Play selected candidate

MFCC count: fixed internally at 12. It is not a user parameter in v1.6.

Preset interaction: built-in presets override most analysis and transformation parameters. Maximum_intensity remains available as the controller ceiling and can therefore constrain a preset search if set below what the search would otherwise require.

Transformation Types

T1: Granular Restructuring

The source is divided into short grains whose duration decreases as granular intensity rises, down to a preset-dependent floor.

grain duration = max(grainSizeFloor, grainSizeBase - granularIntensity × grainSizeShrink) fade duration = max(2 ms, 8% of grain duration)

Swap: the swap scale determines how many random pairwise exchanges are performed.
Reverse: each output grain can be reversed with a clamped probability.
Gate: each output grain can be silenced with a clamped probability.

The tail remainder is retained rather than silently discarded. Grains are reconstructed end-to-end rather than with overlap-add.

T2: Spectral Mutation

The sound is divided into N_mut_segments. Each segment is transformed in the frequency domain using a randomized spectral tilt and sinusoidal spectral emphasis.

spectral multiplier = exp(tilt × normalized_frequency) × (1 + mutationAmplitude × sin(...))

The current script allows the sinusoidal multiplier to become negative when mutation amplitude exceeds 1. This introduces phase inversion in affected spectral bins and is part of the built-in Spectral and Extreme behavior.

T3: Intensity Contrast Shaping

This stage does not detect transients explicitly. It compares local intensity with the mean intensity and expands or suppresses local level contrast.

factor = 1 + exaggeration × (localIntensity - meanIntensity) / 40 factor is bounded approximately to: 0.1 ... 3

T4: Band-Limited Noise Bursts

Generate multiple 40 ms Gaussian-noise bursts.
Scale burst level from current intensity × Noise_level × transform multiplier.
Band-limit each burst around a randomized center frequency and bandwidth.
Apply short fades and add the burst at a random position.

Frequency limits are adapted to the source Nyquist frequency in the normal path.

T5: Pitch Jitter

Praat Manipulation/PitchTier processing perturbs voiced pitch points by a randomized proportion of their current F0, then resynthesizes without intentionally changing duration.

If the PitchTier contains no voiced points, the pitch-jitter stage is skipped.

Visualization

The v1.6 visualization separates signal comparison, stochastic search behavior, raw feature changes, and run summary.

HEADER source | preset | target +/- tolerance | final best distance ORIGINAL WAVEFORM source waveform EXPERIMENTAL WAVEFORM selected output duration and peak ORIGINAL SPECTROGRAM mono visualization up to min(5000 Hz, 0.49 × sample rate) EXPERIMENTAL SPECTROGRAM same display range STOCHASTIC SEARCH TRACE x = iteration y = normalized feature distance green band = accepted target range dashed line = target center red line/points = measured candidate sequence PER-DIMENSION MEAN DIFFERENCE absolute difference between source/output means shown in RAW FEATURE UNITS FEATURE LEGEND 1-12 MFCC 13 Harmonicity 14-17 Bands 18 Centroid proxy 19 Energy-change proxy SUMMARY source duration / sample rate / metric excluded dimensions target / best distance / search status intensity / seed centroid / harmonicity / flux-proxy changes mono output

Search trace: the final red point is the last evaluated iteration, not necessarily the candidate that was ultimately returned. When the search does not enter the target band, the selected output may come from an earlier iteration whose distance was closer to the target.

Per-dimension bars: these use raw feature units. Bar height is therefore not a normalized contribution to the distance, and heights should not be compared directly across unlike feature types.

Applications

Controlled Variation Sets

Generate several transformed versions of one source using different target distances or seeds, while keeping the same feature metric and transform family.

Transformation-Space Exploration

Compare how granular, spectral, pitch, noise, and level-contrast processes move the same source through the normalized bag-of-frames feature space.

Composition and Sound Design

Use the built-in presets as different transformation regimes, or use Custom to control the target band and stochastic search settings directly.

Feature-Distance Experiments

The tool can support experiments that compare an objective feature-distance measure with listening judgments, provided those judgments are collected separately. The built-in distance itself should not be treated as a perceptual threshold or JND scale.

Troubleshooting

Search does not reach the target band:
The enabled transformations may not produce candidates in that region, the intensity ceiling may be too low, or stochastic realizations may miss the region within Max_iterations. The closest candidate is returned.

Distance trace moves irregularly:
Expected. Every iteration redraws the stochastic transformations, so changing intensity does not produce a deterministic monotonic distance sequence.

Controller reaches Maximum_intensity:
Further increases are blocked. The Info report marks the intensity as pinned.

Granular result changes strongly but distance changes little:
The metric summarizes frame distributions and discards temporal order. Reordering can therefore be musically substantial while producing only a modest change in the bag-of-frames statistics.

Dead dimensions are excluded:
This stabilizes normalization but creates a deliberate blind spot for features that are effectively zero in the source and become active only after transformation.