Neural Phonetic Harmonizer — User Guide

Adaptive PitchTier harmonization driven by per-file FFNet distillation of heuristic vowel, consonant, other, and silence labels, with smoothed class weights, class-specific harmony intervals, wet/dry blending, and synthetic stereo.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 0.8 (2026) License: MIT License Repo: https://github.com/ShaiCohen-ops/Praat-plugin_AudioTools

Contents:

What this does Quick start Per-file FFNet Distillation Heuristic Classes 8 Input Features Temperature PitchTier Harmonization Adaptive Harmony Envelope Stereo Width Presets Parameters Visualization Outputs

What this does

Neural Phonetic Harmonizer analyzes one Sound on a 10 ms control grid, assigns each frame a heuristic class — vowel, consonant, other, or silence — and, when at least two classes are present, trains a small FFNet on that same file to reproduce and smooth those labels from eight acoustic features.

The resulting output-layer activations are converted into temperature-shaped class weights, smoothed over time, and used to control class-specific pitch-shifted harmony voices. Pitch shifting follows the source's own PitchTier rather than imposing a fixed pitch, so voiced material retains its original melodic contour while being transposed by the requested intervals.

Key Features:

Per-file FFNet class distillation — the current file generates both the heuristic labels and the training examples.
8 acoustic inputs — MFCC 1–3, F1, F2, intensity, HNR, and pitch.
Four rule classes — vowel, consonant, other, and silence.
Adaptive class weights — output-layer activations are temperature-shaped and normalized.
PitchTier harmonization — shifted voices follow the source's measured intonation.
Two possible vowel intervals — Vowel_interval_2 adds a second vowel harmony voice at 70% of the first vowel gain.
Continuous gain control — smoothed frame weights are linearly interpolated per sample.
Wet/dry synthesis — class levels shape the wet path before the final wet/dry balance.
Synthetic stereo — left and right channels use opposing dry/wet balances rather than separate voice positions.
Conditional limiter — the output is scaled to 0.99 only when its peak would exceed 0.99.

Scope: this is not a corpus-trained phonetic recognizer. There is no external training set, held-out evaluation, or phonetic ground truth. The FFNet learns the current file's own heuristic labels and is evaluated on the same frames.

Quick start

Select exactly one Sound object in Praat.
Run Neural_Phonetic_Harmonizer.praat.
Choose a preset or Manual.
For Manual, set the two vowel intervals, consonant interval, other interval, class levels, Wet_dry_mix, Smoothing_ms, Temperature, Stereo_width, and Random_seed.
Enable Draw_visualization to inspect the source, output, smoothed class weights, and summary information.
Click OK. The result is named <source>_harmonized_<preset>.

Input requirements: the selected Sound must be at least 0.1 s long, must not be silent or near-silent, and must produce at least one MFCC frame.

Channel behavior: multichannel input is downmixed to mono before analysis and processing. Stereo output, when enabled, is synthesized from the mono dry source and mono wet mix; the source stereo image is not preserved.

Per-file FFNet Distillation

What is per-file FFNet distillation?

The script first labels each frame with deterministic acoustic rules. It then uses those same frames and labels to train a feed-forward neural network inside Praat. The network therefore does not discover phonetic categories from an external corpus; it learns a smoother approximation of the rule system for the current file.

The process is best understood as rule distillation: the rules provide the targets, the FFNet learns their mapping from the eight features, and the network outputs are then used as continuous class weights for synthesis.

Network Structure

Inputs: 8 normalized features Hidden layer: 16 units Output layer: one output for each class actually present in the current file Training: up to 1000 iterations 100-iteration chunks learning rate = 0.001 minimum-squared-error cost

If fewer than two classes are present, the FFNet is skipped and the rule-based one-hot class labels are used directly as weights.

Training convergence: the script can stop early when the training cost changes by less than 0.1% for five consecutive 100-iteration chunks. This is a training-cost convergence heuristic, not validation-based early stopping.

Heuristic Phonetic Classes

What are the heuristic phonetic classes?

The four classes are operational categories defined by the script's acoustic rules. They are used as synthesis-control categories, not as validated linguistic labels.

Class	Rule
Silence	Intensity below the file-relative silence threshold.
Vowel	Valid F1/F2, HNR > 5 dB, F0 > 0, and F1 > 300 Hz.
Consonant	Above silence threshold and HNR < 3 dB. The frame may be voiced or unvoiced.
Other	All remaining non-silent frames.

Relative Silence Threshold

silence threshold = maximum file intensity - 35 dB lower bound = 5 dB

This makes the silence decision relative to the current recording level rather than using a fixed absolute dB threshold.

Formant Validity

Undefined F1/F2 values are filled with the current file's mean valid formants for the feature vector, but a frame with invalid formants still cannot satisfy the vowel rule. If no valid formants exist anywhere in the file, 500/1500 Hz are used only as neutral feature fills.

8 Input Features

Feature	Normalization
MFCC 1	Per-file min-max to 0–1
MFCC 2	Per-file min-max to 0–1
MFCC 3	Per-file min-max to 0–1
F1	Per-file min-max to 0–1
F2	Per-file min-max to 0–1
Intensity	Per-file min-max to 0–1
HNR	Per-file min-max to 0–1
Pitch	F0 divided by the maximum voiced F0 in the current file, then clipped to 0–1

Analysis Timing

Pitch, intensity, HNR, and formants are queried on the 10 ms control grid. MFCC timing is aligned separately using the MFCC object's actual first-frame time and frame spacing, so the nearest MFCC frame corresponds to the same physical analysis time rather than merely sharing the same frame number.

The formant ceiling is automatically limited to 5500 Hz or 90% of Nyquist, whichever is lower.

Temperature-Shaped Class Weights

What does Temperature do?

After training, the script reads the FFNet's output-layer activations and applies exponential temperature shaping. Lower Temperature values make the largest activation dominate more strongly; higher values distribute more weight across the available classes.

for each class c: e_c = exp((activation_c - max_activation) / Temperature) weight_c = e_c / sum(e_all_classes)

Classes absent from the current file are assigned effectively zero weight. The resulting values sum to 1, but they should be interpreted as temperature-shaped class weights, not calibrated probabilities.

PitchTier Harmonization

What is PitchTier harmonization?

Each shifted harmony voice is created from the source's own pitch contour. Praat extracts the PitchTier from a Manipulation object, multiplies every pitch point by the requested semitone ratio, replaces the PitchTier, and resynthesizes the Sound with overlap-add. The shifted voice therefore follows the source intonation rather than being forced to a fixed pitch.

pitch multiplier = 2^(interval_semitones / 12)

Harmony Voices

Voice	Control	Gain source
Vowel voice 1	Vowel_interval_1	Vowel weight × Vowel_level
Vowel voice 2	Vowel_interval_2	0.7 × vowel weight × Vowel_level
Consonant voice	Consonant_interval	Consonant weight × Consonant_level
Other voice	Other_interval	Other weight × Other_level

Vowel voice 2 is omitted entirely when Vowel_interval_2 = 0. There is no separate silence harmony voice.

Why can only voiced consonants be pitch-shifted?

PitchTier transposition acts on regions where Praat has pitch points. Truly unvoiced material has no such pitch contour, so an unvoiced consonant-class frame passes through the consonant harmony path essentially unshifted while still being controlled by Consonant_level. The script therefore reports how many consonant-class frames are voiced.

Adaptive Harmony Envelope

What is the adaptive harmony envelope?

The vowel, consonant, and other class weights are smoothed with a moving average and multiplied by their respective class levels. Instead of changing gain abruptly once every 10 ms frame, the script linearly interpolates between neighboring frame centers for every output sample.

wet vowel gain = smoothed vowel weight * Vowel_level wet consonant gain = smoothed consonant weight * Consonant_level wet other gain = smoothed other weight * Other_level

This produces continuous gain motion and avoids frame-boundary steps in the wet mix.

Wet/Dry Mix

dry gain = 1 - Wet_dry_mix wet gain = Wet_dry_mix

The class levels remain meaningful because the wet path is no longer normalized to a fixed peak before the final mix.

Stereo Width

What does Stereo_width do?

Stereo_width does not place harmony voices at different pan positions. Both channels use the same wet mix. Width is created by giving the two channels opposing dry/wet balances: the left channel becomes slightly wetter while the right becomes slightly drier as width increases.

left dry = dry_gain * (1 - Stereo_width * 0.5) left wet = wet_gain * (1 + Stereo_width * 0.5) right dry = dry_gain * (1 + Stereo_width * 0.5) right wet = wet_gain * (1 - Stereo_width * 0.5)

Stereo_width = 0 gives identical left/right balances. A negative Manual value selects mono output. Values above 1 are clamped to 1.

Current preset edge case: mono/stereo mode is determined from the entered Stereo_width before preset substitution. A negative form value can therefore leave a preset in mono even if the preset later supplies a positive width.

Presets

Preset	Vowel 1 / 2	Consonant	Other	V/C/O levels	Wet	Temp.	Width
Manual	User	User	User	User	User	User	User
Octave Chorus	+12 / 0	+12	+12	.60 / .40 / .50	.40	.30	.60
Fifth Harmony	+7 / 0	+7	+7	.70 / .50 / .60	.50	.25	.50
Vowel Choir	+7 / +12	0	+4	.90 / .20 / .40	.60	.20	.70
Dark Consonants	0 / 0	-12	-7	.30 / .80 / .60	.50	.35	.40
Shimmer	+12 / +19	+12	+12	.50 / .30 / .40	.40	.20	.80
Detuned Unison	+.15 / -.15	+.10	+.12	.80 / .60 / .70	.50	.30	.90
Major Chord	+4 / +7	+4	+7	.70 / .50 / .60	.50	.25	.60
Minor Chord	+3 / +7	+3	+7	.70 / .50 / .60	.50	.25	.60

Preset scope: presets override intervals, class levels, Wet_dry_mix, Temperature, and Stereo_width. Smoothing_ms, Random_seed, Draw_visualization, and Play_result remain controlled by the form.

Parameters

Name	Default	Description
Preset	Manual	Manual or one of eight built-in harmony configurations.
Vowel_interval_1	7.0	Primary vowel transposition in semitones.
Vowel_interval_2	0.0	Optional second vowel transposition; zero disables voice 2.
Consonant_interval	-5.0	Consonant-class transposition; only voiced consonant material can actually follow PitchTier transposition.
Other_interval	4.0	Transposition for the other class.
Vowel_level	0.7	Clamped to 0–1.
Consonant_level	0.5	Clamped to 0–1.
Other_level	0.6	Clamped to 0–1.
Wet_dry_mix	0.5	Clamped to 0–1; balances dry source and wet harmony mix.
Smoothing_ms	20	Moving-average smoothing duration for class weights.
Temperature	0.3	Must be > 0; values ≤ 0 are raised to 0.01.
Stereo_width	0.5	Upper-clamped to 1; negative Manual input selects mono.
Random_seed	0	0 = unpredictable; positive integer = reproducible FFNet initialization/training.
Draw_visualization	On	Draw the v0.8 diagnostic page.
Play_result	On	Play the processed Sound after completion.

Internal network settings: frame step = 10 ms, hidden units = 16, maximum training iterations = 1000, learning rate = 0.001, vowel HNR threshold = 5 dB, consonant HNR threshold = 3 dB.

Visualization

HEADER source | preset per-file FFNet class distillation Temperature SOURCE SOUND original waveform HARMONIZED OUTPUT final waveform channel count / peak SMOOTHED PHONETIC WEIGHTS red = vowel blue = consonant green = other SUMMARY STRIP rule-class distribution voiced reach of consonant class harmony intervals and levels wet/dry percentage stereo width or mono status smoothing output duration unvoiced-consonant caveat

How to read the phonetic-weight plot

The three curves show the smoothed class weights for vowel, consonant, and other. Silence is not drawn. These curves are upstream control signals: before reaching the wet audio they are multiplied by Vowel_level, Consonant_level, or Other_level and then interpolated continuously per sample.

The curves should therefore not be read directly as final audible harmony amplitudes or as calibrated phonetic probabilities.

Outputs

Object name: <source>_harmonized_<preset>.
Type: Sound, mono or synthetic stereo depending on Stereo_width / mono mode.
Duration: same processing duration as the source Sound.
Peak handling: final output is scaled to 0.99 only if its peak exceeds 0.99.
Info window: reports preset, intervals, wet/dry state, stereo/mono mode, feature analysis, class distribution, consonant voiced reach, seed, FFNet training progress or single-class fallback, and final output information.

Unvoiced consonants: the class weight and Consonant_level still affect them, but their pitch is not transposed because no PitchTier points exist in genuinely unvoiced regions.