Praat AudioTools · Hybrid Praat + Python

Conductor Mix — v3.3 User Guide

An offline algorithmic ensemble conductor for two or more recorded Sounds. Analyze the ensemble in Praat and Python, then render a continuous stereo mix through coordinated, smoothly varying gain and spatial decisions. Version 3.3 introduces interpretive memory, Responsive and Complementary behaviors, phrase-aware decision making, and optional fixed chamber positioning.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 3.3 (2026) License: MIT License Source: Praat AudioTools on GitHub
Interpretive principle: A conducting decision depends not only on the current acoustic gesture, but also on the recent history of the conductor's decisions and the evolving ensemble context. The two new behaviors implement this idea with deterministic algorithms. No neural network, trained model, or external AI service is used.

What it does

Select two or more Sound objects in Praat. The script exports temporary audio, extracts per-Sound descriptors, and calls a Python conductor that segments and analyzes each source. It assigns acoustic roles and constructs a plan for gain and spatial automation, then renders the original, uncut recordings as a continuous stereo mix named ConductedMix.

What changes

Relative prominence, foreground/background relationships, and optional stereo placement. Responsive and Complementary modes introduce context-sensitive decisions rather than relying only on local role scores.

What stays intact

The source recordings' timing, pitch, articulation, and recorded content. No source separation, MIDI synthesis, EQ, reverberation, or audio-time rearrangement is performed.

Input Sounds may be mono, stereo, or multichannel, and may use different sample rates. The analysis uses a mono representation; the renderer converts every input to stereo as needed. This is a stereo-output instrument, not a multichannel-output mixer.

The ten roles and six states are interpretive descriptors, not ten independent audio processors. The audible changes in v3.3 are made through gain and pan. In the new behaviors, analytic role labels remain distinct from gain emphasis and level-based foreground estimates.

Quick start

  1. In Praat, select 2 or more Sound objects. Their selection order determines the default left-to-right stereo layout.
  2. Run the current v3.3 script AI_Conductor_Mix.praat from Praat AudioTools (the instrument is now titled Conductor Mix).
  3. Leave Segment_mode = Onset-based for a first trial; try Fixed or Hybrid for a different temporal resolution.
  4. Choose a Conductor_style (Dramatic is the current default) and an Interpretive_behaviour: Original, Responsive, or Complementary.
  5. For the new behaviors, start with Interpretive_memory_s = 4.0, Contextual_strength = 0.5, and Phrase_awareness = on.
  6. Choose Spatial_mode: original role-based pan, or fixed chamber positions for stable ensemble placement.
  7. Keep Draw_visualization and Play_result enabled, and click OK. The script creates and imports ConductedMix.

Recommended A/B listening comparison

Use the same selected Sounds, levels, segmentation, style, and spatial mode for three runs: Original, Responsive, and Complementary. Compare their musical character at matched listening loudness. The new modes are alternatives, not a ranking of better and worse mixes.

Dependencies and current file names: Python 3 with numpy and soundfile is required. librosa (onset detection) and scipy (higher-quality resampling) are optional. The current v3.3 Praat script still searches for py/ai_conductor_mix.py inside plugin_AudioTools or for ai_conductor_mix.py in the current Praat directory. Renaming these code files requires updating the lookup in the Praat script separately; this guide changes only the instrument name and HTML filename.

The three conducting behaviors

1 · Original Conductor (v3.2)

The legacy descriptor- and role-based algorithm, preserved in v3.3. Temporal leader rules, a foreground guarantee, a masking heuristic, role diversity, energy memory, tension, and role-dependent gain/pan contribute to the mix.

Compatibility: The default Original + Original role-based pan path reproduces v3.2. Selecting fixed chamber positions changes the legacy pan assignments intentionally.

2 · Responsive Conductor

Responds to salient gestures and changes in the acoustic material. Uses onset strength, spectral/energy descriptors, contextual priorities, recent foreground exposure, and limited anticipation to give meaningful gestures greater contrast without simply boosting every onset.

Typical interpretation: more immediate local contrast and expressive differentiation. Emphasis remains bounded and smoothed.

3 · Complementary Conductor

Makes room for significant gestures primarily by adjusting other voices. It considers concurrent activity, relative levels, spectral-centroid proximity, supporting roles, and recent conducting history. A small direct lift is possible when withdrawing other voices is ineffective.

Typical interpretation: greater restraint, continuity, and space between competing gestures. The conductor may deliberately preserve an already effective balance.

These are conducting strategies, not fixed shifts or machine-learned presets. The Complementary behavior does not apply a 37% temporal shift; its decisions depend on the current material and history. Both contextual behaviors permit a preserve decision, meaning no new emphasis is required.

Interpretive memory & phrase awareness

Two distinct forms of memory

MechanismUsed forControl
Energy memory (v3.2)A running estimate of past acoustic energy, used in the tension and interruption rules.Energy_memory_weight, used directly when Neutral style is selected; other style presets supply their own values.
Interpretive memory (v3.3)Recent foreground exposure and committed gain behavior for each voice, plus the elapsed time since foreground activity, attenuation history, and continuity of gain offsets.Interpretive_memory_s (default 4 seconds). Active only in Responsive and Complementary.

Interpretive memory is updated according to elapsed time rather than simply counting segments. It is based on committed commanded gains and approximate source levels, so the next decision can depend on the conductor's previous actions. This is an algorithmic history of decisions, not learned musical memory or a psychoacoustic foreground model.

Time-based memory:
m(t + Δt) = exp(−Δt / τ) · m(t) + [1 − exp(−Δt / τ)] · u(t)
Here τ is the interpretive memory duration and u(t) summarizes recent conducting activity for a voice.

Phrase-aware context

When enabled, the Python engine estimates broader regions from ensemble-level activity, pauses, energy trajectories, onset density, and prominent changes. The possible context labels are continuing, building, contrasting, and releasing. They adjust the intensity, persistence, and relaxation of contextual decisions.

The analysis can inspect limited upcoming source events (up to approximately 1.5 seconds) to help prepare a transition. This is available because the process runs offline. Detected phrase boundaries are heuristic candidates, not score-informed or formally verified phrase boundaries; low-confidence material receives more neutral behavior.

The contextual conductor also groups nearby onsets into joint decisions, evaluates the active voices against the same memory snapshot, and then commits the coordinated gain changes. Individual segment boundaries remain available for continuous coverage of the source recordings.

The 10 analytic roles

Each analyzed segment receives a descriptive role based on percentile-ranked descriptors and style-dependent weights. In Original, the listed base gains and pan factors directly contribute to the rendered mix. In Responsive and Complementary, roles provide context, while the new conductor calculates gains from a neutral reference level plus contextual offsets. A role label alone does not prove that a voice is foregrounded.

RoleAcoustic interpretationLegacy base gainLegacy pan factor
leaderHigh energy and brightness0.900.3
shadowLower energy, stable material0.451.0
resonanceHarmonicity, stability, low noise0.600.8
noise_fringeNoisy, less pitched material0.351.2
pulse_carrierAttack-rich, rhythmically salient segments0.700.6
interruptionStrong event or contrast after quieter context0.880.1
sustain_bedStable, sustained, low-flux material0.400.9
contrast_voiceBright, pitched, differentiating material0.751.1
memory_traceQuiet, stable, low-flux material0.300.7
silenceVery low energy; intentional omission when assigned0.000.0

Legacy pan is calculated as base_pan(file) × role_pan_factor, clipped to [−1, 1]. It is not an additive offset. The baseline gain is modified further by prominence and polyphony. In contextual modes, a neutral commanded gain is based on 0.70 with a per-voice dB offset and distinct-source polyphony scaling.

The six conducting states

In Original, states are primarily descriptive labels derived from style-shaped tension, energy, and stability. In the two contextual behaviors, the current state also adjusts decision strength, rate limits, and the relaxation of existing gain offsets.

StateInterpretive use in contextual modes
sparseRestrains unnecessary foreground competition.
balancedFavors moderate contrast and steady relationships.
agitatedPermits stronger local contrast and faster commanded transitions.
saturatedReduces further accumulation of competing foreground material.
suspendedFavors sustained continuity and slower change.
releasedEncourages relaxation toward a neutral balance.

These names are heuristic conducting states, not guarantees about the musical or emotional character of the resulting mix. They do not trigger separate spectral, granular, or reverberation processors.

Processing pipeline — five stages

  1. Praat export and macro analysis. Export the selected Sounds to temporary WAV files and calculate per-Sound RMS, pitch, pitch range, intensity, intensity range, harmonicity, and a spectral-centroid estimate.
  2. Python analysis. Segment each source (Fixed, Onset, Hybrid), calculate 10 local acoustic descriptors, normalize across the ensemble, and combine local measurements with lighter Praat-derived macro priors.
  3. Conducting decisions. Original follows v3.2 role and temporal rules. Responsive and Complementary instead apply group decisions, interpretive memory, phrase context, distinct-source polyphony, and bounded gain offsets; all create a tabular mix plan.
  4. Continuous stereo rendering. Load the uncut sources, build coupled gain/pan envelopes, smooth them, render the full-time streams, and write a float32 stereo WAV.
  5. Praat import and optional picture. The Praat script waits for a completion file, imports the WAV as ConductedMix, and can draw waveforms, roles, gain curves, memory curves, and a spectrogram.

Local descriptors calculated in Python

DescriptorMeaning
rmsRoot-mean-square amplitude.
zcrZero-crossing rate.
centroidSpectral centre of gravity (Hz).
bandwidthSpectral dispersion.
fluxPositive spectral change relative to the previous segment.
roughnessProxy based on the proportion of energy above 3 kHz; not a psychoacoustic roughness model.
pitch_salienceAutocorrelation-based periodicity proxy.
hnr_proxyCepstral harmonicity proxy; not a direct HNR measurement in dB.
attack_slopeRelative early-versus-late energy of the segment.
temporal_stabilityStability estimated from short-block energy variation.

Continuous rendering and spatial organization

Sample-level gain and pan envelopes

  1. For every original source, build a per-sample gain target from its plan rows. In overlapping regions the highest commanded gain takes precedence; gain and pan remain coupled to the same chosen command.
  2. Apply a Hann kernel with a 150 ms half-width (approximately 300 ms total) to the gain envelope. Pan is smoothed over active regions with support-aware weighting.
  3. Apply the resulting gain and equal-power left/right coefficients to the uncut source channels; each source is accumulated once at its original position in time.
  4. Preserve sample rate at the highest input rate, resampling lower-rate sources when necessary. Trim trailing silence.
  5. Only attenuate the final stereo mix when its peak exceeds 0.95. There is no automatic upward peak normalization. Write a 32-bit floating-point stereo WAV.

Continuous envelope processing reduces discontinuities associated with cutting and rejoining segments. The renderer also reports short-window RMS dips near plan boundaries, but this check is diagnostic and does not guarantee that every audible artifact has been eliminated.

Spatial modes

ModeBehaviorWhen to use
Original role-based pan DefaultPositions depend on source order and the role-dependent pan factor. Positions may change as roles change.Exploratory ensemble interpretation and exact v3.2 spatial behavior in Original mode.
Fixed chamber positionsStable left-to-right positions based on source order: restrained spans for two, three, or more voices. Role changes do not move the voices.Chamber-music experiments and controlled comparisons with a stable spatial image.
Channel handling: Mono input is duplicated before stereo gain/pan; stereo input is preserved as two streams before channel-specific attenuation, and multichannel input is folded to stereo. For stereo sources the control acts as a left/right balance on existing channels, not object-based repositioning. Fixed chamber mode fixes the pan command; changes in stereo source content can still affect perceived location.

Parameters, defaults, and presets

Segmentation

Praat form controlDefaultFunction
Segment_modeOnset-basedFixed frames, onset boundaries, or Hybrid (onsets with fixed subdivisions of long regions).
Frame_size_ms500 msFrame length for Fixed/Hybrid.
Frame_hop_ms250 msStep between fixed frames; overlapping frames are possible.
Min_onset_gap_ms50 msMinimum gap between accepted onset boundaries.

Existing conductor styles

Conductor_style defaults to Dramatic. Selecting one of the four styles establishes an internal set of energy-memory, tension, silence, and nonlinear-reaction values. Only Neutral uses the four editable form controls below it.

StyleEnergy memoryTension sensitivityAllow silenceNonlinear reactions
Neutral (custom)0.40*0.60*Yes*Yes*
Dramatic Default0.550.80YesYes
Minimal0.300.30YesNo
Dense0.500.60NoYes

*Neutral defaults are the editable form values; the user may replace them. All styles also change the underlying scoring weights and style-dependent tension curve.

New interpretive controls (v3.3)

Praat form controlDefaultFunction
Interpretive_behaviourOriginal Conductor (v3.2)Original, Responsive, or Complementary.
Interpretive_memory_s4.0 sTime constant for per-voice interpretive history in contextual modes. Effective range 0.25–30 s.
Contextual_strength0.5Degree of contextual intervention, constrained to 0–1.
Phrase_awarenessYesEnable source-based phrase-context estimates and limited lookahead in contextual modes.
Spatial_modeOriginal role-based panOriginal role-based pan or fixed chamber positions. Fixed chamber positions also work with Original behavior, but deliberately change pan.

Output

ControlDefaultFunction
Draw_visualizationYesDraw a Praat Picture with waveforms, roles, gain, and contextual diagnostics.
Play_resultYesPlay ConductedMix after importing it.
Export_mix_planNoRetain the plan CSV in Praat's temporary directory; otherwise it is removed during cleanup. Save or copy it before a subsequent run overwrites it.

Two independent selectors: Conductor style sets the general scoring/tension character, while Interpretive behaviour selects the local/contextual decision strategy. They are not additional audio-effect processors.

Visualization in the Praat Picture window

When Draw_visualization is enabled, the Praat script uses the mix plan and a separate 50 Hz diagnostic table from the same processing run.

Read the picture carefully: the role label is an analytic classification. gain_emphasized marks a voice favored by the gain commands. level_foreground estimates the voice at least 3 dB above the others using source RMS and commanded gain in the interval. Neither marker is an objective measure of perceptual musical leadership. The preserve/emphasize/withdraw proportions are derived from a time grid, not independent counts of artistic decisions.

Mix plan and diagnostic fields

The tab-delimited plan file begins with #max_dur=..., followed by one header row and plan rows. The first 18 columns remain compatible with the v3.2 plan format:

time, file_index, file_name, seg_index, src_start, src_dur,
gain, role, layer_group, entry_time, exit_time, transform,
state, density, priority, pan, tension, polyphony

In Responsive and Complementary, the following eight additional columns are appended (26 total):

ColumnMeaning
behaviorResponsive or Complementary decision mode.
fg_memoryRecent foreground exposure snapshot.
ctx_priorityContextual priority (heuristic).
phrase_ctxEstimated phrase context.
ctx_gain_dbCommitted relative gain offset in dB.
decisionPreserve, emphasize, or withdraw.
gain_emphasized1 if the gain offsets clearly favor the voice, otherwise 0.
level_foreground1 if the source-RMS-plus-gain estimate places the voice sufficiently ahead, otherwise 0.

The plan's transform, density, and layer_group columns are descriptive metadata, not active independent DSP processors. The script also calculates an internal fade value, but the renderer uses its own fixed smoothing window rather than this value. Rendering is driven by timing, gain, and pan commands. Exact source sample indices and source sample rate are held internally; they are not extra CSV columns.

Coverage protection in contextual modes: the engine keeps all segment boundaries when grouping nearby decisions, searches overlapping segments without a fixed backward limit, and checks that intended non-silent material has a corresponding plan row. If it detects an unintended coverage gap, processing stops with an error instead of writing a completed result.

FAQ and troubleshooting

Python not found / missing modules

Install the required packages using python -m pip install numpy soundfile. For improved onset detection and resampling, optionally install librosa scipy. Confirm that the Python executable selected by the Praat script has these packages. On Windows, the script invokes python; on macOS/Linux it normally invokes python3.

The mix is unexpectedly quiet or a voice disappears

Inspect the Python log and plan. Check the source levels, whether Allow_silence is enabled through the selected style, whether a silence role was assigned, and whether the contextual coverage check reported an error. Also note that the renderer does not raise quiet outputs to a target loudness.

Responsive and Complementary sound similar

These behaviors are context-sensitive and may both decide to preserve the existing balance. Try material with contrasting, overlapping gestures, compare the gain curves, and, if appropriate, vary Contextual_strength. A bigger measured difference is not necessarily a better musical result.

Clicks, dips, or unnatural movement

The renderer smooths gain and pan, but this does not guarantee artifact-free output in every case. Inspect the boundary RMS diagnostics, segment choices, and the actual gain curves. For chamber music, choose Fixed chamber positions to avoid role-dependent pan movement.

Timed out or Python conductor failed

Praat waits up to ten minutes for completion. Python exceptions write an error status to release the poller sooner; consult the log copied into the Praat Info window. Confirm both files belong to the same v3.3 version and that the Python backend is accessible.

What this version does not do

No machine learning, MIDI score following, pitch shifting, time stretching, dynamic EQ, reverberation, DAW automation export, or multichannel output. Neural Conducting and multidimensional post-production are separate research directions, not features claimed by Conductor Mix v3.3.