Conductor Mix — v3.3 User Guide
An offline algorithmic ensemble conductor for two or more recorded Sounds. Analyze the ensemble in Praat and Python, then render a continuous stereo mix through coordinated, smoothly varying gain and spatial decisions. Version 3.3 introduces interpretive memory, Responsive and Complementary behaviors, phrase-aware decision making, and optional fixed chamber positioning.
What it does
Select two or more Sound objects in Praat. The script exports temporary audio, extracts per-Sound descriptors, and calls a Python conductor that segments and analyzes each source. It assigns acoustic roles and constructs a plan for gain and spatial automation, then renders the original, uncut recordings as a continuous stereo mix named ConductedMix.
What changes
Relative prominence, foreground/background relationships, and optional stereo placement. Responsive and Complementary modes introduce context-sensitive decisions rather than relying only on local role scores.
What stays intact
The source recordings' timing, pitch, articulation, and recorded content. No source separation, MIDI synthesis, EQ, reverberation, or audio-time rearrangement is performed.
Input Sounds may be mono, stereo, or multichannel, and may use different sample rates. The analysis uses a mono representation; the renderer converts every input to stereo as needed. This is a stereo-output instrument, not a multichannel-output mixer.
The ten roles and six states are interpretive descriptors, not ten independent audio processors. The audible changes in v3.3 are made through gain and pan. In the new behaviors, analytic role labels remain distinct from gain emphasis and level-based foreground estimates.
Quick start
- In Praat, select 2 or more Sound objects. Their selection order determines the default left-to-right stereo layout.
- Run the current v3.3 script
AI_Conductor_Mix.praatfrom Praat AudioTools (the instrument is now titled Conductor Mix). - Leave Segment_mode = Onset-based for a first trial; try Fixed or Hybrid for a different temporal resolution.
- Choose a Conductor_style (Dramatic is the current default) and an Interpretive_behaviour: Original, Responsive, or Complementary.
- For the new behaviors, start with Interpretive_memory_s = 4.0, Contextual_strength = 0.5, and Phrase_awareness = on.
- Choose Spatial_mode: original role-based pan, or fixed chamber positions for stable ensemble placement.
- Keep Draw_visualization and Play_result enabled, and click OK. The script creates and imports
ConductedMix.
Recommended A/B listening comparison
Use the same selected Sounds, levels, segmentation, style, and spatial mode for three runs: Original, Responsive, and Complementary. Compare their musical character at matched listening loudness. The new modes are alternatives, not a ranking of better and worse mixes.
numpy and soundfile is required. librosa (onset detection) and scipy (higher-quality resampling) are optional. The current v3.3 Praat script still searches for py/ai_conductor_mix.py inside plugin_AudioTools or for ai_conductor_mix.py in the current Praat directory. Renaming these code files requires updating the lookup in the Praat script separately; this guide changes only the instrument name and HTML filename.The three conducting behaviors
1 · Original Conductor (v3.2)
The legacy descriptor- and role-based algorithm, preserved in v3.3. Temporal leader rules, a foreground guarantee, a masking heuristic, role diversity, energy memory, tension, and role-dependent gain/pan contribute to the mix.
Compatibility: The default Original + Original role-based pan path reproduces v3.2. Selecting fixed chamber positions changes the legacy pan assignments intentionally.
2 · Responsive Conductor
Responds to salient gestures and changes in the acoustic material. Uses onset strength, spectral/energy descriptors, contextual priorities, recent foreground exposure, and limited anticipation to give meaningful gestures greater contrast without simply boosting every onset.
Typical interpretation: more immediate local contrast and expressive differentiation. Emphasis remains bounded and smoothed.
3 · Complementary Conductor
Makes room for significant gestures primarily by adjusting other voices. It considers concurrent activity, relative levels, spectral-centroid proximity, supporting roles, and recent conducting history. A small direct lift is possible when withdrawing other voices is ineffective.
Typical interpretation: greater restraint, continuity, and space between competing gestures. The conductor may deliberately preserve an already effective balance.
preserve decision, meaning no new emphasis is required.Interpretive memory & phrase awareness
Two distinct forms of memory
| Mechanism | Used for | Control |
|---|---|---|
| Energy memory (v3.2) | A running estimate of past acoustic energy, used in the tension and interruption rules. | Energy_memory_weight, used directly when Neutral style is selected; other style presets supply their own values. |
| Interpretive memory (v3.3) | Recent foreground exposure and committed gain behavior for each voice, plus the elapsed time since foreground activity, attenuation history, and continuity of gain offsets. | Interpretive_memory_s (default 4 seconds). Active only in Responsive and Complementary. |
Interpretive memory is updated according to elapsed time rather than simply counting segments. It is based on committed commanded gains and approximate source levels, so the next decision can depend on the conductor's previous actions. This is an algorithmic history of decisions, not learned musical memory or a psychoacoustic foreground model.
m(t + Δt) = exp(−Δt / τ) · m(t) + [1 − exp(−Δt / τ)] · u(t)
Here τ is the interpretive memory duration and u(t) summarizes recent conducting activity for a voice.
Phrase-aware context
When enabled, the Python engine estimates broader regions from ensemble-level activity, pauses, energy trajectories, onset density, and prominent changes. The possible context labels are continuing, building, contrasting, and releasing. They adjust the intensity, persistence, and relaxation of contextual decisions.
The analysis can inspect limited upcoming source events (up to approximately 1.5 seconds) to help prepare a transition. This is available because the process runs offline. Detected phrase boundaries are heuristic candidates, not score-informed or formally verified phrase boundaries; low-confidence material receives more neutral behavior.
The contextual conductor also groups nearby onsets into joint decisions, evaluates the active voices against the same memory snapshot, and then commits the coordinated gain changes. Individual segment boundaries remain available for continuous coverage of the source recordings.
The 10 analytic roles
Each analyzed segment receives a descriptive role based on percentile-ranked descriptors and style-dependent weights. In Original, the listed base gains and pan factors directly contribute to the rendered mix. In Responsive and Complementary, roles provide context, while the new conductor calculates gains from a neutral reference level plus contextual offsets. A role label alone does not prove that a voice is foregrounded.
| Role | Acoustic interpretation | Legacy base gain | Legacy pan factor |
|---|---|---|---|
| leader | High energy and brightness | 0.90 | 0.3 |
| shadow | Lower energy, stable material | 0.45 | 1.0 |
| resonance | Harmonicity, stability, low noise | 0.60 | 0.8 |
| noise_fringe | Noisy, less pitched material | 0.35 | 1.2 |
| pulse_carrier | Attack-rich, rhythmically salient segments | 0.70 | 0.6 |
| interruption | Strong event or contrast after quieter context | 0.88 | 0.1 |
| sustain_bed | Stable, sustained, low-flux material | 0.40 | 0.9 |
| contrast_voice | Bright, pitched, differentiating material | 0.75 | 1.1 |
| memory_trace | Quiet, stable, low-flux material | 0.30 | 0.7 |
| silence | Very low energy; intentional omission when assigned | 0.00 | 0.0 |
Legacy pan is calculated as base_pan(file) × role_pan_factor, clipped to [−1, 1]. It is not an additive offset. The baseline gain is modified further by prominence and polyphony. In contextual modes, a neutral commanded gain is based on 0.70 with a per-voice dB offset and distinct-source polyphony scaling.
The six conducting states
In Original, states are primarily descriptive labels derived from style-shaped tension, energy, and stability. In the two contextual behaviors, the current state also adjusts decision strength, rate limits, and the relaxation of existing gain offsets.
| State | Interpretive use in contextual modes |
|---|---|
| sparse | Restrains unnecessary foreground competition. |
| balanced | Favors moderate contrast and steady relationships. |
| agitated | Permits stronger local contrast and faster commanded transitions. |
| saturated | Reduces further accumulation of competing foreground material. |
| suspended | Favors sustained continuity and slower change. |
| released | Encourages relaxation toward a neutral balance. |
These names are heuristic conducting states, not guarantees about the musical or emotional character of the resulting mix. They do not trigger separate spectral, granular, or reverberation processors.
Processing pipeline — five stages
- Praat export and macro analysis. Export the selected Sounds to temporary WAV files and calculate per-Sound RMS, pitch, pitch range, intensity, intensity range, harmonicity, and a spectral-centroid estimate.
- Python analysis. Segment each source (Fixed, Onset, Hybrid), calculate 10 local acoustic descriptors, normalize across the ensemble, and combine local measurements with lighter Praat-derived macro priors.
- Conducting decisions. Original follows v3.2 role and temporal rules. Responsive and Complementary instead apply group decisions, interpretive memory, phrase context, distinct-source polyphony, and bounded gain offsets; all create a tabular mix plan.
- Continuous stereo rendering. Load the uncut sources, build coupled gain/pan envelopes, smooth them, render the full-time streams, and write a float32 stereo WAV.
- Praat import and optional picture. The Praat script waits for a completion file, imports the WAV as
ConductedMix, and can draw waveforms, roles, gain curves, memory curves, and a spectrogram.
Local descriptors calculated in Python
| Descriptor | Meaning |
|---|---|
rms | Root-mean-square amplitude. |
zcr | Zero-crossing rate. |
centroid | Spectral centre of gravity (Hz). |
bandwidth | Spectral dispersion. |
flux | Positive spectral change relative to the previous segment. |
roughness | Proxy based on the proportion of energy above 3 kHz; not a psychoacoustic roughness model. |
pitch_salience | Autocorrelation-based periodicity proxy. |
hnr_proxy | Cepstral harmonicity proxy; not a direct HNR measurement in dB. |
attack_slope | Relative early-versus-late energy of the segment. |
temporal_stability | Stability estimated from short-block energy variation. |
Continuous rendering and spatial organization
Sample-level gain and pan envelopes
- For every original source, build a per-sample gain target from its plan rows. In overlapping regions the highest commanded gain takes precedence; gain and pan remain coupled to the same chosen command.
- Apply a Hann kernel with a 150 ms half-width (approximately 300 ms total) to the gain envelope. Pan is smoothed over active regions with support-aware weighting.
- Apply the resulting gain and equal-power left/right coefficients to the uncut source channels; each source is accumulated once at its original position in time.
- Preserve sample rate at the highest input rate, resampling lower-rate sources when necessary. Trim trailing silence.
- Only attenuate the final stereo mix when its peak exceeds
0.95. There is no automatic upward peak normalization. Write a 32-bit floating-point stereo WAV.
Continuous envelope processing reduces discontinuities associated with cutting and rejoining segments. The renderer also reports short-window RMS dips near plan boundaries, but this check is diagnostic and does not guarantee that every audible artifact has been eliminated.
Spatial modes
| Mode | Behavior | When to use |
|---|---|---|
| Original role-based pan Default | Positions depend on source order and the role-dependent pan factor. Positions may change as roles change. | Exploratory ensemble interpretation and exact v3.2 spatial behavior in Original mode. |
| Fixed chamber positions | Stable left-to-right positions based on source order: restrained spans for two, three, or more voices. Role changes do not move the voices. | Chamber-music experiments and controlled comparisons with a stable spatial image. |
Parameters, defaults, and presets
Segmentation
| Praat form control | Default | Function |
|---|---|---|
Segment_mode | Onset-based | Fixed frames, onset boundaries, or Hybrid (onsets with fixed subdivisions of long regions). |
Frame_size_ms | 500 ms | Frame length for Fixed/Hybrid. |
Frame_hop_ms | 250 ms | Step between fixed frames; overlapping frames are possible. |
Min_onset_gap_ms | 50 ms | Minimum gap between accepted onset boundaries. |
Existing conductor styles
Conductor_style defaults to Dramatic. Selecting one of the four styles establishes an internal set of energy-memory, tension, silence, and nonlinear-reaction values. Only Neutral uses the four editable form controls below it.
| Style | Energy memory | Tension sensitivity | Allow silence | Nonlinear reactions |
|---|---|---|---|---|
| Neutral (custom) | 0.40* | 0.60* | Yes* | Yes* |
| Dramatic Default | 0.55 | 0.80 | Yes | Yes |
| Minimal | 0.30 | 0.30 | Yes | No |
| Dense | 0.50 | 0.60 | No | Yes |
*Neutral defaults are the editable form values; the user may replace them. All styles also change the underlying scoring weights and style-dependent tension curve.
New interpretive controls (v3.3)
| Praat form control | Default | Function |
|---|---|---|
Interpretive_behaviour | Original Conductor (v3.2) | Original, Responsive, or Complementary. |
Interpretive_memory_s | 4.0 s | Time constant for per-voice interpretive history in contextual modes. Effective range 0.25–30 s. |
Contextual_strength | 0.5 | Degree of contextual intervention, constrained to 0–1. |
Phrase_awareness | Yes | Enable source-based phrase-context estimates and limited lookahead in contextual modes. |
Spatial_mode | Original role-based pan | Original role-based pan or fixed chamber positions. Fixed chamber positions also work with Original behavior, but deliberately change pan. |
Output
| Control | Default | Function |
|---|---|---|
Draw_visualization | Yes | Draw a Praat Picture with waveforms, roles, gain, and contextual diagnostics. |
Play_result | Yes | Play ConductedMix after importing it. |
Export_mix_plan | No | Retain the plan CSV in Praat's temporary directory; otherwise it is removed during cleanup. Save or copy it before a subsequent run overwrites it. |
Two independent selectors: Conductor style sets the general scoring/tension character, while Interpretive behaviour selects the local/contextual decision strategy. They are not additional audio-effect processors.
Visualization in the Praat Picture window
When Draw_visualization is enabled, the Praat script uses the mix plan and a separate 50 Hz diagnostic table from the same processing run.
- Input waveforms: up to four original Sounds, each identified by name.
- Conducted mix waveform: mono overview of the rendered stereo result.
- Analytic role timeline: color-coded role intervals for each voice; bar height represents plan priority. In contextual modes, gray marks indicate gain-emphasized voices and black marks indicate level-foreground estimates.
- Smoothed gain curves: one curve per input Sound, derived from the renderer's smoothed gain envelopes. The curves represent the per-voice envelopes before any final global peak attenuation.
- Estimated phrase regions and boundaries: continuing, building, contrasting, and releasing; dashed vertical lines mark estimated boundaries when phrase awareness is enabled.
- Interpretive-memory graph: recent per-voice foreground exposure in the contextual behaviors.
- Mix spectrogram: displayed up to 5 kHz, subject to sample rate.
- Summary: number of plan rows, duration, style, memory settings, and, in contextual modes, time proportions labeled preserve/emphasize/withdraw.
gain_emphasized marks a voice favored by the gain commands. level_foreground estimates the voice at least 3 dB above the others using source RMS and commanded gain in the interval. Neither marker is an objective measure of perceptual musical leadership. The preserve/emphasize/withdraw proportions are derived from a time grid, not independent counts of artistic decisions.Mix plan and diagnostic fields
The tab-delimited plan file begins with #max_dur=..., followed by one header row and plan rows. The first 18 columns remain compatible with the v3.2 plan format:
time, file_index, file_name, seg_index, src_start, src_dur, gain, role, layer_group, entry_time, exit_time, transform, state, density, priority, pan, tension, polyphony
In Responsive and Complementary, the following eight additional columns are appended (26 total):
| Column | Meaning |
|---|---|
behavior | Responsive or Complementary decision mode. |
fg_memory | Recent foreground exposure snapshot. |
ctx_priority | Contextual priority (heuristic). |
phrase_ctx | Estimated phrase context. |
ctx_gain_db | Committed relative gain offset in dB. |
decision | Preserve, emphasize, or withdraw. |
gain_emphasized | 1 if the gain offsets clearly favor the voice, otherwise 0. |
level_foreground | 1 if the source-RMS-plus-gain estimate places the voice sufficiently ahead, otherwise 0. |
The plan's transform, density, and layer_group columns are descriptive metadata, not active independent DSP processors. The script also calculates an internal fade value, but the renderer uses its own fixed smoothing window rather than this value. Rendering is driven by timing, gain, and pan commands. Exact source sample indices and source sample rate are held internally; they are not extra CSV columns.
FAQ and troubleshooting
Install the required packages using python -m pip install numpy soundfile. For improved onset detection and resampling, optionally install librosa scipy. Confirm that the Python executable selected by the Praat script has these packages. On Windows, the script invokes python; on macOS/Linux it normally invokes python3.
Inspect the Python log and plan. Check the source levels, whether Allow_silence is enabled through the selected style, whether a silence role was assigned, and whether the contextual coverage check reported an error. Also note that the renderer does not raise quiet outputs to a target loudness.
These behaviors are context-sensitive and may both decide to preserve the existing balance. Try material with contrasting, overlapping gestures, compare the gain curves, and, if appropriate, vary Contextual_strength. A bigger measured difference is not necessarily a better musical result.
The renderer smooths gain and pan, but this does not guarantee artifact-free output in every case. Inspect the boundary RMS diagnostics, segment choices, and the actual gain curves. For chamber music, choose Fixed chamber positions to avoid role-dependent pan movement.
Praat waits up to ten minutes for completion. Python exceptions write an error status to release the poller sooner; consult the log copied into the Praat Info window. Confirm both files belong to the same v3.3 version and that the Python backend is accessible.
No machine learning, MIDI score following, pitch shifting, time stretching, dynamic EQ, reverberation, DAW automation export, or multichannel output. Neural Conducting and multidimensional post-production are separate research directions, not features claimed by Conductor Mix v3.3.