AudioTools Call & Response

What this does

AudioTools Call & Response is a phrase-based, interactive co-improvisation system. It listens to a live performer, analyses a completed acoustic gesture and chooses an appropriate musical response. A stable pitched sound, a noise texture, a sequence of attacks and a glissando lead to different transformation possibilities. The same performance can also recall earlier material, transform a recalled sound or play unprocessed sounds from a prepared corpus.

The system accumulates a sonic memory of live phrases (Corpus A), computer-generated results (Corpus B) and optional prepared recordings (Corpus C). Its retrieval considers both the current phrase and a decaying history of earlier phrases. Independently, its stereo mixer can add secondary memory voices and a quieter evolving background.

Key capabilities in v0.4.12:

• Continuous microphone listening: capture continues during analysis, Praat rendering and playback; a new gesture does not have to wait for the previous answer.
• Six gesture-dependent transformations: Paulstretch, Markov Soundscape Weaver, MDS Space Navigator, Spiral Pitch Dance, 8-Channel Speed Deviations and Percussive Audio Groove Creator.
• Three separate corpora: live phrases, processed computer responses and an optional read-only folder of prepared audio recordings.
• Phrase-level influence memory: earlier gestures can shape retrieval with decaying weights, coherence gating and an optional trend projection.
• Several response strategies: transform, recall, recall-and-transform, weighted Hybrid and strict, unprocessed Corpus C recall.
• Polyphonic, spatial performance: foreground answers, delayed secondary voices, an optional background texture and dynamic stereo placement.
• Perceptual output balancing: stereo-aware, burst-aware level matching, optional follow-live adjustment and peak protection.
• Visual feedback: separate performer and system status, an end-of-phrase silence indicator and a Memory Map of sounds and relationships.
• Temporary working memory: generated session files are cleaned up after successful shutdown; supplied corpus recordings are not rewritten.

Quick start

1. Connect the microphone or instrument to the intended input device, and select an output device. Use headphones or isolated monitoring to reduce audio feedback into the microphone.
2. Set input gain before starting. While you are silent, ordinary room noise should not trigger the detector. During normal playing, the input should register clearly without clipping. See Microphone setup and sensitivity below.
3. In Praat, run AudioTools_CallResponse.praat and choose Source: Live - continuous (v0.2). For the full memory system, choose Preset: Hybrid memory (default); the form's initial preset is Custom (form and dialogs).
4. Keep Transformation: Automatic (gesture classifier). You do not need a prepared corpus for Hybrid memory: leave Corpus_C_folder empty to build memory from the performance.
5. Start the session and remain quiet while the display reads STAY QUIET (input calibration). Once it reads PLAY WHEN READY, you can begin.
6. When it reads RECORDING YOUR PHRASE, finish the gesture and leave a pause below the end threshold. The default required quiet interval is 0.6 seconds. The display then shows PHRASE CAPTURED briefly.
7. You may play again immediately, even while a response is sounding. Processing and playback are separate from the microphone's listening state.
8. Use the on-screen STOP button or Esc / S (with the Demo window focused) to end the session and allow cleanup to finish.

Microphone setup, input gain and sensitivity

The phrase detector reacts to audio arriving at the input, not only to sounds intentionally performed by the musician. Footsteps, handling noise, air conditioning or nearby speech can therefore appear to be musical gestures, particularly with inappropriate microphone placement or gain. This is a practical limitation of energy-based phrase detection, not a feature of the Hybrid memory response mode.

Practical sound-check

1. With your instrument silent, observe the input meter and the DETECTOR values. Routine background noise should stay below start; the screen should remain at PLAY WHEN READY.
2. Play the quietest gesture you expect to use. Check that the screen changes to RECORDING YOUR PHRASE and that your full gesture is captured.
3. Stop playing. The quiet counter should advance toward the required duration, and PHRASE CAPTURED should appear.
4. Play a normal and a strong gesture. Confirm that the input is clear and not distorted or clipped. The meter is a detection guide, not a substitute for an audio-interface clipping indicator.
5. If external noises trigger responses, adjust the hardware gain and, when necessary, increase Sensitivity_dB in small steps. Restart the session after changing input gain so that the detector can recalibrate under the new conditions.

Core concept: listen, transform, remember and spatialise

Python captures and analyses live sound, detects phrase boundaries, selects from memory, schedules responses and mixes playback. The existing Praat AudioTools scripts render the six transformations. These transformations act on completed phrases, rather than running as continuous sample-by-sample effects.

In Continuous mode the next phrase can begin while earlier answers play or render. Hybrid memory may produce a response, a later memory layer and an independent background, but those activities do not determine when the performer must play or stop.

Input and performance modes

The Transformation field may be automatic or forced to a particular script. Test all six is only for selected Sounds. A forced selection does not make ineligible material necessarily produce a valid transformation.

Six gesture-dependent transformations

The automatic classifier compares several acoustic and morphological measurements rather than using a single instrument label. The following are musical tendencies, not promises that every example will select the named transformation. Several candidates can be plausible, and a candidate must meet its own eligibility requirements.

The system also considers pitch continuity, harmonic-to-noise ratio, spectral shape, MFCC-related timbre, onset density, envelope complexity, segmentation and other measurements. If no eligible transformation exceeds its confidence threshold, the source phrase can be returned unchanged instead of forcing an inappropriate effect. A rendering failure likewise has a documented passthrough fallback.

Three sonic-memory corpora

The memory engine distinguishes the origin of a stored sound. These collections are separate, but they share one similarity-search and response-selection system.

Corpus A is not the same as Corpus B. A played flute phrase may be stored in A, while an extended, noisy or varispeed-derived answer to that phrase enters B as a different sound with different acoustic measurements. Material from B can later return as a memory layer or be subjected to another appropriate transformation.

Stored items carry ancestry and generation information. The system excludes the currently analysed phrase, can avoid very recent material, enforces recall cooldowns and limits successive transformations. Simply replaying a stored phrase does not count as creating a new processed sound.

The live and processed corpora are held in RAM with limits on item count and total stored audio. An optional prepared corpus can be indexed in RAM and reread from its original files on demand when memory is constrained. The performance memory is session-specific: it is not automatically saved as a permanent sound library.

How sonic memory is retrieved

The engine describes each captured phrase using acoustic and temporal features. Candidate sounds in enabled memory corpora are compared on normalised timbre, pitch gesture, rhythmic pattern, amplitude envelope and duration. It can refine matches using time-varying pitch and envelope contours.

Candidate selection includes similarity thresholds, controlled randomness, recall cooldown and exclusion of very recent or closely related sources. The similarity scores describe measured acoustic relationships; they are not a semantic interpretation of a phrase's musical intention.

Phrase-level influence memory

New in v0.4.8, refined in v0.4.9. Retrieval is no longer based exclusively on the last phrase. The recent history of gestures can influence the next memory match. Each older gesture has a smaller weight, and its relevance decreases with elapsed time. This provides short-term contextual memory while preserving the priority of the most recent phrase.

The system uses the phrase's actual ending time to age its influence, rather than the time an analysis thread happens to process it. The Memory Map can show pale-lilac influence arcs whose weights represent contributions to a chosen memory match.

How a new response is chosen

The default Hybrid action weights are 40 : 30 : 30 (Transform : Recall : Recall + Transform). They are conditional weights, not guaranteed playback percentages. By default, corpus source priorities are A 40, B 30, C 30; unavailable corpora do not supply candidates. Corpus-only recall bypasses the transformations and automatic memory layers/background.

Performance presets

The form's Preset menu provides ready-made musical configurations for Continuous mode. The first choice, Custom (form and dialogs), uses the current form/dialog settings. A named preset pre-fills and overrides the associated Sonic Memory and Playback values; tick Memory_settings or Playback_settings to inspect and fine-tune those values before starting. Presets do not replace the selected input device or microphone sensitivity.

Polyphony: responses, layers and background

In Continuous mode, foreground answers can overlap. The mixer also supports lower-priority secondary memory layers and an evolving background, creating an autonomous sonic environment based on previously heard material.

Polyphonic_density controls how often a secondary layer is attempted; eligibility and mixer capacity also matter. Max_voices defaults to four. When resources are crowded, a new high-priority response may fade out an older background or secondary voice. A departing voice may remain audible briefly during its fade.

Layer_shape defaults to Whole sounds: recalled and layered sounds are played in their entirety where possible. The alternative Excerpts (onset -> natural end) starts at an onset earlier in the sound and continues to its natural end. Processed Corpus B retains complete rendered answers up to the playback cap, rather than unconditionally keeping only the first 15 seconds.

Turn off Polyphony for one computer answer at a time; the evolving background is also inactive in that mode. This is useful for a controlled comparison but is not required for normal interaction.

Stereo space, dynamics and level protection

The mixer positions answers independently of the transformation engine. It preserves genuine stereo sources, pans mono material using equal-power placement, and optionally moves voices smoothly over time.

The system applies one gain to each sound to preserve its internal dynamics as far as possible. Gain caps and a peak ceiling still constrain the result. Level matching cannot prevent false phrase detection caused by external microphone noise.

Evolving background texture

With Background_layer enabled, the system can construct low-level textures from its sonic memory. It selects available source material, favours suitable sustained sounds, combines overlapping excerpts or grains and gently filters and moves the result. New textures replace old ones through overlapping fades rather than hard switching.

The default reference level is −33 dBFS (a low active-RMS target when level matching is enabled). The background is reduced while the performer is active or when foreground answers occupy the space; it gradually returns when the foreground becomes less busy. If microphone leakage becomes strong, the background may pause to avoid repeatedly exciting the phrase detector.

From a compositional perspective, Background ON creates an environment with continuity; Background OFF allows silence to remain a more explicit part of the musical dialogue. Both are useful, and neither changes the source sounds stored in memory.

Main Praat interface

Performance and listening controls

Sonic Memory settings

Corpora, similarity and decision weights

Memory limits and influence

Older items can be evicted as memory limits are reached. The limits control the in-session model, not the user's prepared source files. See Phrase-level influence memory for the musical role of the additional context.

Playback settings

All playback gains are applied to in-memory signals; the original Corpus C files are not modified. These controls do not change microphone input sensitivity. Use input gain and Sensitivity_dB for unwanted background-noise triggers.

Advanced performance settings

Live performer display and stopping

New in v0.4.12. The Praat Demo interface deliberately separates the performer's recording state from the computer's response state. Playback or processing does not mean that you must stop, and a background layer is not a new microphone event.

AudioTools Call & Response 0.4.12
MODE: CONTINUOUS   PRESET: Hybrid memory (default)

YOUR PERFORMANCE
RECORDING YOUR PHRASE
Stop playing for 0.6 s when your gesture is complete.
[quiet progress toward phrase end]

SYSTEM RESPONSE
PLAYING: Markov Soundscape Weaver
VOICES: 3/4 (response, memory layer, background)
MEMORY: A 18 | B 12 | C 0
RECALL: Corpus B   MATCH: 0.82 (3 phrases)
ACTION: Recall

DETECTOR: input -28 dB | start -46 | end -50 | floor -55
          quiet 0.2 / 0.6 s

The technical DETECTOR line shows measured input level, onset threshold (start), end threshold (end), estimated background floor and elapsed quiet time. The level meter has threshold marks. A LAST event briefly distinguishes a genuine pause from split at Max_phrase_s (continues), ignored noise or rejected short input.

The independent SYSTEM RESPONSE area shows what is processing or sounding, voice count, memory contents and recent selection. These are descriptions of computer activity, not cues telling the performer when to play.

On some Praat versions a Demo-window click may produce a Mouse click not completely handled message; dismissing that message allows the stop operation to finish. The normal session does not require saving any intermediate Sound objects.

Memory Map visualisation

Introduced in v0.4.6 and extended in v0.4.8-0.4.9. The Memory Map is an arc-based view of the sounds collected or loaded during the session and the relationships among them. When Sonic Memory is on it is the default visualisation; press G to switch to the technical status view. The performer headline remains visible above the map.

The map provides an interpretable picture of retrieval and action. It is a visual account of the memory relationships, not itself a separate sequencing engine or a Factor Oracle. It does not instruct the performer to wait for the displayed links before starting another gesture.

Compositional workflows

Requirements and installation

Place the two application files in the existing AudioTools library:

plugin_AudioTools/
  Vector Chain/
    AudioTools_CallResponse.praat
  py/
    audiotools_call_response.py

Install any missing Python dependencies into the interpreter used by the Praat launcher:

python -m pip install numpy scipy soundfile sounddevice

The launcher first tries its configured Python interpreter and then common system Python commands. When necessary, set the pyCandidate1$ path near the beginning of the Praat script. The praatExe$ setting can identify the executable for the independent, headless rendering process; when empty, the system attempts to find Praat automatically.

The six transformation scripts remain in their regular AudioTools locations; do not move or rename them merely to run Call & Response. The two launcher files must belong to the same installation and remain compatible with one another.

Diagnostics: the Python file has a command-line self-test command for synthetic classification, memory and mixer checks. These tests are useful for installation troubleshooting but do not replace a live sound-card test.

Session files, privacy and cleanup

Most musical memory resides in RAM. However, the system must sometimes exchange audio and job instructions with Praat through files. It creates a private temporary working directory for each session, normally under the operating system's temporary directory, and may write temporary WAVs, job descriptions, status files and optional diagnostic logs.

On an ordinary, successful stop, the engine is designed to close streams, release or terminate its own rendering processes and remove the session directory and its generated contents. If cleanup cannot finish because a process is still active or a file remains locked, the system should report CLEANUP INCOMPLETE rather than pretend that the files were deleted. Recovery at a later launch can remove abandoned sessions after checking that they are no longer active.

Limitations and performance considerations

• Not zero-latency effects: the system analyses completed phrases and may need time for corpus search and headless Praat rendering before a newly transformed response can sound.
• Microphone setup is essential: a detector based primarily on level and background estimation cannot always tell intended music from a loud external noise. Reduce excessive microphone input gain and adjust Sensitivity_dB before performance.
• Quiet or noise-based gestures need testing: a soft crescendo, sustained noisy texture or very quiet tail can resemble changes in background level. The calibrated detector and background-release safeguards improve reliability but do not give perfect musical segmentation.
• Long phrases can be split: Max_phrase_s is a safety boundary, not a musical command to stop. The system seeks a quiet point and continues the remainder when appropriate.
• Memory similarity is acoustic: retrieval estimates resemblance in timbre, contour, rhythm and envelope, not semantic meaning, harmonic function or long-form musical intention.
• Finite computer resources: bounded queues, memory duration and voice count can cause pending items to be omitted during heavy performance load.
• No full echo cancellation: leakage detection estimates playback reaching the microphone; it does not remove or perfectly reject acoustic spill. Headphones or isolated routing are recommended.
• Stereo output: the main mixer has two channels, even though one of its transformation scripts computes eight internal varispeed voices.
• Audible level decisions: role-based matching, peak trims and limiting can change the resulting balance. All these changes act in memory and do not rewrite Corpus C sources.
• No automatic permanent archive: Sonic Memory and generated session files are temporary unless you separately arrange to record the performance.
• Offline self-tests are not a hardware test: confirm input routing, phrase thresholds, actual Praat rendering, monitoring and cleanup on the intended performance machine.

Practical recommendation: start with a short sound-check; establish a reliable balance between quiet musical gestures and background noise. Once the detector behaves well, explore memory weights, influence depth, spatial density and background layering one control at a time.