Wave Gesture Path Performer — Wearable Timbral-Path Performance

What this does

Wave Gesture Path Performer transforms a folder of audio files into a performable timbral space. Rather than assigning gestures directly to arbitrary file numbers, the script first analyses the corpus with MFCCs and constructs a nearest-neighbour path through timbrally related sounds. A recorded Wave-ring gesture then moves through that ordered path and controls how successive voices are selected and shaped.

Key Features:

• Wearable gesture control — Genki Wave ring via BLE-MIDI.
• MFCC timbral ordering — the corpus is reorganised as a nearest-neighbour path before performance.
• Three independent gesture streams — Tilt, Pan, and Roll.
• Global + local navigation — Tilt selects a location on the path; Pan scrubs locally around that location.
• Dynamic amplitude control — Roll controls the level of each triggered voice.
• Polyphonic overlap-add rendering — triggered sounds can overlap instead of cutting each other off.
• Auto-fit or absolute Tilt mapping — use the full corpus or retain a fixed physical 0-1 mapping.
• Multi-format corpus loading — WAV, AIFF/AIF/AIFC, FLAC, and MP3.
• MIDI activity diagnostics — the Python helper reports the CCs, ranges, counts, and channels actually received.
• Explanatory Praat Picture visualisation — raw gesture, mapped corpus path, and rendered waveform in one display.
• Offline rendering — the gesture is captured in real time, but the final audio is rendered reproducibly in Praat after the take.

Quick start

1. Pair the Genki Wave ring as a BLE-MIDI device and make sure it appears as a MIDI input port.
2. Install the Python MIDI dependencies: mido and python-rtmidi.
3. Place wave_capture.py in plugin_AudioTools/py/.
4. Run Wave_Gesture_Path_Performer.praat.
5. Choose a folder containing at least two supported audio files.
6. For a first run, keep Tilt mapping = Auto-fit take to full corpus, Grain seconds = 0.25, Voice seconds = 1.5, and Pan scrub amount = 4.
7. Click Start. After the countdown, move the ring until the low end-beep.
8. The rendered take appears in the Praat Objects list as Wave_Take_1, Wave_Take_2, and so on.

How it works

1. The corpus is loaded and normalised

The script scans the selected folder for supported audio files, loads each file as a Praat Sound, converts multichannel files to mono, and resamples all successfully loaded sounds to the sample rate of the first valid corpus item.

2. MFCC mean vectors describe timbral similarity

Each sound is analysed with 12 MFCC coefficients using a 15 ms analysis window and 5 ms time step. The mean value of each coefficient across time becomes the sound's compact timbral descriptor.

3. A nearest-neighbour timbral path is constructed

Euclidean distances are calculated between the MFCC mean vectors. Starting from the first loaded sound, the script repeatedly chooses the nearest unvisited sound until every corpus item has been placed on a single ordered path.

4. Python captures the wearable gesture

Praat launches wave_capture.py, which opens the Wave MIDI port, records the configured CC streams at approximately 100 Hz, and writes a temporary tab-separated gesture file containing time, tilt, pan, and roll.

5. Praat renders the take

At each trigger time, the gesture is sampled, mapped to a corpus-path position, and used to select a source sound. A Hanning-windowed excerpt is extracted and summed into an output buffer. Overlapping excerpts form a polyphonic texture whose trajectory is determined by the recorded hand movement.

Gesture mapping

Controls

Tilt mapping

Corpus loading

Version 1.8 explicitly validates the corpus before any gesture is recorded.

Visualisation

After each take, the script produces a Praat Picture display organised as a compact explanation of the performance:

MIDI diagnostics

The Python helper records all incoming MIDI activity during each take, not only the three configured control streams. The Info window therefore shows which CC numbers actually arrived, their value ranges, message counts, and MIDI channels.

Detected CC activity (this take):
  CC 2:   25-126   (357 messages, ch 1)  <- Pan
  CC 1:    0-126   (317 messages, ch 1)  <- Tilt
  CC 3:    0-126   (239 messages, ch 1)  <- Roll

If a configured control shows too few messages or too little movement, the helper issues a warning and reports other active CCs. It does not remap controls automatically; the user remains in control of the Wave / Softwave configuration.

Technical behavior

• Uses 12 MFCC coefficients per corpus sound, averaged across analysis frames.
• Constructs a greedy nearest-neighbour path using Euclidean distance in MFCC mean-vector space.
• Captures Wave CC data at approximately 100 Hz through mido / python-rtmidi.
• Python-side MIDI port auto-detection prefers names containing Wave, Genki, Bluetooth, or BLE; an exact or partial port override can be set in the script.
• At each trigger, the nearest captured gesture sample to the trigger centre is used.
• Tilt determines the base corpus-path position; Pan adds a bounded integer scrub offset; Roll scales amplitude.
• Voice excerpts use Hanning windows and are summed into a mono output buffer.
• If the voice pool is full, the oldest active voice is stolen and a short taper is applied around the steal point.
• Source excerpt position moves through each chosen file according to performance time rather than always starting at the beginning.
• The rendered take is trimmed for trailing silence, peak-scaled to 0.99, and retained as a Praat Sound object.
• Temporary CSV, log, and done-sentinel files are cleaned automatically after use.
• The original corpus Sounds are removed from the Objects list after the session; rendered Wave_Take_N objects remain.

Requirements & installation

Limitations

• Top-level folders only: corpus files inside subfolders are not searched.
• Mono rendering: corpus Sounds are converted to mono and the generated take is mono.
• Path quality depends on the descriptor: the MFCC mean-vector path represents broad timbral similarity, not every perceptually relevant property of the source sounds.
• Greedy ordering: the nearest-neighbour path is locally constructed and is not a globally optimal travelling path through the corpus.
• Auto-fit is take-relative: the same physical Tilt can map to different corpus positions in different takes because each take's robust range is fitted independently.
• Pan is intentionally local: Pan scrub cannot replace global Tilt navigation; its range is limited by Pan scrub amount.
• Gesture data depends on MIDI configuration: a silent or incorrectly configured CC stream cannot be reconstructed by the Praat renderer.
• Dense triggering can mask corpus identity: short grain intervals and long overlapping voices can produce blended textures in which individual source changes become less obvious.

Outputs

Each completed take remains in the Praat Objects list as:

Wave_Take_1
Wave_Take_2
Wave_Take_3
...

The script does not save an audio file automatically. The user can audition, inspect, rename, edit, or save any take from Praat after the session.

Applications

Embodied corpus navigation

Use case: perform a corpus by moving through a timbrally ordered path rather than selecting files manually or triggering fixed pads.

Gestural timbre improvisation

Use case: use broad Tilt motion for structural traversal while Pan creates local detours and Roll shapes articulation and dynamics.

Performance-derived offline composition

Use case: record physically expressive takes, then treat the resulting Wave_Take_N objects as fixed compositional material for further Praat AudioTools processing.

Timbral-path exploration

Use case: use the body as an exploratory interface for hearing how an MFCC similarity ordering behaves across heterogeneous instrumental, environmental, percussive, or textural corpora.

Research on embodied sound control

Use case: study the relationship between wearable gesture, corpus-space navigation, and rendered sound while retaining the captured MIDI trajectory and an explicit visual account of the mapping.