Ikeda-inspired Audiovisual Studies — User Guide

Four compact Praat audiovisual studies inspired by selected principles in Ryoji Ikeda's work. Each study derives sound and image from a shared data or control representation and uses periodic hard re-synchronization to keep audiovisual timing error bounded.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 2.1.3 (2026) License: MIT License Repo: Praat AudioTools
Contents:

Scope

This script does not reconstruct, emulate, or model Ryoji Ikeda's works at sample level. It creates four small audiovisual studies that investigate selected ideas associated with test pattern, datamatics, spectra, and supercodex.

Governing relation:
shared data/control object → audio mapping
shared data/control object → visual mapping

The important relationship is therefore not merely “a sound happens when an image happens.” In three of the four studies, the same generated data directly determines both domains. In the spectra-inspired study, one analytic energy control drives both the sound field and the visual beam.

Historical caution: the labels are intentionally “-inspired.” Praat's Demo window cannot reproduce the display rates, physical scale, multi-channel systems, luminous intensity, architecture, or site-specific conditions of Ikeda's original projects.

Quick start

  1. Run Ikeda_audiovisual.praat. No input Sound is required.
  2. Choose one of the four Studies.
  3. Set Duration, Audio sample rate, Visual frame rate, and Sync block duration.
  4. Optionally choose a fixed Random seed, adjust Master volume, enable color accents, or enable High flicker mode.
  5. Run the script. The full stereo master Sound is generated first, then divided into frame-aligned synchronization blocks for audiovisual playback.
  6. After playback, the generated master Sound remains in Praat's Objects window for analysis, reuse, or saving.

Controls

ControlRange / defaultBehavior
Study4 choicesSelects the shared-data mapping and visual grammar.
Duration0–120 s; default 16 sExact master-Sound duration. The visual frame count uses ceiling(Duration × fps), so the final partial frame is retained.
Audio sample rate16–192 kHz; default 44.1 kHzDirect synthesis rate of the retained stereo master Sound.
Visual frame rate4–20 Hz; default 8 HzTarget Demo-window update rate. The frame duration is 1 / frame rate.
Sync block0.20–2.0 s; default 0.5 sRequested duration of each periodic hard-sync interval. It is aligned to an integer number of visual frames.
Random seed0 or positive integer0 = unpredictable. Positive values reproduce generated binary words and numerical-grid data. The spectra-inspired audio/beam trajectory itself is deterministic.
Master volume>0 to 2; default 0.75Multiplies the completed master Sound before down-only peak protection.
Enable color accentson/offAffects only selected visual accents; it does not change the audio.
High flicker modeon/offRemoves the default multi-frame word hold in the two binary studies.

Frame-aligned sync duration

The requested Sync block is converted to an integer number of frames:

framesPerSyncBlock = round(Sync block × frame rate) alignedSyncBlock = framesPerSyncBlock / frame rate

For the defaults, 0.5 s × 8 fps = 4 frames, so the aligned block remains exactly 0.5 s.

Audiovisual synchronization

Version 2.1.3 uses the hybrid synchronization architecture introduced in v2.1.2. The complete master audio is generated first and then pre-sliced into frame-aligned sync blocks. Playback is asynchronous inside each block, but a new block creates a periodic hard re-lock.

for each sync block: start asynchronous playback of block k reset local stopwatch pace due visual frames against the local clock if a visual frame is already badly late: skip that stale frame at next block: start new block playback = hard re-lock

This is a compromise between two earlier approaches:

With the hybrid method, timing error can develop only inside the current sync block. It is reset when the next block begins.

Late-frame policy

If a frame is more than 80% of one frame period late, the stale visual frame is skipped so the image can catch up to the audio clock. A frame more than 25% late but still rendered is counted as “late.” Both counts are reported at the end.

After the final sync block, the final data frame is held for a short latency guard of approximately half a frame, bounded to 40–150 ms, to absorb a small output-device tail.

The four studies

1. test pattern-inspired Binary Transduction

A generated 16-bit integer is the shared data object. The same word determines the barcode and several audio properties.

Data featureAudio mappingVisual mapping
16 individual bitsIndirectly contribute to Hamming weight and transition count.Sixteen vertical barcode cells: white for 1, dark for 0.
Number of 1-bitsPitch: 1800 + 570 × HammingWeight, limited by 0.45 × Fs.Reported in the HUD.
Bit-to-bit transitionsControls event amplitude and tone/noise mixture.Reported in the HUD.
Normalized 16-bit valueEqual-power stereo pan from 0.05 to 0.95 and starting phase.The complete word is displayed as the barcode state.

Each frame produces a short event of up to 42 ms with a Hann envelope. The source combines a sine tone and Gaussian noise using equal-power tone/noise gains.

Ikeda's original test pattern operates at a far more extreme technological and perceptual scale. This study keeps the central data-transduction principle, not the original system's frame rates or installation geometry.

2. datamatics-inspired Data Scan

A seeded 12×12 matrix of random values is generated once. The scan advances one cell per visual frame and the active cell value drives both audio and image.

Grid propertyMapping
Cell value, 0–1Logarithmic pitch map from approximately 180 Hz toward 8 kHz before sampling-headroom limiting; also controls amplitude, event duration, and starting phase.
Grid columnEqual-power stereo pan from 0.05 to 0.95.
Full 12×12 fieldDisplayed as grayscale cells, with the scanned cell outlined and a vertical scan marker.

Each scanned cell generates one short Hann-windowed sine event. The scan wraps after all 144 cells.

3. spectra-inspired White-Light Field

This mode is deliberately an abstraction of spectra, not a screen reconstruction of a large-scale white-light installation. A single slow energy trajectory is shared by sound and image:

energy(t) = 0.5 + 0.5 × sin(2π × 0.055 × t) audio gain = 0.16 + 0.34 × energy(t) beam half-width = 3 + 43 × energy(t) beam brightness = 0.20 + 0.80 × energy(t)

The stereo audio is intentionally minimal. The left channel contains 50 and 50.8 Hz components; the right contains 50.3 and 51.1 Hz components. Each side also carries a weak high-frequency sine component.

v2.1.3 high-frequency headroom fix

The nominal high-frequency pair is 9800 / 9813 Hz. At low output sample rates those frequencies could exceed safe sampling headroom, so v2.1.3 applies one common scale:

safeTop = 0.45 × sample rate hfScale = min(1, safeTop / 9813) left HF = 9800 × hfScale right HF = 9813 × hfScale

This preserves the pair's relative frequency geometry while avoiding aliasing from the previously fixed 9.8-kHz components. At the default 44.1-kHz sample rate, no scaling is needed.

4. supercodex-inspired Data Pulse Field

A 16-bit word is decomposed into sixteen ordered micro-events within each visual frame. The visual left-to-right barcode order is exactly the temporal order of those audio micro-events.

Bit valueFrequencyAmplitude
0800 + 310 × bitIndex, limited by safeTop0.022
11600 + 620 × bitIndex, limited by safeTop0.070

Both zeros and ones are sonified, so the entire word is represented rather than only its set bits. Bit index also determines equal-power pan from 0.03 to 0.97. Each micro-event has a Hann envelope and a maximum duration of 1.8 ms or 60% of its sub-slot, whichever is smaller.

The script intentionally does not describe supercodex as a “genome barcode” work. The study uses its own 16-bit data-pulse mapping as an AudioTools compositional design.

Color accents and flicker

Enable color accents

This control does not globally recolor the visualization. Its current effects are specific:

High flicker mode

Only the two binary studies use the shared-word hold. With High flicker off:

holdFrames = round(frameRate / 4)

At the default 8 fps, one binary word is therefore held for 2 frames, giving about four binary state changes per second. High flicker mode sets holdFrames = 1, allowing a new word every frame.

High flicker increases visual change rate but still does not approach the hundreds-of-frames-per-second conditions documented for some versions of Ikeda's test pattern.

Audio output and level

PropertyBehavior
InputNo input Sound is used.
Output channelsAlways stereo.
DurationExactly the requested Duration.
Sample rateSelected 16–192 kHz rate; synthesis occurs directly at this rate.
Master volumeApplied once to the complete generated stereo master.
Peak protectionAutomatic down-only protection: if the post-volume peak exceeds 0.98, the complete master is scaled down to 0.98.
NormalizationNo upward normalization. If peak ≤ 0.98, the user-selected level is preserved.
Output objectIkedaStudy_<study name>, retained in Praat after audiovisual playback.

Synchronization and output QC

The Info window reports the conceptual scope, duration, sample rate, target frame rate, requested and aligned sync-block durations, number of frames and blocks, binary hold length, seed, Master volume, and pre/post-protection peak/RMS.

After playback it also reports:

These values are diagnostic measurements of the practical Praat playback loop. They do not imply sample-accurate audiovisual synchronization.

Official project references

The conceptual descriptions above are intentionally limited to claims supported by Ryoji Ikeda's official site: