Percussive Image Sonification — User Guide

Deterministic parameter-mapping sonification of a selected Praat Photo. Horizontal position maps to fixed scan time, vertical position to logarithmic pitch, luminance to event strength, local contrast to upper-partial attack content, and red/blue balance to equal-power stereo position.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 0.4.3 (2026) License: MIT License Repo: Praat AudioTools

Contents:

Scope Quick start Image grid Data → sound mapping Activity & events Percussive source Controls Sampling safety Output & level Visualization Further reading

Scope

This script performs parameter-mapping sonification. It does not recognize objects, classify image content, infer semantics, or attempt to turn a photograph into a literal acoustic scene.

The mapping is intentionally explicit and inspectable:

image X position → fixed scan time
image vertical band → logarithmic pitch
perceptual luminance → event strength
local 2-D luminance contrast → upper-partial attack content
red/blue chromatic balance → equal-power stereo position

The same Photo and the same settings produce the same event map and the same Sound. No stochastic source is used in synthesis.

Version 0.4.3 uses the Photo's RGB channel values directly on their 0–1 scale. It does not renormalize every image to its own darkest and brightest pixel. A uniform white image therefore remains bright, a 50% gray image remains mid-level, and a uniform black image remains dark.

Quick start

Select one Photo object in Praat.
Run Percussive_Image_Sonification.praat.
Set Duration, Sample rate, Horizontal scan bins, Vertical bands, pitch range, Hit duration, Activity threshold, Edge emphasis, Colour pan strength, and Master amplitude.
Use Invert vertical pitch if the image's upper region should map to the lower register instead of the higher register.
Run the script. The Photo is reduced to a 2-D analysis grid, features are extracted, active cells are rendered into stereo, and the result remains selected in Praat.

The requested number of horizontal/vertical divisions is automatically limited by the actual Photo dimensions: the script never creates more X bins than image columns or more vertical bands than image rows.

2-D image reduction

The Photo is divided into a regular grid of:

Horizontal scan bins × Vertical bands

Each cell represents a rectangular image region, not a single pixel. The script extracts the red, green, and blue Photo channels and converts each channel Matrix into a 2-D integral image. This allows the mean R, G, and B values of each cell to be calculated efficiently from four integral-image corners.

Absolute RGB scale

For each cell, the mean RGB values are safety-clamped to 0–1 and used directly:

R = clamp(meanRed, 0, 1)
G = clamp(meanGreen, 0, 1)
B = clamp(meanBlue, 0, 1)

The minimum and maximum raw channel values of the Photo are still reported in the Info window, but they are diagnostics only and do not rescale the sonification.

Luminance

Cell luminance is calculated with the standard weighted RGB coefficients:

Y = 0.2126R + 0.7152G + 0.0722B

This is used as a transparent perceptual weighting of the Photo channels. The script does not claim a full color-managed photometric conversion.

Data → sound mapping

X position → fixed time

Horizontal position determines onset time directly. Brightness does not speed up or slow down the scan.

scanSpan = Duration - HitDuration
scanStep = scanSpan / (numberOfXBins - 1)

The first X bin starts at time 0 and the last starts at Duration - HitDuration. If there is only one X bin, it starts at 0.

This fixed X→time relationship is a central change from older versions. The image is scanned once from left to right; columns do not wrap or repeat merely because the requested Duration is long.

Vertical position → logarithmic pitch

Each vertical band maps to one percussion register:

f = effectiveMinPitch × (effectiveMaxPitch / effectiveMinPitch)^verticalPosition

With the default orientation, lower-numbered bands map from Minimum pitch toward Maximum pitch. Invert vertical pitch reverses that mapping.

Red / blue balance → pan

Stereo position uses the relative red-versus-blue balance rather than absolute brightness:

rbDen = max(0.08, R + B)
colourBalance = (B - R) / rbDen
pan = 0.5 + 0.48 × ColourPanStrength × colourBalance

The pan coordinate is then limited to .02–.98. Red-dominant cells move left, neutral R/B balance stays near center, and blue-dominant cells move right.

Stereo gains are equal-power:

leftGain = sqrt(1 - pan)
rightGain = sqrt(pan)

Local contrast, activity, and audible cells

Coarse 2-D contrast

For each grid cell, the script compares luminance with the previous horizontal cell and the previous vertical cell:

edgeRaw = sqrt( ΔX(luminance)² + ΔY(luminance)² )

This is a coarse grid-level contrast descriptor. It is not a pixel-accurate computer-vision edge detector.

All raw edge magnitudes are divided by the maximum edge magnitude in the current analysis grid, producing a normalized edge value from 0 to 1. Unlike luminance, this edge normalization is intentionally relative to the image because it represents contrast within the current image.

Activity

A cell's event activity is:

activity = max( luminance, EdgeEmphasis × normalizedEdge )

The cell becomes audible only when:

activity ≥ ActivityThreshold

This allows a dark region with a strong local boundary to remain audible when Edge emphasis is sufficiently high. A completely black, featureless image remains silent.

What edge emphasis does not do

Edge magnitude does not change X→time and does not move an event's pitch. It affects whether a cell crosses the activity threshold and increases the strength of the upper inharmonic partials in the event source.

Percussive event source

Every active cell produces one deterministic short inharmonic hit. The source contains four sinusoidal components:

1.00 × f
0.38 × 1.73f
(0.10 + 0.28 × edge) × 2.37f
(0.03 + 0.19 × edge) × 3.11f

The upper two partials therefore become stronger as local contrast increases. Fixed phase offsets are used for the inharmonic components, and the complete source receives an approximate energy normalization before event gain.

Envelope

The event envelope combines a fast exponential rise, exponential decay, and cosine terminal taper:

attack = min(3 ms, 18% of realized hit duration)

The complete requested Hit duration is retained unless the final time boundary of the Sound truncates it; by construction, the last scan onset is placed early enough to fit the requested hit duration.

Strength and simultaneous-band compensation

Event amplitude is:

amplitude = MasterAmplitude × activity × 1 / sqrt(activeBandsInThisXBin)

If several vertical bands fire at the same horizontal scan position, the 1/sqrt(N) factor reduces their combined energy. This prevents image height/density from acting as an unintended master-volume control.

Controls

Control	Default / range	Behavior
Duration	4 s; >0–180 s	Exact stereo output duration and total left-to-right scan span.
Sample rate	44.1 kHz; 8–192 kHz	Direct synthesis and final output rate.
Horizontal scan bins	72; 1–256	Number of temporal image divisions, limited by actual image width.
Vertical bands	12; 1–32	Number of pitch/register divisions, limited by actual image height.
Minimum pitch	180 Hz	Lower requested register boundary before common sampling-headroom scaling.
Maximum pitch	3600 Hz	Upper requested register boundary before common sampling-headroom scaling.
Hit duration	.055 s	Length of each active-cell percussion hit; must not exceed total Duration.
Activity threshold	.14; 0–1	Minimum combined luminance/edge activity required for an audible event.
Edge emphasis	.65; 0–1	Amount of normalized contrast entering the activity trigger and attack-spectrum weighting.
Colour pan strength	.90; 0–1	Depth of red-left / blue-right stereo displacement.
Master amplitude	.58; >0–2	Global event gain before polyphony compensation and final peak protection.
Invert vertical pitch	off	Reverses the mapping between vertical band and logarithmic pitch.

Frequency and sampling safety

The highest source component is 3.11 times the event fundamental. The script reserves:

safeTop = 0.45 × Fs

and calculates one common scale for the complete requested pitch range:

frequencyScale = min( 1, safeTop / (3.11 × MaximumPitch) )

Minimum and maximum pitch are both multiplied by that same scale, preserving their ratio and the logarithmic vertical mapping. If the resulting effective minimum pitch would fall below 20 Hz, the script stops instead of distorting the mapping further.

Output and level

Property	Behavior
Input	One selected Praat Photo.
Duration	Exactly the requested Duration.
Sample rate	Exactly the selected 8–192 kHz rate.
Channels	Always stereo.
Randomness	None in the sonification or DSP. Temporary object naming does not affect the sound.
Common fade	Linear 15 ms fade-in/out, capped at 10% of total Duration.
Silent output	Valid when no cell reaches Activity threshold; the Info window reports this explicitly.
Peak protection	If enabled and the mixed peak exceeds .92, the complete stereo Sound is scaled down once to .92.
Normalization	No upward normalization. If peak ≤ .92, the generated level is preserved.
Object name	`Percussive_Image_<temporary ID>`.

Visualization and QC

The visualization is built from the same extracted features and rendered events used by the sonification.

Panel	What it shows
A — Binned Image Data	The cell-grid luminance values actually used by the event mapper.
B — Actual Audible Events	Event time versus logarithmic pitch. Marker color visualizes pan: redder = left, bluer = right. Marker size follows event activity.
C — X-Scan Descriptors	Mean luminance, normalized contrast, and mean pan for each horizontal scan bin.
D — Model → Measurement	Measured spectrogram of the higher-RMS stereo channel with sampled actual event fundamentals overlaid as short guides.

The QC area reports the grid size, fixed X→time mapping, requested/effective pitch range, common frequency scale, mean luminance and edge value, audible-cell fraction, number of active X bins, mean pan coordinate, pre/final peak and RMS, and whether down-only peak protection was applied.