Photo Sonification — User Guide

A deterministic image-mapping / stochastic-source sonification that scans a Photo from left to right. Each horizontal analysis bin averages the full image height; weighted brightness controls overall amplitude, RGB proportions control the energy balance of three filtered-noise bands, and red/blue balance controls equal-power stereo position.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 0.5 (2026) License: MIT License Repo: Praat AudioTools
Contents:

Scope

Photo Sonification is intentionally a 1-D column-projection sonification of a 2-D Photo. Horizontal image position is preserved as time, while the vertical dimension is deliberately collapsed inside each analyzed horizontal bin.

Mapping:
image X position → time
full-height RGB projection → colour + brightness controls
weighted brightness → overall amplitude, once
normalized RGB proportions → low/mid/high spectral energy balance
red/blue balance → equal-power stereo position

The engine does not claim to preserve full X/Y image geometry. For a sonification where vertical image position is mapped explicitly to pitch, use Percussive Image Sonification.

The image mapping is deterministic. The filtered-noise source is stochastic; use a positive Random seed when the same Photo should produce exactly the same noise realization again.

Quick start

  1. Select one Photo object in Praat.
  2. Run Photo_sonification.praat.
  3. Set Duration, Sample rate, Analysis columns, and the three requested frequency bands.
  4. Set Master amplitude, Colour pan strength, and optionally a positive Random seed.
  5. Run the script. The Photo is projected into continuous control trajectories, three independent noise bands are generated and RMS-matched, then the result is mixed and spatialized in stereo.
  6. The final Sound remains selected as Photo_Sonification_<Photo name>.
Analysis columns is automatically capped by the Photo's actual width. Every realized analysis bin averages all image rows over that bin's horizontal extent.

Full-height horizontal-bin projection

The script extracts the red, green, and blue channel matrices from the selected Photo. Each Matrix is converted in place to a two-dimensional integral image.

For every analysis bin, the engine calculates the exact mean R, G, and B values over:

all image rows × all pixel columns belonging to that X bin

This replaces sparse vertical sampling and center-column lookup with an exact full-height regional average.

Input scale

The extracted RGB matrices are interpreted according to their observed range:

Observed rangeWorking reference
Approximately 0…1Fixed black=0, white=1.
Approximately 0…255Fixed byte-like fallback: black=0, white=255.
Other valuesObserved minimum/maximum fallback, reported explicitly in Info.

This preserves absolute brightness for normal Photo data. A spatially uniform white or gray image is not renormalized to black merely because it contains no internal contrast.

Brightness and RGB spectral balance

Weighted brightness → overall amplitude

Each projected column receives one weighted brightness value:

L = 0.2126R + 0.7152G + 0.0722B

This is a Rec.709-like perceptual weighting of the extracted RGB values, not a claim of a full color-managed photometric luminance measurement.

Brightness is applied to the audio once:

output amplitude ∝ MasterAmplitude × L

RGB proportions → spectral balance

R, G, and B do not multiply the signal as three additional brightness envelopes. Instead they determine the energy distribution among the low, mid, and high noise bands:

colourSum = R + G + B wR = sqrt(R / colourSum)
wG = sqrt(G / colourSum)
wB = sqrt(B / colourSum)

For a non-black column:

wR² + wG² + wB² = 1

This makes RGB mainly a spectral-colour control rather than a second overall level control. A neutral gray column distributes energy among all three bands; a red-dominant column emphasizes the low band, green the mid band, and blue the high band.

For a fully black column all three weights are zero, and luminance is also zero.

Three independent noise bands

The source uses three independent uniform-noise realizations rather than filtering one shared noise buffer three ways.

Image componentDefault bandAudio role
Red proportion100–800 HzLow-band energy weight.
Green proportion800–3000 HzMid-band energy weight.
Blue proportion3000–9000 HzHigh-band energy weight.

Each noise source is Hann-band filtered using its effective band edges. Filter smoothing is derived from 20% of the effective band width and limited to 5–100 Hz.

Equal-RMS source normalization

After filtering, each band is independently scaled to the same reference RMS:

bandReferenceRMS = 0.18

This prevents a wider band from becoming louder simply because it contains more noise bandwidth. The RGB weights can therefore act more cleanly as an energy-distribution control.

The three user-defined bands are validated individually, but they are not required to be non-overlapping or contiguous. Overlapping bands are therefore possible if intentionally requested.

Image X → continuous time control

One sample per analyzed image bin is stored in low-rate control Sounds for R, G, B, luminance, the three spectral weights, and pan.

Output time is mapped exactly from the first control-sample center to the last:

controlTime = 0.5 + x × (analysisColumns - 1) / Duration

Praat's time-based object access linearly interpolates between adjacent control samples. Consequently:

Red / blue balance → stereo pan

The pan mapping uses relative red-versus-blue balance:

denominator = max(0.08, R + B)
balance = (B - R) / denominator pan = 0.5 + 0.48 × ColourPanStrength × balance

The coordinate is limited to .02–.98, with:

Red-dominant regions therefore move left, blue-dominant regions move right, and neutral red/blue balance stays near center.

Stereo gains are equal-power:

leftGain = sqrt(1 - pan)
rightGain = sqrt(pan)

Controls

ControlDefault / rangeBehavior
Duration3 s; >0–180 sExact final Sound duration and complete left-to-right scan time.
Sample rate44.1 kHz; 8–192 kHzDirect noise synthesis, filtering, and final output rate.
Analysis columns160; 2–1000Requested horizontal projection resolution, capped by Photo width.
Low band100–800 HzRed-associated filtered-noise band.
Mid band800–3000 HzGreen-associated filtered-noise band.
High band3000–9000 HzBlue-associated filtered-noise band.
Master amplitude.80; >0–2Global gain before luminance modulation and final peak protection.
Colour pan strength.90; 0–1Depth of red-left / blue-right stereo displacement.
Random seed00 = unpredictable noise; positive = reproducible low/mid/high noise realizations.
Peak protectiononEnables final down-only ceiling at .92.

Frequency and sampling safety

The script reserves:

safeTop = 0.45 × Fs

It finds the highest requested upper band edge and, if necessary, applies one common scale to all six band edges:

requestedTop = max(lowMax, midMax, highMax) frequencyScale = min(1, safeTop / requestedTop)

Using one scale preserves the relative spectral geometry among all three requested bands. If this common scaling would push any lower band edge below 20 Hz, the script stops and asks for safer settings.

Output and level

The mono spectral mixture is:

mono = MasterAmplitude × luminance × ( wR × lowNoise + wG × midNoise + wB × highNoise )
PropertyBehavior
InputOne selected Praat Photo.
DurationExactly the requested Duration.
Sample rateExactly the selected 8–192 kHz rate.
ChannelsStereo. One mono spectral mixture is distributed by continuous equal-power pan.
RandomnessOnly the three noise sources are stochastic; the Photo-to-control mapping is deterministic.
Edge fadeCommon 20 ms linear fade-in/out, capped at 10% of Duration.
Peak protectionIf enabled and the final stereo peak exceeds .92, the complete Sound is scaled down once to .92.
NormalizationNo unconditional final normalization and no upward scaling. If peak ≤ .92, the generated level is preserved.
Object namePhoto_Sonification_<Photo name>.

Visualization and QC

The current visualization is built from the actual extracted controls and realized audio.

PanelWhat it shows
A — Actual Column ProjectionFull-height R, G, B means and weighted brightness through the scan.
B — Audio ControlThe three sqrt-energy spectral weights plus the red-left / blue-right pan coordinate.
C — Model → MeasurementMeasured spectrogram of the realized mono spectral mixture, with the effective low/mid/high band boundaries overlaid.
D — Brightness ValidationTarget weighted brightness versus short-time RMS of the realized mono mixture. The measured RMS curve is shape-normalized for comparison rather than shown as an absolute level.

The summary block reports the intentional 1-D scope, realized brightness range, mean pan, effective band geometry and common frequency scale, pre-protection peak/RMS, Random seed status, and whether down-only peak protection was applied.