Photo Sonification — User Guide
A deterministic image-mapping / stochastic-source sonification that scans a Photo from left to right. Each horizontal analysis bin averages the full image height; weighted brightness controls overall amplitude, RGB proportions control the energy balance of three filtered-noise bands, and red/blue balance controls equal-power stereo position.
Scope
Photo Sonification is intentionally a 1-D column-projection sonification of a 2-D Photo. Horizontal image position is preserved as time, while the vertical dimension is deliberately collapsed inside each analyzed horizontal bin.
image X position → time
full-height RGB projection → colour + brightness controls
weighted brightness → overall amplitude, once
normalized RGB proportions → low/mid/high spectral energy balance
red/blue balance → equal-power stereo position
The engine does not claim to preserve full X/Y image geometry. For a sonification where vertical image position is mapped explicitly to pitch, use Percussive Image Sonification.
The image mapping is deterministic. The filtered-noise source is stochastic; use a positive Random seed when the same Photo should produce exactly the same noise realization again.
Quick start
- Select one Photo object in Praat.
- Run
Photo_sonification.praat. - Set Duration, Sample rate, Analysis columns, and the three requested frequency bands.
- Set Master amplitude, Colour pan strength, and optionally a positive Random seed.
- Run the script. The Photo is projected into continuous control trajectories, three independent noise bands are generated and RMS-matched, then the result is mixed and spatialized in stereo.
- The final Sound remains selected as
Photo_Sonification_<Photo name>.
Full-height horizontal-bin projection
The script extracts the red, green, and blue channel matrices from the selected Photo. Each Matrix is converted in place to a two-dimensional integral image.
For every analysis bin, the engine calculates the exact mean R, G, and B values over:
This replaces sparse vertical sampling and center-column lookup with an exact full-height regional average.
Input scale
The extracted RGB matrices are interpreted according to their observed range:
| Observed range | Working reference |
|---|---|
| Approximately 0…1 | Fixed black=0, white=1. |
| Approximately 0…255 | Fixed byte-like fallback: black=0, white=255. |
| Other values | Observed minimum/maximum fallback, reported explicitly in Info. |
This preserves absolute brightness for normal Photo data. A spatially uniform white or gray image is not renormalized to black merely because it contains no internal contrast.
Brightness and RGB spectral balance
Weighted brightness → overall amplitude
Each projected column receives one weighted brightness value:
This is a Rec.709-like perceptual weighting of the extracted RGB values, not a claim of a full color-managed photometric luminance measurement.
Brightness is applied to the audio once:
RGB proportions → spectral balance
R, G, and B do not multiply the signal as three additional brightness envelopes. Instead they determine the energy distribution among the low, mid, and high noise bands:
wG = sqrt(G / colourSum)
wB = sqrt(B / colourSum)
For a non-black column:
This makes RGB mainly a spectral-colour control rather than a second overall level control. A neutral gray column distributes energy among all three bands; a red-dominant column emphasizes the low band, green the mid band, and blue the high band.
For a fully black column all three weights are zero, and luminance is also zero.
Three independent noise bands
The source uses three independent uniform-noise realizations rather than filtering one shared noise buffer three ways.
| Image component | Default band | Audio role |
|---|---|---|
| Red proportion | 100–800 Hz | Low-band energy weight. |
| Green proportion | 800–3000 Hz | Mid-band energy weight. |
| Blue proportion | 3000–9000 Hz | High-band energy weight. |
Each noise source is Hann-band filtered using its effective band edges. Filter smoothing is derived from 20% of the effective band width and limited to 5–100 Hz.
Equal-RMS source normalization
After filtering, each band is independently scaled to the same reference RMS:
This prevents a wider band from becoming louder simply because it contains more noise bandwidth. The RGB weights can therefore act more cleanly as an energy-distribution control.
Image X → continuous time control
One sample per analyzed image bin is stored in low-rate control Sounds for R, G, B, luminance, the three spectral weights, and pan.
Output time is mapped exactly from the first control-sample center to the last:
Praat's time-based object access linearly interpolates between adjacent control samples. Consequently:
- the first analyzed image bin controls the beginning of the Sound;
- the last analyzed image bin controls the end;
- RGB, luminance, spectral weights, and pan evolve continuously between bins rather than jumping in hard steps.
Red / blue balance → stereo pan
The pan mapping uses relative red-versus-blue balance:
balance = (B - R) / denominator pan = 0.5 + 0.48 × ColourPanStrength × balance
The coordinate is limited to .02–.98, with:
- 0 = left
- .5 = center
- 1 = right
Red-dominant regions therefore move left, blue-dominant regions move right, and neutral red/blue balance stays near center.
Stereo gains are equal-power:
rightGain = sqrt(pan)
Controls
| Control | Default / range | Behavior |
|---|---|---|
| Duration | 3 s; >0–180 s | Exact final Sound duration and complete left-to-right scan time. |
| Sample rate | 44.1 kHz; 8–192 kHz | Direct noise synthesis, filtering, and final output rate. |
| Analysis columns | 160; 2–1000 | Requested horizontal projection resolution, capped by Photo width. |
| Low band | 100–800 Hz | Red-associated filtered-noise band. |
| Mid band | 800–3000 Hz | Green-associated filtered-noise band. |
| High band | 3000–9000 Hz | Blue-associated filtered-noise band. |
| Master amplitude | .80; >0–2 | Global gain before luminance modulation and final peak protection. |
| Colour pan strength | .90; 0–1 | Depth of red-left / blue-right stereo displacement. |
| Random seed | 0 | 0 = unpredictable noise; positive = reproducible low/mid/high noise realizations. |
| Peak protection | on | Enables final down-only ceiling at .92. |
Frequency and sampling safety
The script reserves:
It finds the highest requested upper band edge and, if necessary, applies one common scale to all six band edges:
Using one scale preserves the relative spectral geometry among all three requested bands. If this common scaling would push any lower band edge below 20 Hz, the script stops and asks for safer settings.
Output and level
The mono spectral mixture is:
| Property | Behavior |
|---|---|
| Input | One selected Praat Photo. |
| Duration | Exactly the requested Duration. |
| Sample rate | Exactly the selected 8–192 kHz rate. |
| Channels | Stereo. One mono spectral mixture is distributed by continuous equal-power pan. |
| Randomness | Only the three noise sources are stochastic; the Photo-to-control mapping is deterministic. |
| Edge fade | Common 20 ms linear fade-in/out, capped at 10% of Duration. |
| Peak protection | If enabled and the final stereo peak exceeds .92, the complete Sound is scaled down once to .92. |
| Normalization | No unconditional final normalization and no upward scaling. If peak ≤ .92, the generated level is preserved. |
| Object name | Photo_Sonification_<Photo name>. |
Visualization and QC
The current visualization is built from the actual extracted controls and realized audio.
| Panel | What it shows |
|---|---|
| A — Actual Column Projection | Full-height R, G, B means and weighted brightness through the scan. |
| B — Audio Control | The three sqrt-energy spectral weights plus the red-left / blue-right pan coordinate. |
| C — Model → Measurement | Measured spectrogram of the realized mono spectral mixture, with the effective low/mid/high band boundaries overlaid. |
| D — Brightness Validation | Target weighted brightness versus short-time RMS of the realized mono mixture. The measured RMS curve is shape-normalized for comparison rather than shown as an absolute level. |
The summary block reports the intentional 1-D scope, realized brightness range, mean pan, effective band geometry and common frequency scale, pre-protection peak/RMS, Random seed status, and whether down-only peak protection was applied.