Self-Similarity Matrix — User Guide

Computes a raw frame-by-frame audio self-similarity matrix from one of seven feature representations, then draws a contrast-enhanced visualization for exploring repetition, local continuity, texture, pitch-class recurrence, and larger-scale structure.

Author: Shai Cohen Affiliation: Department of Music, Bar-Ilan University, Israel Version: 1.5.0 (2026) Category: Music Information Retrieval / Audio Analysis License: MIT License Repo: Praat AudioTools

Contents:

What this does Quick start Feature types Similarity rules Reading the matrix Performance Parameters Applications

What this does

The script takes exactly one Sound object, extracts a frame-wise feature representation, and creates a square matrix in which cell (i,j) measures the similarity between frame i and frame j. The output object is named originalName_SSM_featureName.

Two outputs are intentionally separated. The Matrix object contains the raw similarity values. Auto contrast, gamma, and the selected colour scheme are applied only to a temporary display copy used for the Picture-window rendering. They do not alter the returned Matrix.

Current feature choices:

Pitch (fast) — local similarity in semitone space.
Pitch + Intensity — weighted pitch and level similarity.
MFCC (recommended) — spectral-envelope / timbral comparison.
Spectral Entropy — scalar spectral-disorder comparison.
LPC coefficients — all-pole spectral-envelope representation.
Mel filterbank — 40-band triangular mel representation.
Chroma (12 pitch classes) — octave-folded pitch-class content.

Multichannel input: the script does not average channels, because opposite-phase material could cancel. It analyzes the channel with the strongest full-duration RMS and reports that channel in the Info window.

Quick start

Select exactly one Sound in Praat.
Run Self-Similarity_Matrix_Calculator.praat.
Choose a feature. MFCC (recommended) is the default.
Leave Time_step = 0.01 s and Frame_skip = 1 for a first run.
Choose a colour scheme. The default is Heat.
Keep Auto_contrast enabled and Gamma = 1.0 unless you specifically want to reshape display brightness.
Run the script. It prints the requested and effective frame skip, effective time step, and final matrix size.

Adaptive frame cap: the SSM is quadratic in frame count. If the requested Frame_skip would produce more than 2000 frames, the script automatically increases the effective skip so the returned SSM remains at or below approximately 2000 × 2000 cells. This safety behavior is part of the current implementation.

Feature Types

1. Pitch (fast)

Melodic similarity

Praat pitch estimates are converted from Hz to a MIDI-like semitone coordinate:

p = 69 + 12 · log2(F0 / 440)

Unvoiced frames are stored as zero. Pitch similarity is not a cosine measure: it uses a Gaussian kernel with σ = 1.5 semitones. This makes similarity local in musical pitch space rather than dependent on the overall pitch range of the recording.

Best suited to: monophonic or clearly pitched material where melodic recurrence is the main question.

2. Pitch + Intensity

Melody plus dynamics

Pitch and Intensity are computed as separate Praat analysis objects and aligned by physical time, not merely by assuming matching frame numbers.

If both frames are voiced: S = 0.75 · K_pitch + 0.25 · K_intensity K_pitch = exp[-0.5 · (Δsemitones / 1.5)^2] K_intensity = exp[-0.5 · (ΔdB / 12)^2] If either frame is unvoiced: S = 0.25 · K_intensity

The representation therefore gives pitch the larger weight while still preserving dynamic similarity.

3. MFCC (recommended)

Timbral / spectral-envelope similarity

The script uses Praat's built-in MFCC analysis with 12 coefficients, a 25 ms window, and the user-selected time step. Each frame vector is normalized to unit length, then compared by cosine similarity.

Best suited to: general musical-structure exploration when changes in spectral character and instrumentation are meaningful.

4. Spectral Entropy

Spectral disorder / texture

A Gaussian-window spectrogram is converted to a per-frame normalized Shannon entropy. If a frame has frequency-bin powers P_k, the script forms p_k = P_k / ΣP_k and computes:

H = -Σ p_k ln(p_k) / ln(N_bins)

The result is already bounded approximately to 0–1 and is deliberately not min-max stretched across the recording. Similarity is 1 - |H_i - H_j|.

5. LPC coefficients

All-pole spectral-envelope similarity

The script uses Praat's autocorrelation LPC analysis with fixed order 16, 25 ms window length, and the selected time step. It compares the LPC coefficients themselves; it does not convert them to formant frequencies.

Each coefficient vector is normalized to unit length and compared by cosine similarity, clamped to 0–1.

6. Mel filterbank

Perceptually spaced spectral-energy similarity

This is a true 40-band triangular mel filterbank, not a set of single-frequency samples. The filters span 100 Hz to 8000 Hz, or to Nyquist when Nyquist is lower. Weighted spectrogram power is averaged inside each triangular band, converted to a non-negative dB-like value, then the 40-element frame vector is unit-normalized and compared by cosine similarity.

7. Chroma (12 pitch classes)

Pitch-class recurrence

Chroma uses a dedicated 90 ms analysis window and 5 Hz frequency grid so that semitone structure is more resolvable than with the 25 ms window used by the other spectral features. The analyzed range is approximately C3 (130.81 Hz) to C7 (2093 Hz), limited by Nyquist if necessary.

Spectrogram rows are mapped to the nearest MIDI semitone, averaged within each semitone band so high frequencies do not dominate merely because they contain more linear-frequency rows, then folded into 12 pitch classes. The script folds magnitude rather than raw power and compares unit-normalized chroma vectors by cosine similarity.

Best suited to: repeated harmonic or pitch-class material that may recur in different registers.

Feature selection guide

Goal	Useful starting feature	What the matrix emphasizes
Melodic recurrence	Pitch	F0 proximity in semitones
Melody with dynamics	Pitch + Intensity	Pitch plus level
General structure / timbre	MFCC	Spectral-envelope direction
Texture / noise–tone contrast	Spectral Entropy	Spectral disorder
All-pole envelope changes	LPC coefficients	Predictor-coefficient pattern
Broad spectral-energy shape	Mel filterbank	Mel-band distribution
Harmony / pitch-class recurrence	Chroma	12-class pitch content

Similarity Rules

Raw SSM values

The current implementation uses feature-specific similarity functions; there is no single universal metric for all seven feature types.

Feature	Similarity	Range
Pitch	Gaussian kernel in semitone distance, σ = 1.5	0–1
Pitch + Intensity	0.75 pitch Gaussian + 0.25 intensity Gaussian (σ = 12 dB)	0–1
MFCC	Cosine of unit-normalized vectors, negative values clamped	0–1
Spectral Entropy	1 − absolute normalized-entropy difference	0–1
LPC coefficients	Cosine of unit-normalized vectors, negative values clamped	0–1
Mel filterbank	Cosine of non-negative unit-normalized vectors	0–1
Chroma	Cosine of non-negative unit-normalized vectors	0–1

Main diagonal: after the similarity matrix is computed, the script explicitly sets every diagonal cell to 1. This preserves the definition of self-similarity even for unvoiced pitch frames or zero feature vectors, without making those frames spuriously similar to one another off the diagonal.

Reading the Matrix

Both axes are frame index. The effective time step is printed in the figure metadata and Info window, so an approximate time can be recovered as a frame offset multiplied by that effective step.

Typical visual cues

Main diagonal: self-comparison; always maximal.
Parallel off-diagonal lines: similar evolving sequences occurring at different times.
Off-diagonal blocks: longer regions whose frame-level feature content resembles another region.
Locally uniform regions: sustained similarity according to the selected feature.
Abrupt dark/light changes: possible feature transitions; they are not automatically classified as musical boundaries.

Interpretation depends on the feature. A bright chroma block means similar pitch-class content; a bright MFCC block means similar normalized cepstral direction; a bright entropy block only means similar spectral disorder. The script does not infer verse, chorus, phoneme, speaker identity, or formal labels by itself.

Display processing

When Auto_contrast is enabled, the display copy is raised to a power chosen from its mean value:

mean > 0.95 → power 20 mean > 0.90 → power 10 mean > 0.80 → power 5 otherwise → power 3 Then the display copy is rescaled to 0–1.

Gamma is applied after auto contrast as display = display^Gamma. The returned Matrix remains unchanged.

Colour schemes

Option	Rendering
Grayscale	Built-in Matrix paint
Heat	Black → red → yellow → white; default
Viridis	Blue/green/yellow approximation
Plasma	Purple/red/yellow approximation
Inverted Grayscale	Display copy is inverted before painting

Performance and Safety

Quadratic matrix size

An SSM with N retained frames contains N² cells. The script therefore enforces max_ssm_frames = 2000. At that cap, the raw SSM contains at most about four million cells, before temporary display objects are considered.

Frame skipping

Requested versus effective skip

effective_skip = max(Frame_skip, ceil(total_frames / 2000)) num_frames ≈ ceil(total_frames / effective_skip)

If the adaptive cap is triggered, the Info window explicitly reports it. Increasing Frame_skip reduces matrix dimensions and therefore reduces both computation and memory approximately quadratically.

Sample-rate preprocessing

There is no user-facing downsampling switch. If the selected analysis channel has a sample rate above 22050 Hz, the script resamples the working copy to 22050 Hz with precision 50 before feature extraction. Lower-rate material is left at its original sample rate.

Minimum duration: sounds shorter than 0.05 s are rejected as too short for reliable frame-based SSM analysis.

Parameters

User-facing controls

Parameter	Default	Description
Feature	MFCC (recommended)	One of seven feature representations.
Time_step	0.01 s	Requested analysis step. Some Praat objects report their actual frame step, which the script uses where appropriate.
Frame_skip	1	Use every k-th feature frame, subject to the automatic 2000-frame cap.
Color_scheme	Heat	Grayscale, Heat, Viridis, Plasma, or Inverted Grayscale.
Auto_contrast	1	Applies adaptive power contrast only to the display copy.
Gamma	1.0	Additional display-only power mapping; 1.0 leaves gamma unchanged.

Fixed implementation parameters

Setting	Value	Used by
Pitch floor / ceiling	75 / 600 Hz	Pitch, Pitch + Intensity
MFCC coefficients	12	MFCC
LPC order	16	LPC coefficients
Mel bands	40	Mel filterbank
General spectral/LPC window	0.025 s	MFCC, entropy, LPC, mel
Chroma window	0.09 s	Chroma
Chroma frequency step	5 Hz	Chroma
Chroma nominal range	130.81–2093 Hz	Chroma
Pitch similarity σ	1.5 semitones	Pitch kernels
Intensity similarity σ	12 dB	Pitch + Intensity
Maximum SSM frames	2000	All features

Applications

Exploratory musical-structure analysis

Use MFCC, Mel, or Chroma to inspect whether distant passages share similar timbral or harmonic content. Repeated sections may produce off-diagonal lines or blocks, but the script leaves boundary detection and formal labeling to the analyst.

Melodic and harmonic recurrence

Pitch highlights local F0 recurrence in semitone space. Chroma is more appropriate when similar pitch-class material may return in another octave or register.

Texture comparison

Spectral Entropy can expose regions with similar degrees of spectral concentration or disorder. It is a one-dimensional descriptor, so it should not be interpreted as a full timbral match.

Speech or vocal exploration

MFCC and LPC-coefficient SSMs can be used to inspect recurring spectral-envelope patterns in speech or singing. The script does not perform phoneme recognition, speaker verification, forensic identification, or automatic linguistic labeling.

Troubleshooting

Matrix looks uniformly bright: keep Auto_contrast enabled, try a feature better matched to the question, or inspect the raw Matrix values before drawing conclusions.
No clear repetition pattern: the chosen feature may not encode the repeated property. For example, pitch recurrence can disappear in MFCC, while octave-shifted harmonic recurrence may appear more clearly in Chroma.
Less temporal detail than expected: check the effective frame skip. The 2000-frame safety cap may have increased it automatically.
Multichannel result seems unexpected: remember that only the strongest-RMS channel is analyzed.