Sound as Form

What this does

Sound as Form is a voice-controlled synthesis instrument, not a voice effect. You sing a single phrase into a microphone; Praat extracts the phrase's pitch movement, intensity, resonance, articulation and spectral character. The script then synthesizes four new sounds, mixes their contributions continuously through a gesture-controlled vector and organizes them in stereo, four-channel or eight-channel space.

The original recording is a source of control information only. Its recorded samples do not become part of the finished music. What the output retains is the shape of the performance: for example, an upward glide, a dynamic swell, a sustained vowel, a sequence of articulations or a breathy ending.

When Show_graphic_score is enabled, the script also draws a Spectromorphological Listening Score in the Praat Demo window. Its notation combines tonal ribbons, resonant ellipses, pluck strokes, noise textures and spatial trajectories, all derived from actual synthesis events and control data rather than decorative animation.

Key features:

• One-phrase microphone workflow: record once, analyse, synthesize, listen and retain the result in Praat.
• Four native synthesis methods: FM, artificial formant voice, modal/Karplus–Strong, and stochastic textures.
• Gesture-driven vector synthesis: all four engines can contribute simultaneously; their balance changes during the phrase.
• Multichannel spatial composition: continuous individual engine positions in Stereo, Quad or Octophonic formats.
• Graphic listening score: a score-like notation of frequency, resonance, attacks, stochastic activity and spatial motion.
• Immediate audition: the completed result is played automatically with the existing Play command.
• Memory-only operation: no temporary audio files, saved analyses, project files or exported graphics.
• One final object: only the new multichannel Sound remains in Praat Objects; temporary objects and the microphone recording are removed.

Quick start

1. Connect a microphone and confirm that Praat can record from it. In the Praat Objects window, launch Sound_as_Form.praat from your AudioTools installation. You do not need to select a Sound: ordinary operation records a new phrase.
2. Keep the initial settings for a first trial: 6 seconds, 44,100 Hz, Stereo (2), full pitch following and full envelope influence.
3. Leave Show_graphic_score enabled to generate the listening score, or untick it if you only want the synthesized audio.
4. Confirm the settings. At the Sound as Form – ready prompt, choose Record and sing one phrase. The recording runs for the selected fixed duration; the script does not listen continuously.
5. Allow Praat to analyse the gesture and render the four synthesis engines, their evolving vector mix and the output channels.
6. The result appears as a new Sound in Praat Objects. The script opens the Demo score if requested and then calls Play on the finished Sound.
7. Keep working with the output in Praat. Save or export it manually only if you want to preserve it outside the current Praat session.

Core concept: the voice becomes the form

The process is offline and phrase-based. The input recording is completed before synthesis begins; this is not a low-latency voice-to-sound live instrument or a continuous microphone effect. The control information is sampled on a common 10 ms timeline, with the resulting sound synthesized at the chosen audio sampling rate.

The four sound generators are implemented inside the same Praat script. They are not launched as four external AudioTools scripts and do not require Python. The microphone waveform is analysed but never mixed as a dry component into the output.

How the voice is analysed

The script calculates several complementary descriptors. These describe what changes during the performance, not merely a single average pitch or loudness value. The trajectories are aligned on one timing grid so that events in the voice can affect multiple synthesis and spatial processes coherently.

When pitch is uncertain: unvoiced and noisy portions do not receive an invented sung melody. The synthesis engine uses a slowly smoothed centroid-derived tonal reference where necessary; the listening score distinguishes reliable pitched regions from inferred or bridged pitch. Long or brief unvoiced gaps may be held or interpolated for control continuity.

Four native synthesis engines

Each engine generates new audio from the same vocal analysis, but uses a different synthesis method. The vector mixer then controls how strongly each contributes throughout the composition. All four operate on the same duration and time base.

Gesture-driven vector synthesis

Sound as Form uses one vector mode only: Gesture-Driven Vector. There is no manual XY joystick, drawn trajectory or preset vector sequence. The singing controls a point (X(t), Y(t)) that moves continuously among the four synthesis identities.

X primarily reflects pitch-corrected spectral brightness and movement of the second formant. Y combines voicing, continuity, novelty and pitch stability. Descriptors are robustly normalized per performance and smoothed before mixing; silent regions hold the most recent active vector rather than jumping to an arbitrary position.

These four nonnegative weights always add up to one. The engines are first brought to comparable working RMS levels, then mixed with gains related to the square roots of their weights, so the weights function as intended energy shares. The result is a continuously changing combination of sound generators—not a series of hard switches.

Vector_motion_depth determines how far the singing can move away from the performance's automatically estimated reference vector. At 0% the reference mix is held; near 100% the measured trajectory moves naturally, and values above 100% can exaggerate the movement subject to bounded coordinates.

Gesture-driven spatial composition

The spatial engine is not simply a final pan applied to a mono synthesis mix. Each of the four synthesis engines has its own spatial position, and each output channel receives a changing combination of the engines.

The four engines occupy distinct reference directions around a moving centre. Their actual positions combine that reference layout with a shared gesture-derived rotation and a changing separation. The spatial motion is related to, but distinct from, the sonic XY vector: a change of timbre does not automatically mean an identical move in space.

For multichannel rings, speaker contributions follow a circularly distributed, smooth gain function rather than hard channel switching. The gains for a given engine are normalized in squared sum, helping preserve its energy through spatial movement. Stereo uses an equal-power pan with controllable spread.

Stereo, Quad and Octophonic output

The Output_channels setting chooses one finished Praat Sound with exactly 2, 4 or 8 channels. A multichannel result is not an export file or a collection of separate mono objects; it is an ordinary Sound object that the user can subsequently inspect, transform or export.

Angles are measured clockwise from the front of the listening field, with the listener conceptually at the centre. The specified channel order is important if you later route the Sound to a physical loudspeaker array in another application.

Spectromorphological Listening Score

The optional Demo-window display is a graphic listening score of the resulting electroacoustic work. Its visual language is inspired by graphic notation and spectromorphological thinking: the important objects are moving sonic gestures, resonances, attacks, textures and spatial relationships—not staves, note names or conventional audio-editor waveforms.

The score combines a broad main time–frequency field with a smaller spatial notation strip beneath it. Both read from left to right. The main field's vertical axis is logarithmic frequency, from approximately 60 Hz to 8 kHz (or the available upper frequency limit). Light grid lines give temporal and frequency references without becoming the principal image.

The score is constructed using measured envelope-like activity from the actual synthesized engines; the spatial strip additionally uses energy measured in the completed multichannel result. This makes it more informative than a decorative animation.

How to read the four graphic vocabularies

The drawing uses four distinct visual families. Unlike a conventional timeline with one separate lane per algorithm, they can overlap in the same score space as the vector mixer combines the sounds.

Meaning of graphic intensity

Thicker or more visible marks generally indicate a stronger engine contribution at that moment, taking into account its synthesized level, vector weight and the final envelope. A quiet passage may contain the same synthesis family as a loud passage but show it much more faintly. Where an engine is nearly inaudible, its marks can disappear entirely.

Events versus textures: a teal attack stroke corresponds to an actual excitation event; a grey particle dot corresponds to a synthesized grain. In contrast, the noise stipple is a controlled graphic texture that summarizes noise presence rather than depicting every individual noise sample.

One example of reading a phrase

Imagine singing a rising vowel with a gradual crescendo and ending on a breathy consonant. The pitch-following violet ribbon may rise across the score, amber resonance forms may shift with the vowel, teal excitation strokes may punctuate new articulations and the grey texture may become more apparent as voicing weakens. The exact balance depends on the acoustic analysis; the example describes how to interpret the score, not a guaranteed graphic pattern.

Reading the spatial notation strip

The narrow strip below the main score describes the spatial organization of the same phrase. It is not an unrelated XY display; it shares the score's horizontal time axis and is informed by both the gesture-derived spatial controls and the measured output-channel energy.

Important distinction: the coloured lines represent where the internal engines are directed. The dashed line and grey band are derived from the finished multichannel mixture. These can differ because the four engines contribute different amounts of energy at any moment.

Channel and angular values in the strip are references for a multichannel sound object; the score is not a substitute for testing a finished work in a physical loudspeaker installation.

Main Praat interface and controls

The user dialog is intentionally compact. It does not ask you to select one of the four engines or to move a vector manually. The voice always drives the synthesis vector and spatial behavior.

Playback and graphic-score display

After the multichannel Sound has been rendered and all intermediate objects have been removed, the script selects the result and calls the existing Play command. The completed Sound is therefore auditioned automatically; a separate playback setting is not needed.

When Show_graphic_score is on, the Demo-window listening score is drawn after the temporary-object cleanup and before the final Play call. The score remains visible without a blocking event loop, while the resulting Sound stays selected in Praat Objects.

The Demo image is a temporary, on-screen listening aid. The script does not automatically export it as an image or create a graphics file. It does not add any score-specific objects to the Praat Objects list.

Compositional explorations

Output, object management and privacy

On a successful run the script creates exactly one new multichannel Sound object. The names follow the chosen output arrangement; if a name already exists, a collision-safe numeric suffix is added.

Every original user object that was present in Praat before the script is intended to remain untouched. The newly recorded microphone Sound and all temporary analysis, synthesis and spatial-control objects are tracked and removed before the script completes. The listener score uses internal arrays and the Demo display; no persistent score objects or exported graphics are created.

Requirements and installation

Keep the script in the library's normal synthesis location. It does not depend on calling four separate synthesis scripts, nor does it need a Python helper beside it.

plugin_AudioTools/
  Generative & Synthesis/
    Sound_as_Form.praat

The recording begins only after the settings dialog and the subsequent Record confirmation. Praat's microphone and sound-device permissions must be configured on the target computer.

Limitations and interpretation

• One completed phrase: there is no continuous capture or real-time synthesis. Rendering begins after the requested recording duration has elapsed.
• Acoustic tracking is an estimate: soft vowels, noisy consonants, strong breathiness, extreme pitch motion and background noise may challenge Pitch, harmonicity and formant analysis. The script uses fallbacks but cannot infer an exact melody from a fully unvoiced phrase.
• One performed shape, not one fixed sound: the four synthesis engines are stochastic or nonlinear in different ways. Identically sung phrases are not guaranteed sample-for-sample identical, especially in randomized components.
• Graphic notation is interpretive: the score shows generated events, model frequencies, engine activity and rendered channel-energy relationships. It is not a complete spectral measurement of the finished multichannel waveform.
• FM strands are schematic: modulation-related graphic strands illustrate synthesis behaviour; they should not be read as exact measured FM sideband frequencies.
• Diffuseness weakens directionality: the spatial strip's rendered direction is less meaningful when channel energy is symmetrically distributed.
• Rendering time varies: long recordings, four full synthesis engines, sample-rate choices, higher channel counts and score rendering can increase computation time and memory use.
• Peak scaling, not loudness mastering: the complete result is scaled to a shared sample-peak ceiling. This does not guarantee a uniform perceived loudness across different performances, or a true-peak ceiling between samples.
• Playback routing is external: the multichannel Sound is constructed in Praat; monitoring it over a particular physical loudspeaker array depends on the user's audio configuration or subsequent export workflow.
• Temporary display: the Demo listening score is not automatically saved, and no permanent audio archive is created.

Practical troubleshooting

Practical recommendation: begin with a six-second stereo phrase and leave the graphic score enabled. Once pitch tracking and the relationship between sung morphology and synthesized sound are clear, explore longer forms and Quad or Octophonic output. Change one musical parameter at a time to hear its contribution.