Corpus Concatenative Codec — Neural Codec Corpus Synthesis
Corpus-based concatenative synthesis using a neural audio codec (EnCodec or DAC) as the matching token space. Six modes: Match Build corpus Draw Gesture rhyme Continuation tail Polyphonic layers. Transform sounds, draw timbral gestures, re-voice kinetic patterns, extend a phrase in time, or generate up to three related voices that sound alongside it.
What this does
This script implements corpus-based concatenative synthesis using a neural audio codec (EnCodec or DAC) as the matching token space. It offers six modes: Match (synthesise from a selected Sound), Build corpus (encode a corpus once), Draw (compose a time-varying brightness path in an interactive Demo window), Gesture rhyme (re-voice an abstract kinetic gesture using codec-token transition structure), Corpus continuation tail (extend the ending of a selected mono Sound with a coherent grain sequence), and Corpus polyphonic layers (generate one to three simultaneous corpus-based voices, aligned with the entire source phrase).
Key Features:
- 6 Modes — Match, Build corpus, Draw, Gesture rhyme, Corpus continuation tail, Corpus polyphonic layers
- Corpus polyphonic layers — generate one to three synchronized corpus-derived voices alongside the original phrase
- Three polyphonic relationships — Parallel, Complementary and Independent, with Relationship strength and Inter voice diversity controls
- Stereo and DAW output — automatic or manual equal-power panning, stereo mix, aligned mono voice objects and optional 32-bit float WAV stems
- Corpus continuation tail — Tight or Textural phrase extension using a cached transition graph
- Controllable continuation — percent or fixed length, strength, repeatable Textural variations with a random seed, and four dynamic envelopes
- Level-aware joining — phrase/ending level choice, source-anchored RMS matching and peak control, with a sample-exact in-memory join in Praat
- 4 Match Presets — Rhythmic, Textural, Faithful, Sparse
- Interactive Draw surface — edit a 0..1 brightness contour directly in a Praat Demo window; no RealTier editor is required
- Corpus-aware Draw feedback — grain-density shading, red no-grain bands, and an orange step trace showing the nearest available brightness at each grain-rate step
- Draw generators + transforms — Sine, Triangle, Angular, cycle count, Same wave faster ×2, Same wave slower ÷2, Undo, Reset, and optional range stretching
- Audible Draw preview — Preview uses the same synthesis path and parameters as Render, then returns to editing
- Reusable Draw contours — every rendered Draw gesture is saved as a compact
corpus_draw_contourfile for exact replay and controlled comparisons - Gesture rhyme mode — hashed-bigram kinetic matching, re-voice abstract gestures
- Sequence context — average source features over preceding sub-windows for sustained transition structure
- Codec support — EnCodec, DAC, or mock (no torch dependencies for testing)
- Onset-detected segmentation — source rhythm drives output timing
- Sub-window subdivision — decaying/evolving segments match new corpus material as they change
- Timbre + loudness matching — cosine distance on token features + log-RMS energy term
- Optional TextGrid — provenance for the selected grains; polyphonic mode provides a source-analysis tier and a separate tier for each generated voice
Quick start
- Prepare a corpus audio folder (sounds to use as source material).
- First time: run Build / replace corpus — set
Corpus_audio, choose DAC or EnCodec, and set the corpus grain/hop values. The analysed corpus is cached and reused. - Choose a mode. Match, Gesture rhyme, Corpus continuation tail and Corpus polyphonic layers require a selected Sound object; the two latter modes require exactly one mono Sound. Draw does not require a Sound selection.
- For Draw, choose Interactive (Demo window) or Contour file. Interactive Draw opens the dedicated contour surface; a saved contour replays its stored time→brightness trajectory and duration.
- In Interactive Draw, click empty space to add points; click a point and then click elsewhere to move it; Shift-click a point to delete it. Use the on-screen generators/transforms, Preview (P), and Render (Enter).
- For Gesture rhyme, adjust Bigram_weight (high = kinetic matching), Hist_weight (low = de-emphasise timbre), and Sequence_context (context smoothing).
- For Corpus continuation tail, select a mono phrase and choose the tail length, Tight or Textural, strength, random seed, envelope, level and normalization. The existing corpus is used; on first run its transition graph is built automatically.
- For Corpus polyphonic layers, choose the number of new voices, Parallel / Complementary / Independent, strength, diversity and Seed. Then set stereo panning and levels, and choose Stereo mix, Separate voices or Both; enable aligned WAV-stem export only if you want files for a DAW.
- The Python backend synthesises the requested material. For Continuation tail, Python renders the tail and Praat joins it to the source in memory. For Polyphonic layers, Python renders aligned mono voices and Praat builds the stereo mix using the original Sound in memory. A rendered Draw run saves the contour actually used for later replay.
pip install numpy scipy soundfile. For EnCodec: pip install encodec. For DAC: pip install dac. The corpus index is built once and reused. Gesture rhyme, Corpus continuation tail and Corpus polyphonic layers require an existing corpus index; none of these modes automatically replaces the corpus. Continuation's separate transition graph is cached and rebuilt when needed. The "mock" codec is for structural testing rather than musical-quality output.
6 Modes
Match (synthesise from corpus)
Select a Sound in Praat. The script detects onsets, subdivides into analysis windows, matches each sub-window against the corpus (timbre + loudness), and reconstructs the output.
Use: Transform any sound using a corpus of your choice.
Build corpus index
One-time encoding of a corpus folder. Slices audio into overlapping grains, encodes each into codec tokens, stores features and metadata.
Use: Prepare a reusable corpus for Match, Draw, Gesture rhyme, Corpus continuation tail and Corpus polyphonic layers.
Draw (brightness contour → corpus)
Draw and edit a time→brightness contour in a dedicated Praat Demo window. The display shows the corpus's own brightness distribution and previews which available brightness each grain-rate step will select.
Use: Compose a path through the corpus, audition it before rendering, and save the exact contour for later replay.
Gesture rhyme
Re-voice an abstract kinetic gesture using codec-token transition (hashed-bigram) rhyming. High bigram weight, low histogram weight. A click-train can be voiced by speech syllables or field recordings that move the same way.
Use: Voice abstract kinetic shapes — accelerating clicks, bouncing balls, explosive attacks, tremolo flutter.
Corpus continuation tail MODE 5
Extend the end of a selected mono phrase with a corpus-generated continuation. Tight favors chronological passages and stable identity; Textural explores statistically plausible transitions with seed-controlled variations. Set duration, dynamic envelope, and level matching.
Use: Turn a short recorded flute, vocal or instrumental phrase into a longer evolving musical gesture without replacing the original recording.
Corpus polyphonic layers NEW — MODE 6
Analyse an entire mono phrase and create 1–3 simultaneous corpus-derived voices, each aligned to the source duration. Choose Parallel, Complementary or Independent behaviour, control diversity and spatial placement, and return a stereo mix, separate Sound objects, or both.
Use: Expand a solo instrumental gesture into a polyphonic sound texture, with optional synchronized WAV stems for a DAW.
Draw Mode — Interactive Brightness-Contour Composition
Draw mode treats the analysed corpus as a navigable brightness field. Each corpus grain has a spectral-centroid value; the backend maps the corpus's log spectral-centroid range to a normalized 0..1 brightness axis (0 = darkest, 1 = brightest). The drawn contour is sampled at the chosen grain rate, and each step selects the corpus grain whose available brightness is nearest to the target, with repeat penalty applied during the actual render.
Editing the contour
| Action | Result |
|---|---|
| Click empty space | Add a contour point. |
| Click a point, then click elsewhere | Move the selected point. Interior points move in time and value; the two end points remain fixed in time and move vertically only. |
| Shift-click a point | Delete an interior point. End points cannot be deleted. |
| U / Undo | Undo the last point edit, generator, or time transform. |
| R / Reset | Return to the default dark→bright ramp. |
| S / Stretch | Toggle legacy min/max stretching of the drawn range to 0..1. |
| P / Preview | Render and play the current gesture using the same backend Draw call and synthesis parameters as the final render, then return to editing. |
| Enter / Render | Render the current gesture, import the result, and save the normalized contour actually used. |
| Esc / Cancel | Leave Draw without rendering. |
Built-in gesture generators and time transforms
Sine, Triangle, and Angular rewrite the editable point list across the whole Draw duration. The cycles − / + controls set 1–32 cycles. The generator range is taken from the current drawing's lowest and highest values; if the drawing is essentially flat (range < 0.02), the generator uses the full 0..1 brightness range. Angular creates a new jittered zigzag variant each time it is invoked.
Same wave faster ×2 compresses the current drawing into half the duration and repeats it twice. Same wave slower ÷2 takes the first half of the current drawing and stretches it over the full duration. These operations modify the current contour and remain fully editable and undoable.
Absolute brightness vs. Stretch
By default, the vertical axis is absolute within the current corpus: a point at 0.25 targets the lower quarter of that corpus's normalized log-brightness range, while a point at 0.75 targets the upper quarter. Turning Stretch drawn range on reproduces the older RealTier behaviour: the minimum and maximum values in the current drawing are expanded to 0 and 1 before synthesis; a completely flat drawing maps to 0.5. When Stretch is active, the window also shows the effective stretched contour.
Preview, Render, and reusable contour files
Preview writes a temporary contour and calls the same Draw backend with the same grain rate, crossfade, repeat penalty, duration, and effective stretch result used by Render. It produces a temporary WAV, plays it in full, deletes it, and returns to the editor. Preview does not save a persistent contour copy.
Render writes the gesture in the compact human-readable corpus_draw_contour 1 format and asks the backend to save the normalized time→brightness trajectory actually used. The persistent copy is stored under Praat's preferences directory in corpus/draw_contours/. You can later choose Draw source = Contour file and load that file to reproduce the same gesture exactly; the contour file's stored duration is reused.
--tier path for backward compatibility, but the current Praat front-end writes and uses explicit contour files.
4 Match Presets
| Preset | Onset Interval (ms) | Analysis Grain (ms) | Analysis Hop (ms) | Energy Weight | Crossfade (ms) | Repeat Penalty | Character |
|---|---|---|---|---|---|---|---|
| Rhythmic | 40 | 40 | 20 | 1.5 | 10 | 0.10 | Follows transients closely — fast, articulated. |
| Textural | 120 | 120 | 60 | 0.5 | 60 | 0.02 | Smeared, washed — ambient texture. |
| Faithful | 60 | 50 | 25 | 1.0 | 20 | 0.15 | Track source closely — balanced. |
| Sparse | 80 | 100 | 80 | 1.0 | 15 | 0.30 | Distinct granular stutter — sparse. |
Codec Token Space — Features for Matching
Token extraction
Each audio grain is encoded by a neural codec (EnCodec or DAC) into tokens: [n_codebooks, n_frames] of integer token IDs.
Token IDs are categorical symbols — we never take Euclidean distance on raw IDs.
Searchable features
- Per-codebook token histograms — marginal distribution of each codebook (1024 bins) — timbre
- Hashed bigram transitions — ordered token-pair structure kept separate per codebook; each codebook hashes transitions through a 64-bit avalanche mix into 1021 buckets — local sequence / kinetic structure
- Features are L1-normalised and concatenated; compared with cosine distance.
Energy term
Cosine distance on token features captures timbre but not loudness. A near-silent grain and a loud one can have similar token-histogram shapes. The match function adds a separate log-RMS energy term: energy_dist = |log(RMS_corpus) - log(RMS_source)| / 4, weighted by energy_weight.
Gesture rhyme weights
- Bigram_weight (default 4.0) — high = kinetic motion drives the match
- Hist_weight (default 0.5) — low = literal timbre de-emphasised
- Energy_weight (default 0.2) — low = energy can't dominate kinetic match
Gesture Rhyme Mode — Hashed-Bigram Kinetic Matching
- Select a Sound object — the abstract gesture source. This can be a click train, a bouncing ball, an explosive attack decaying into hiss, a microtonal dive, tremolo flutter — anything with a kinetic shape.
- Choose Mode = Gesture rhyme.
- Set Bigram_weight (high = kinetic matching), Hist_weight (low = de-emphasise timbre), Energy_weight (low = energy de-emphasised).
- Set Sequence_context (0–3) to smooth matching over preceding sub-windows.
- Click OK — the system searches the EXISTING corpus for grains whose token-transition structure rhymes with the source.
- Output is voiced by corpus grains that move the same way, regardless of their literal sound.
- Bigram_weight 4.0 — strong kinetic matching
- Hist_weight 0.5 — timbre de-emphasised
- Energy_weight 0.2 — loudness de-emphasised
- Sequence_context 1–3 — average over preceding windows for smoother transitions
- Accelerating clicks — rapid attack → voiced by grains with increasing token-change rate
- Bouncing ball — decaying impacts → voiced by grains with damping structure
- Explosive attack → hiss — sharp onset, noisy decay → voiced by grains with similar envelope
- Microtonal dive — descending pitch glide → voiced by grains with descending token trajectory
- Tremolo flutter — rapid amplitude modulation → voiced by grains with oscillating tokens
Corpus Continuation Tail — Extending a Musical Phrase
MODE 5 · NEW IN v1.15.0 Select a recorded mono Sound in Praat and create a new continuation from the already analysed corpus. Rather than replacing the phrase with matched grains, the tool retains the original sound and adds new material whose timbre, pitch/tonality, energy and temporal behavior relate to the end of the phrase. This is corpus-guided continuation, not neural generation of new codec tokens and not a conventional reverb or delay.
Getting started
- Run Build / replace corpus once, if you have not already created an analysed corpus. You can reuse the existing one; no new corpus is required for this mode.
- Select exactly one mono Sound containing the phrase to extend.
- Choose Mode = Corpus continuation tail. In dialog 1, specify length, continuation behavior, strength and random seed.
- In dialog 2, choose the dynamic envelope, target tail level, join crossfade, silence handling, output type and normalization. Play_result and Import_textgrid come from the main form.
- Run the script. On first use, Praat AudioTools builds a separate transition graph cache from the stored corpus grains. Later runs reuse it. The new Sound is imported into Praat and can be played immediately.
Continuation modes
Tight — follow the corpus, keep identity
A deterministic continuation that strongly favors the real next grain from a corpus recording when it remains suitable. Other transitions are allowed when the chronological sequence ends or a better continuation is needed. Source identity and local musical direction have relatively strong influence.
Best for: sustained instrumental sounds, melodic continuity, and controlled extensions of a recorded gesture.
Textural — statistical, drifting
A seeded, probabilistic choice among compatible grain transitions. The connection starts close to the original ending, then permits more exploration as the tail unfolds. The system discourages rapid repeated jumps and short repetitive cycles.
Best for: evolving timbres, alternative endings, corpus-based variation and emergent phrases.
Continuation parameters
| Parameter in Praat | Default | What it does |
|---|---|---|
| Tail duration mode | Percentage of source duration | Choose the new material's length as a percentage of the effective source duration, or specify seconds. |
| Tail percent / Tail seconds | 50% / 4.0 s | Only the value for the chosen duration mode is used. The maximum rendered tail is 300 seconds. |
| Continuation mode | Tight | Choose deterministic chronological continuity or stochastic textural development. |
| Continuation strength | 75 (0–100) | Controls attachment to the source ending and the permissible grain choices. Higher values favor closer identity; lower values allow more exploration. |
| Random seed | 1234 | Controls repeatability in Textural mode. Tight selection does not depend on random sampling. |
| Dynamic envelope | Natural exponential decay | Shapes the amplitude of the new material, independently of the selected grain sequence. |
| Tail level | Phrase level | Phrase level rises over roughly 0.3 s from the join to the phrase's median active RMS. Ending level instead follows the quieter level measured at the source's end. |
| Join crossfade ms | 50 ms | Overlap between the source ending and the start of the generated tail; constrained by available material and limited to 2,000 ms in the Praat interface. |
| Trim trailing silence | On | Conservatively removes clearly silent material before the join. A quiet tonal release is treated as meaningful sound rather than automatically discarded. |
| Output mode | Original + tail | Either produce the combined phrase or the new material alone (Tail only, excluding the crossfade prefix). |
| Normalize | Loudness match tail to source, then peak | Balances the weaker or stronger corpus continuation against the original phrase. See the distinction below. |
| Keep metadata | Off | Optionally retains a JSON record of the chosen grains, transitions and render parameters. |
Four dynamic envelopes
A smoothly falling amplitude envelope that reaches silence at the end. This is not a physical simulation of room reverberation.
Holds the tail at approximately steady envelope gain for most of its duration, then performs a short, smooth final fade.
Applies a gently decaying envelope with approximately 5.5 Hz amplitude modulation.
Briefly increases envelope gain after the join and then fades smoothly to zero; subsequent level and peak controls prevent uncontrolled peaks.
Tail level and normalization
The stored corpus recordings may have a much lower or higher recording level than the selected source phrase. To avoid a conspicuous level mismatch, the engine smooths grain-to-grain level changes and provides two independent decisions: Tail level controls the desired level trajectory, while Normalize controls the final peak/RMS matching strategy.
| Normalize option | Behavior |
|---|---|
| Off (source untouched) | No normalization gain is applied. The source's samples outside the join region are preserved in the new Sound, subject to optional silence trimming. |
| Peak, source and tail separately | Normalizes source and tail peaks separately toward −1 dBFS before joining. Equal peaks do not necessarily mean equal perceived loudness. |
| Loudness match tail to source, then peak DEFAULT | Uses RMS-based matching against the phrase's active level, constrains tail peaks relative to the source, and applies a common gain so the source peak reaches approximately −1 dBFS. The join is checked for excessive peaks. |
Duration and source preservation
The requested duration means new audio after the effective end of the phrase; it does not include the preliminary crossfade region. For an 8-second phrase and a 50% tail, the combined result is 12 seconds (assuming no trailing silence is trimmed). With a 50 ms join, Python supplies 50 ms of overlap material plus 4 seconds of new material. Praat joins these at the original Sound's sample rate.
Python performs corpus analysis, grain selection and tail rendering; Praat performs the final join on a copy of the source using an in-memory crossfade. The original selected Sound object remains unchanged. In Normalize = Off, copied samples outside the crossfade are preserved. In normalization modes, a gain may be applied to the new result.
How the transition graph works
Each corpus grain is a node in a separately cached Corpus Transition Graph. A chronological transition continues to the genuine next overlapping grain from the same source recording. A graph transition moves to another corpus position chosen using boundary similarity, timbral descriptors, energy and reliable pitch/tonality information. A direct-search fallback is available if necessary. Textural mode selects probabilistically among promising transitions, while Tight mode favors chronological continuity.
The graph is saved alongside the internal corpus index as current_corpus_graph.json and current_corpus_graph.npz. It is built on first use and recreated automatically when obsolete. The original schema-3 corpus index and stored grains are reused; building the graph does not replace the corpus. A first run over a large corpus can take longer because boundary features need to be analyzed.
TextGrid and metadata: follow the generated path
Enable Import_textgrid on the main Praat form to see where the material comes from. Continuation mode produces two interval tiers in output-time coordinates:
- sections — marks the original
source,joinand generatedtailregions (for Original + tail). - grains — identifies the selected recording/time and the transition type. The prefix is
Sfor the starting match,Cfor genuine chronological continuation,Jfor a transition-graph jump, andFfor a fallback search.
For example, C flute_take.wav@1.575 means a chronological continuation using material near 1.575 s of that corpus recording. The S/C/J/F prefixes describe the algorithmic transition, not the first letter of the filename. Optional Keep metadata stores a detailed JSON report in Praat's preferences under corpus/continuation_meta/, including the Seed, chosen grains, transition costs, and processing settings.
Corpus Polyphonic Layers — Generating Voices Alongside a Phrase
MODE 6 · NEW IN v1.16.0 Select one recorded mono Sound and ask the corpus to generate one, two or three new voices that sound at the same time as the original. Each voice spans the same number of samples as the selected phrase, at its original sampling rate, and follows its own sequence of corpus grains. Unlike Continuation Tail, which analyzes the ending and appends a future, Polyphonic Layers analyzes the entire phrase and constructs related musical activity across its timeline.
Quick workflow
- Build an analyzed corpus once using Build / replace corpus. Polyphonic Layers reuses its stored grains and the existing Corpus Transition Graph; it does not build a new source corpus.
- Select exactly one mono Sound in Praat, for example a 12-second flute phrase.
- Choose Mode = Corpus polyphonic layers. In the behaviour dialog choose 1–3 generated voices, a relationship (Parallel / Complementary / Independent), Strength, Diversity and Random seed.
- In space + level, select automatic or manual panning, source-relative voice level, and whether to normalize the completed stereo mix.
- In output, choose the Praat objects you want: Stereo mix, Separate voices or Both. Optionally enable WAV-stem export to a folder and retain detailed JSON metadata.
- Run the script. The new voices are generated from the corpus and imported into Praat. Play_result plays the polyphonic stereo mix; Import_textgrid displays the grain paths and source activity analysis.
Three relationships to the source
Parallel
Generated voices broadly follow the original phrase's activity, dynamics, brightness and pauses. The system favors grain changes near source onsets but may select different pitches and timbres.
Try it for: extending a sustained flute line into related timbral strands or creating a larger ensemble-like gesture from one instrument.
Complementary
Generated voices answer and contrast with the source: they may become active during rests or sustained notes and retreat toward more stable material during dense passages. Activity is also distributed across the additional voices.
Try it for: contrasting musical counterpoint and a conversation between a solo phrase and corpus-based companions.
Independent
Each new voice develops its own phrasing and evolving trajectory, while retaining some connection to the original sonic identity. As the phrase unfolds, attachment to the source can decrease.
Try it for: autonomous, evolving layers and experimental textures whose relationships emerge over time rather than copying one another.
These are different planning behaviors, not merely labels. All three still use the same corpus transition graph for grain-to-grain continuity. The quality of each relationship depends on the range and character of sounds available in the corpus.
Polyphonic settings in Praat
The mode uses three short configuration dialogs to avoid crowding the screen. Defaults below are from Praat front-end v1.16.6.
| Dialog / Parameter | Default | Meaning |
|---|---|---|
| 1 · Number of new voices | 2 | Generate 1, 2 or 3 new voices in addition to the original phrase. |
| 1 · Polyphonic relationship | Parallel | Choose Parallel, Complementary or Independent behavior. |
| 1 · Relationship strength | 75 | 0–100: controls how strongly voice planning follows the relationship's targets and the source's behavior. |
| 1 · Inter voice diversity | 50 | 0–100: discourages simultaneous repeated grains, copied passages and overly similar trajectories. Higher does not mean random. |
| 1 · Random seed | 1234 | Controls seeded candidate choices and autonomous trajectories. Reusing the same inputs and settings makes a run reproducible; different seeds can produce alternatives, but not every seed must sound different. |
| 1 · Refine voices jointly | On | After initial sequential planning, make a refinement pass in which each voice is considered against the others. Disable it for a stricter sequential plan. |
| 2 · Stereo panning | Automatic stereo spread | Automatically distribute voices across the stereo field, or switch to Manual pan positions. |
| 2 · Voice 1 / 2 / 3 pan | −60 / +60 / 0 | Manual settings, used only when Manual pan positions is selected. Range −100 (left) through 0 (center) to +100 (right). |
| 2 · Voice level mode | Match source active RMS | Balance generated voices relative to the source's active level; alternatively choose Keep corpus level. |
| 2 · Generated voice level dB | −6 dB | Relative level for each new voice, limited to −40 through +12 dB; independent of final stereo-mix normalization. |
| 2 · Normalize stereo mix | On | Apply one common gain to the finished stereo mix, preserving the balance established between its voices. |
| 2 · Mix peak target dB | −1 dBFS | Target stereo-mix sample peak, adjustable from −40 to 0 dBFS when normalization is enabled. |
| 3 · Layer output | Both | Choose Stereo mix, Separate voices or Both as the returned Praat objects. |
| 3 · Export aligned WAV stems | Off | When enabled, additionally write synchronized floating-point WAV files for use in a DAW. |
| 3 · Export folder | Empty | Enter a destination or leave empty to choose a folder when export is enabled. |
| 3 · Keep layer metadata | Off | Retain a JSON record of the parameters, grain paths, transitions, voice statistics, panning and gains. |
How the voices interact
Every new voice follows a coherent path through the existing Corpus Transition Graph. The planner balances acoustic transitions, the chosen source relationship, the voice's own recent trajectory, repeated material and interaction with other voices. It favors actual chronological corpus passages when appropriate, and can jump to another compatible passage when necessary.
Voices are initially planned in order, with later voices considering earlier ones. When Refine voices jointly is on, a further pass considers all voices together and retains improvements. Inter voice diversity is designed to avoid accidental layering of the same grain or near-identical behavior at the same time, while permitting meaningful recurrence at other moments.
Stereo panning and level balance
The original source remains at center. The added voices are placed using equal-power stereo panning. Panning changes only the stereo mix, not the separate mono voice objects.
| Number of added voices | Automatic pan locations | Listening result |
|---|---|---|
| 1 | Voice 1: +45 | One new voice appears to the right of the centered source. |
| 2 | Voice 1: −60; Voice 2: +60 | A left/right pair surrounding the centered source. |
| 3 | Voice 1: −70; Voice 2: +70; Voice 3: 0 | Two wider side voices and a third centered layer alongside the source. |
Match source active RMS addresses a common practical issue: corpus recordings may be much quieter than the source phrase. The algorithm matches the generated voices to the source's active level, applies controlled smoothing of grain-level fluctuations and retains mode-specific dynamic shapes. Keep corpus level leaves the material closer to its original level, with the requested dB adjustment and behavioral contour.
Normalize stereo mix is a separate operation: when on, the complete mixture receives one shared gain so that its highest sample approaches the selected peak target. This gain does not independently renormalize the returned mono voice objects. In very dense textures, the common gain can lower the overall mix while keeping voice relationships unchanged.
Output modes and DAW-compatible stems
| Layer output | Objects returned in Praat | Best use |
|---|---|---|
| Stereo mix | OriginalName_polyphonic_mix (2 channels) | Listen immediately to the composed stereo texture. |
| Separate voices | OriginalName_source plus OriginalName_poly_voice_1 (and 2/3 if requested) | Work with each mono voice independently. |
| Both DEFAULT | The stereo mix, source copy and every requested mono voice | Compare the premixed result with its individual layers. |
All generated voices have the same sample rate and exactly the same number of samples as the selected source. They share the source's Praat timeline origin, even if an individual voice begins with a rest. The selected original Sound object remains unchanged.
Enable Export aligned WAV stems and choose a folder. The export writes a matching set of 32-bit floating-point WAV files named, for example:
Flute_Source.wav, Flute_Voice_1.wav, Flute_Voice_2.wav, Flute_Voice_3.wav, and, if the stereo mix is requested, Flute_Stereo_Mix.wav.
Import the files at a common start position. The source and generated voices are time-aligned; voice stems remain mono and unpanned so you can mix, automate, process or spatialize them independently. The stereo mix file contains the selected panning and normalization.
The export uses a no-overwrite naming policy: if the chosen set of names is already present, a suffix such as _set2 is added. When export is off, temporary working files are removed after Praat has imported the results.
Source-stem precision: Praat uses a temporary 32-bit integer PCM WAV to pass source audio to Python; the final stereo mix in Praat uses the selected source in memory. The source stem is written as float WAV from that export. This is suitable for typical 16/24-bit source recordings, but it is not guaranteed to preserve every original floating-point sample exactly; source values exceeding full scale may be clipped by the temporary integer WAV. Check levels before exporting unusually hot material.
Polyphonic TextGrid and metadata
With Import_textgrid enabled, Praat receives a multi-tier TextGrid synchronized to the source phrase:
source analysis— labels analyzed source regions such as rest, sustained and active.poly voice 1, and additional voice tiers when requested — identify the source corpus recording, its time, and the chosen transition at each step. Empty intervals indicate rests.
On each voice tier, S means a newly started sequence, C genuine chronological continuation in the corpus, J a graph-guided jump, and F fallback search. The TextGrid provides a readable path through selected grains; overlapping synthesis windows are represented as non-overlapping annotation intervals.
Enable Keep layer metadata to retain detailed JSON under Praat's preferences directory (corpus/polyphonic_meta/), including chosen grains, costs, Random seed, refinement information, voice statistics, pan gains and mix normalization data. Otherwise metadata remains temporary.
Technical notes and limits
The system uses real stored corpus audio grains. DAC/EnCodec representations guide analysis and selection, but this is not a neural language model that autoregressively invents new audio tokens. It also does not automatically guarantee tonal consonance or traditional harmonic counterpoint. The result is a polyphony of related sonic behaviors, rather than score-derived harmonization.
Version 1.16.6 passes its Python command arguments through a temporary argument file, avoiding a Windows command-line truncation issue encountered in Polyphonic Layers. If something fails, consult the Praat Info window for the backend's diagnostic details. Existing corpus and graph caches remain available, and no GitHub operation is performed.
Applications
Corpus polyphonic layers — From one flute to a four-part texture
Use case: Start with a 12-second solo flute phrase and generate three simultaneous corpus-derived voices, keeping the original at the center. Try Complementary to encourage interaction between sustained and articulated passages, or Independent to explore more autonomous phrasing.
Starting settings: 3 new voices, Complementary, Relationship strength 75, Inter voice diversity 50, Refine voices jointly On, Automatic stereo spread, Match source active RMS, Generated voice level −6 dB, Both, Normalize stereo mix On. Enable Export aligned WAV stems to bring the source and voices into a DAW as separately editable tracks.
Corpus continuation tail — Alternative futures for a flute phrase
Use case: Take a short monophonic instrumental phrase and explore several possible endings without altering the original performance. In Tight, aim for continuity of the instrument's sustained sound; in Textural, let a different Seed reveal another coherent path through the same corpus.
Starting settings: Textural continuation, 50–100% tail, Continuation strength 40–75, Phrase level, Natural exponential decay, 50 ms crossfade, and the default Loudness match normalization. Changing Seed can yield new variations, depending on candidate dominance.
Gesture rhyme — Voice abstract kinetic shapes
Use case: Design a kinetic gesture (accelerating clicks, bouncing ball, explosive attack, microtonal dive) and voice it with corpus grains that move the same way.
Settings: Gesture rhyme mode, Bigram_weight=4.0, Hist_weight=0.5, Sequence_context=1–3.
Match — Rhythmic corpus synthesis
Use case: Replace drum hits or percussive sounds with corpus grains that match the rhythm.
Settings: Rhythmic preset (short analysis windows, high energy weight).
Draw mode — Brightness contour composition
Use case: Compose a piece by drawing brightness contours, using the corpus as a timbral palette.
Settings: Draw mode, choose Interactive (Demo window), set Draw duration and Grain rate, then shape the contour manually or with Sine / Triangle / Angular generators. Use Preview before Render when auditioning live alternatives.
Workflow: Gesture rhyme — Click train → speech syllables
Gesture source: A click train (rapid, decaying clicks).
Corpus: Folder of speech syllables (vowels, consonants).
Settings: Gesture rhyme mode, Bigram_weight=4.0, Hist_weight=0.3.
Result: The output is voiced by speech syllables whose token-transition structure matches the click train — the rhythm and kinetic shape of the clicks, but filled with vocalic content.
Workflow: Gesture rhyme — Bouncing ball → field recordings
Gesture source: A bouncing ball (decaying impacts, decreasing intervals).
Corpus: Field recordings (doors, footsteps, water drops).
Settings: Gesture rhyme mode, Sequence_context=2.
Result: The output voices the bouncing ball with field recordings that share the same kinetic structure — decaying impacts, decreasing time intervals — but using environmental sounds.
Workflow: Draw mode — Brightness glissando
Corpus: Synthesiser patches (bright to dark).
Settings: Draw mode, Interactive (Demo window), 8 s duration, 100 ms grain rate.
Contour: Start from the default 0→1 ramp or draw a new path; the window immediately shows corpus density and the nearest available brightness sequence.
Result: The output travels from darker to brighter regions of the corpus — a corpus-driven spectral glissando. Use Preview to audition the current path before Render.
• No corpus index found: Build corpus mode first, or run Match once to auto-build. Gesture rhyme, Corpus continuation tail and Corpus polyphonic layers never build the source corpus.
• Polyphonic mode requires mono: Select exactly one mono Sound; extract or convert one channel before running if needed.
• Polyphonic voices sound too similar: Increase Inter voice diversity or try Complementary / Independent; the available corpus still limits how much variety can be produced.
• Polyphonic voices too quiet or loud: Try Voice level mode = Match source active RMS and adjust Generated voice level dB. Normalize stereo mix controls the overall mixed peak, not each voice separately.
• Changing the Seed does not change every voice: Strongly constrained searches may select the same best grains. Try a different Relationship strength or mode; the same Seed and settings should reproduce the same result.
• DAW stems are missing: Enable Export aligned WAV stems, choose a writable folder, and check the Info window. The export uses suffixes such as
_set2 instead of overwriting existing files.• Polyphonic backend fails on Windows: Use the matching Praat v1.16.6 and Python v1.8.3 files together. Mode 6 passes its arguments through a file to avoid long command-line truncation.
• Continuation first run takes longer: The transition graph may need to be built from the existing grains; it is cached and reused for subsequent renders.
• Seed makes no audible difference: Tight mode is deterministic. Choose Textural; even there, a high Continuation strength can keep selecting the same best candidate. Lower the strength to explore more alternatives.
• Tail much weaker than source: Choose Tail level = Phrase level and Normalize = Loudness match tail to source, then peak (the default); this compensates for disparities in corpus recording levels.
• Continuation sounds jumpy: Try Tight or increase Continuation strength. The algorithm favors uninterrupted chronological passages and discourages rapid consecutive jumps, but results depend on available corpus material.
• Continuation does not run on stereo input: Select or convert to one mono Sound first.
• Gesture rhyme output sounds like timbre matching: Increase Bigram_weight (8.0–12.0) and decrease Hist_weight (0.1–0.3). The match should be driven by bigram transitions, not histograms.
• Gesture rhyme output is jerky / discontiguous: Increase Sequence_context (1–3) to smooth the match over preceding windows. This privileges sustained transition structure.
• Codec not installed: For EnCodec:
pip install encodec. For DAC: pip install dac. Use "mock" to test without torch.• Draw window has red horizontal regions: those brightness ranges contain no corpus grains; redraw through denser regions or rebuild the corpus with more varied material.
• Draw reports that brightness data is missing: the corpus index was built by an older version; rebuild / replace the corpus so per-grain spectral-centroid data is stored.
• Preview or Render pauses Praat: this is expected while the backend renders (and while Preview plays the result). Preview uses the same synthesis path as Render. On Praat 7.0, file-writing / subprocess operations may also trigger the normal trust prompt.
• Closing the Draw Demo window: closes the interactive Draw session and stops the script; use Cancel if you want an explicit exit.
Visualisation & TextGrid
sections and grains tiers. Polyphonic layers adds one source analysis tier and one poly voice N tier for each new voice. Both graph-based modes mark transitions S, C, J or F. Gesture rhyme records separate histogram_distance, bigram_distance, weighted_energy_distance and combined_distance fields in optional metadata. These distances are not simply additive contributions to a single score.