DDSP Neural Drawing Synthesizer

Drawing → sound

The drawing remains a reusable control representation. Transformations are applied before or during conversion to DDSP frames; the neural model is the renderer, not the editor.

What this does

DDSP Neural Drawing Synthesizer is a gesture-driven neural instrument inside Praat. Instead of analysing an existing recording, it lets you construct the conditioning signals directly: a pitch trajectory f0_hz and a loudness trajectory loudness_db. These are sent to one of five pretrained DDSP solo-instrument models: Violin, Flute, Flute2, Trumpet, or Tenor Saxophone.

The result is not conventional MIDI synthesis and not audio-to-audio timbre transfer. The user composes a continuous control gesture first, then DDSP interprets that gesture through the learned behaviour of an instrument model.

Quick start

1. Run DDSPNeuralDrawingSynthesizer.praat. No Sound object needs to be selected.
2. Choose a duration and target model.
3. Click in the PITCH panel to place points. Consecutive clicks form a stroke.
4. Click in the LOUDNESS panel to shape dynamics and articulation.
5. Press Preview (or P) to hear the gesture as a fast harmonic sketch and to display the processed orange overlay.
6. Press Synthesize (or S) to render the gesture with the selected DDSP model.
7. The result appears as a new Praat Sound object, for example DDSP_Drawing_Violin.

Useful shortcuts: M changes model, K changes pitch mode, Z undoes, Space lifts the pen, G toggles single-gesture mode, and Q closes the canvas.

Drawing the gesture

Praat's Demo window exposes mouse clicks but not continuous mouse motion while a button is held, so the interface uses click-to-draw. Each click adds a breakpoint; consecutive clicks in the same panel are joined into a stroke. Redrawing over a time region replaces the existing points in that region.

Lift pen: gaps, silence and leaps

Press Space or click Lift pen before the next point to start a new stroke. In the loudness panel, the gap becomes silence. In the pitch panel, the previous pitch is held through the gap and the next stroke begins as a pitch leap. Short tapered edges prevent clicks and unstable transitions.

Save and reload gestures

Gestures can be saved independently of the rendered audio. The format is plain text and human-readable:

format ddsp_drawing_gesture 1
duration 5.000000
[PITCH]
0.000000 262.0000 1
0.700000 392.0000 0
[LOUDNESS]
0.000000 -60.000 1
0.700000 -24.000 0

The third column is a break flag: 1 means that the pen was lifted before that point. A saved gesture can later be loaded completely, or only its pitch or loudness component can be fitted to the current canvas.

Controls

Advanced settings

Gesture transformations

Transforms are applied before or during conversion to DDSP control frames. Multiple transforms are separated by semicolons.

Example chain:

transpose 7; contour 1.4; vibrato 5.2 22; perturb 8 3 2

Single Gesture mode

Single Gesture mode is an experimental one-line interface. The user draws only the pitch trajectory. Loudness is then derived from the speed of pitch motion: faster semitone movement produces greater excitation, while slow or static regions become softer. Motion depth controls the resulting dynamic range.

This mode deliberately treats the drawing as a physical gesture rather than as two independent automation lanes. It can produce results that are less predictable but often more gesturally coherent.

Theory: from drawing to DDSP

The tool separates the user's gesture from the neural renderer. This is the important architectural idea:

1. Gesture representation: sparse breakpoint curves are stored in time, Hz and dB.
2. Symbolic transformation: transpose, inversion, retrograde and contour operations act on the breakpoint representation.
3. Control rendering: the gesture is sampled at the model's feature rate and converted to f0_hz, loudness_db, voicing and edge tapers.
4. Frame processing: smoothing, scale quantisation, glide, vibrato and stochastic perturbation shape the final conditioning.
5. DDSP inference: the pretrained Autoencoder is restored and receives the generated features directly. A silent reference audio tensor is supplied only to satisfy the model interface; no source audio is analysed.
6. Output: silence gating, output-level handling and optional resampling produce the final WAV loaded into Praat.

Model geometry

During full synthesis, the engine reads the checkpoint's operative GIN configuration to determine the training time-step count, synthesis sample count, hop size and sample rate. The drawn controls are then rendered at that model-specific feature rate rather than assuming a fixed geometry.

Preview versus neural synthesis

Preview uses NumPy only. It turns the processed controls into a simple additive harmonic sketch, so interaction is quick and does not require loading TensorFlow or a DDSP checkpoint. Synthesize performs the actual neural render. In engine v0.1.1, silent regions and fully silent gestures are multiplied by the voicing taper and therefore produce exact digital silence in the preview as well.

Applications

Drawn melodic synthesis

Sketch a continuous pitch phrase and a dynamic envelope, then audition the same gesture through Violin, Flute, Trumpet and Tenor Saxophone. The gesture stays conceptually stable while the learned instrument behaviour changes.

Microtonal contour composition

Use Continuous pitch mode with little or no smoothing to draw glissandi, bends, unstable intonation and non-tempered contours. The result is driven directly by the trajectory rather than by discrete MIDI notes.

Quantised drawings

Switch the same hand-drawn contour to Chromatic, Major, Minor or Pentatonic mode. The processed orange overlay shows the exact pitch path the model receives, including any glide between quantised values.

Gesture variation

Save one drawing and generate a family of related versions using inversion, retrograde, contour expansion, transposition, vibrato and perturbation. The gesture file becomes reusable compositional material independent of the final sound.

Articulation by drawing

Use loudness strokes and Lift pen to design note boundaries directly. This is particularly useful with sustained target models, because articulation is specified before synthesis rather than inferred from an input recording.

Honest limitations

• Click-to-draw, not continuous freehand: Praat's scriptable Demo window does not expose continuous mouse-drag motion.
• Out-of-range gestures: the pretrained models were learned from real solo-instrument recordings. Extreme pitch or loudness trajectories may still render but can produce degraded or unstable timbre.
• Native bandwidth: the standard solo-instrument checkpoints synthesise at 16 kHz. Writing the result at 44.1 or 48 kHz does not create new content above approximately 8 kHz.
• Reverb tail: output duration equals the drawing canvas. If a model's learned reverb should decay naturally, end the loudness gesture before the right edge and leave time for the tail.
• Not real time: full DDSP synthesis includes model preparation and neural inference. Preview exists specifically to make gesture editing faster.
• Model identity is fixed: the five supplied targets are separate pretrained checkpoints; the tool is not a continuous latent timbre morphing system.

Requirements

Preview: Python with NumPy is sufficient.

Full synthesis: the same Python environment used by DDSP Neural Revoicing, including TensorFlow, DDSP, CREPE and the supporting dependencies. CREPE is present because the DDSP package imports it, but this tool does not run pitch tracking.

The first use of a target model may require internet access so the shared DDSP cache can obtain the pretrained checkpoint. Subsequent renders use the cached model locally.

Praat 7 and later may ask for trust because the script writes temporary files and launches the local Python process.

Troubleshooting

Citation & further reading

Praat AudioTools: Cohen, S. (2026). Praat AudioTools: An Offline Analysis–Resynthesis Toolkit for Experimental Composition.

• Praat AudioTools repository
• Magenta DDSP
• Engel, J., Hantrakul, L., Gu, C., & Roberts, A. (2020). DDSP: Differentiable Digital Signal Processing. ICLR 2020.
• Magenta DDSP Timbre Transfer