RL Rupture Agent — User Guide

What this does

RL Rupture Agent is a reinforcement-learning-driven sound transformation tool that learns how to alternate between continuity and rupture in an existing sound. It analyzes the source’s amplitude envelope and converts its local shape and motion into a sequence of states that describe how the sound is evolving over time.

A tabular Q-learning agent learns a policy for these states: when to preserve the original sound and when to introduce one of three rupture gestures — Stutter, Stutter + Reverse, or Stutter + Soft Clip. The learning process therefore creates a transformation strategy that responds to the temporal behavior of the source rather than placing effects at fixed or purely random positions.

The process has three stages:

1. Envelope analysis — the source is divided into short frames and described in terms of envelope level, direction of change, and recent continuity or rupture history.
2. Reinforcement learning — the agent repeatedly explores this state sequence, updating a Q-table according to the desired balance between continuity and rupture and the suitability of different rupture gestures for different envelope contexts.
3. Audio rendering — the learned policy is applied to the source. Keep regions remain continuous, while selected regions become short buffer-based gestures using repetition, reversal, or soft saturation.

The result is a source-responsive temporal recomposition: the original recording remains the material, while the learned policy determines where its continuity is maintained and where it is deliberately fractured.

Processing pipeline

Selected Sound
   ↓
Mono analysis copy (multichannel input only)
   ↓
Absolute value → 0–20 Hz Hann low-pass envelope
   ↓
Envelope velocity → Continuity / Moderate / Rupture class
   ↓
36-state Markov representation
   ↓
Tabular Q-learning over fixed source analysis
   ↓
Final greedy action sequence
   ↓
Merge consecutive identical rupture actions into runs
   ↓
Choose one source block per run
   ↓
Loop / reverse / soft-saturate as required
   ↓
Crossfade rupture with untouched original
   ↓
Output Sound

Frame duration is quantized to a whole number of samples before analysis and rendering. This keeps frame boundaries and the sample-level action lookup aligned exactly.

Quick start

1. Select exactly one Sound object.
2. Run RL_RuptureAgent.praat.
3. Choose one of the four musical presets, or Custom.
4. For Custom, set the frame duration and the target rupture ratio first; these have the clearest structural effect.
5. Use Lookback window and Stutter block duration to control where repeated material comes from and how long the repeated grain/block is.
6. Use a non-zero Random seed when you want to reproduce a particular render.
7. Leave Draw summary enabled while designing a preset: the plots show the state sequence, realised actions, rupture runs, source-block locations, and training curve.

State representation

The agent uses 36 discrete states. Each state combines three observable properties:

The total is therefore 3 × 4 × 3 = 36 states. Unlike a simple envelope-class lookup, this state can distinguish, for example, a Rupture-class frame reached after a long calm stretch from the same envelope class reached inside an already long rupture run.

Envelope classes

For analysis, multichannel input is downmixed to mono. The mono copy is rectified with abs(self) and low-pass filtered with Praat’s Filter (pass Hann band) from 0 to 20 Hz. The envelope is sampled once at the center of each analysis frame.

The automatic threshold tau is the mean absolute frame-to-frame envelope change. Classification is:

Continuity : |Δ envelope| < tau
Moderate   : tau ≤ |Δ envelope| < 2 × tau
Rupture    : |Δ envelope| ≥ 2 × tau

To prevent an almost static signal from being classified from numerical/filter ripple alone, tau has a floor equal to 0.0005 × maximum envelope.

Actions & rupture runs

How a rupture run is formed

Consecutive frames merge into one rupture run only while they carry the same non-Keep action. A change from Stutter to Reverse or Clip starts a new run, so all three learned flavours can be heard independently.

Normally, one block is chosen randomly from within the Lookback window behind the start of the run. If a run begins too close to the beginning of the Sound to provide a full historical block, the run loops its own opening block instead of collapsing to a one-sample or extremely short repeat.

The requested stutter duration is converted to samples and given a hard musical floor of about 4 ms. The script also constrains block length and read position so all source reads remain inside the original Sound.

Crossfades

Each loop wrap uses a linear crossfade between the end and beginning of the source block. Each rupture run is also faded against the dry source at its entry and exit. These fades reduce hard splice discontinuities; they do not constitute an equal-power crossfade.

Crossfade ms is limited to at most one quarter of the stutter block duration and is shortened further when a rupture run itself is very short.

Soft Clip flavour

Stutter + Clip is intentionally a distortion flavour, but v2.1.2 uses progressive saturation rather than hard clipping. Samples up to 55% of the original source peak pass unchanged. Above that knee, the remaining range toward the original peak is compressed with a tanh curve. This keeps the action audible without producing a flat clipped plateau.

Q-learning & reward

The script uses a tabular Q-table with 36 rows × 4 actions. During each episode it walks the fixed envelope-class sequence from beginning to end and updates one state–action value at each frame with the standard discounted Q-learning form:

Q(s,a) ← Q(s,a) + α [ r + γ max Q(s′,a′) − Q(s,a) ]

Learning rate is α and Discount factor is γ. Exploration uses epsilon-greedy action selection. The episode epsilon decreases linearly from the requested Epsilon toward zero across training, and the final render is produced by a fully greedy rollout.

What the reward encourages

• Keep the running rupture ratio close to Target rupture ratio.
• When the ratio is below target, give a small bonus to rupture actions.
• When the ratio is above target, give a small bonus to Keep.
• Reward a rupture after a long continuity streak.
• Reward a return to Keep after a long rupture streak.
• Penalize extending a rupture when already deep in a long rupture streak.
• Add a mild envelope alignment: Continuity slightly favors Keep; Rupture-class frames slightly favor rupture.
• Add a deliberately small flavour preference: Stutter is favored in Continuity-class frames, Reverse in Moderate frames, and Clip in Rupture-class frames.

What “learning” means here

The environment is not listening to or re-analyzing each processed render. All episodes use the same source envelope-class sequence; only the Q-table and the agent’s memory variables evolve. This makes the process fast and deterministic under a fixed random seed, while keeping the learned policy dependent on continuity/rupture history and the running density.

Presets

Presets 2–5 override the learning and rupture parameters shown below. Random seed, Draw summary, and Play result remain user-controlled.

The names describe the structural tendency of the parameter set, not a guaranteed percentage or exact event pattern. Random exploration and random source-block placement can change the result when Random seed = 0.

Parameters

Randomness & reproducibility

Randomness enters the process in two places:

• Training exploration: epsilon-greedy episodes sometimes choose a random action instead of the current greedy one.
• Rupture synthesis: each rupture run chooses a source-block start randomly from its allowed lookback range when enough history exists.

With Random seed = 0, the script does not initialize Praat’s generator and successive runs may differ. With a non-zero seed, both the learning exploration and block-placement sequence are reproducible under the same script, source, settings, and Praat random-number implementation.

Input & output behavior

Visualization

When Draw summary is enabled, the script draws a compact diagnostic page with three main panels and a summary strip.

1. Envelope and envelope-class states

The analyzed envelope is drawn as a line. Background bands identify the fixed per-frame class: green for Continuity, amber for Moderate, and red for Rupture. The horizontal axis is relative time from 0 to source duration, even when the Sound itself has a non-zero xmin.

2. Realised actions and rupture runs

Each frame is colored according to the final greedy action: grey = Keep, black = Stutter, blue = Stutter + Reverse, red = Stutter + Clip. Green marks and lines show where each rupture run copied its looped source block from.

This panel represents the rendered run structure, not merely the envelope state. Consecutive non-Keep frames of different flavours appear as separate runs.

3. Greedy rupture ratio during training

The blue training curve periodically evaluates the current Q-table greedily. The dashed green line shows the requested target rupture ratio. The curve is useful for seeing whether training is moving toward a stable density, but it is not a formal convergence test.

Summary strip

The bottom strip reports class percentages, tau, number of visited Markov states, action counts, rupture-run count, block and crossfade duration, final versus target rupture ratio, output peak, and seed.

Notes & limitations

• Offline only: the complete Sound is analyzed and trained before synthesis; this is not a real-time agent.
• The envelope does not change during learning. The RL policy learns from a fixed source analysis, not from the acoustic consequences of previous actions.
• Target density is soft. Reward shaping encourages a ratio, but does not impose an exact number of rupture frames.
• Frame duration changes the problem itself. It changes temporal resolution, streak buckets, number of decisions, and the granularity at which the target ratio can be approached.
• Action flavours are learned weakly. Placement/density rewards deliberately dominate; the small class-specific flavour bonus only breaks the former equivalence between the three rupture actions.
• Analysis is shared across channels. A stereo/multichannel source keeps its channels during synthesis, but all channels receive the same temporal action schedule.
• Clip is intentional saturation. It is softer than the earlier hard-clipping version, but it remains a deliberate distortion colour.

Further reading

• Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8, 279–292. doi:10.1007/BF00992698. The primary reference for the tabular Q-learning update used by the script.
• Praat Manual: Formulas and Formulas for modification. Direct documentation for the whole-Sound Formula operations used in the synthesis passes.
• Praat Manual: Sound. Reference for Sound channel/sample data and time-domain metadata such as xmin, xmax, sample count, and sampling interval.