Putting it together: a synth to play

The earlier pages each built one way of making a sound: adding sine waves, carving harmonics away, modulating frequency, playing recordings and simulating the instrument. Each ended with something to play. But a sound isn’t yet an instrument. An instrument has to cope with ten fingers at once, answer the hands as well as the keys, and sit in a room. This page builds one synth with everything, and adds what the others left out.

Much of it has a history. The Minimoog (1970) played one note at a time. The Prophet-5 (1978) played five, and remembered its sounds; its makers worked out how to share five voices between ten fingers. Oberheim’s Matrix-12 (1985) gave the modulation matrix its name. Roland’s Juno synths had a chorus built in, and its JP-8000 (1996) brought the “supersaw”, seven detuned saws, which went on to define a lot of dance music.

It leans on:

Carving away harmonics, for oscillators, PolyBLEP, the ladder filter and envelopes. This synth reuses that page’s ladder filter and envelopes.
FM synthesis, for what happens when one oscillator moves another.
Designing IIR filters, for feedback and filters, and changing the sample rate for aliasing.
The first page’s instructions for playing the figures: every figure here has a keyboard, which the computer’s keys and a MIDI keyboard play too.

1 · Voices

A synth that plays several notes at once has several copies of its whole sound-making chain, oscillators, filter and envelopes: one for each note sounding. Each copy is a voice. When you press a key, the synth has to choose a voice to play it, and with more keys down (or still ringing) than voices, it has to take one back. That’s voice stealing.

This one picks a voice already playing that note, else a free one, else the quietest one that’s been let go (it’s fading anyway), and only then the one that started longest ago. Real synths differ in exactly this, and players notice.

Some sounds want one voice. Mono plays one note at a time, the newest key, and when you let go returns to the one still held. Legato is mono too, but a note played while another is held doesn’t restart the envelopes: it glides into the new pitch, as a singer or a trombone would.

Each box is a voice: its note, and its loudness envelope (the bar on its right). Play more notes than there are voices, or play quickly while the long release is still ringing, and watch the voices get taken back. With 4 voices, a fifth note takes the quietest of those already let go. In mono and legato there’s one box: hold a key, play another, let it go, and the first comes back.

2 · Unison: many voices on one key

One way to use spare voices is to stack them on the same note. Each unison copy is tuned a few cents away from the others and starts at its own random point in its cycle. They drift in and out of step, so the waveform never repeats exactly: the sound is wide and shimmering, as a section of violins is, where no two players are exactly in tune. Spread across the stereo field, the copies fill it. Seven saws spread like this are the supersaw.

Top: the copies added, on A3: the shape keeps changing. Bottom: close up on one harmonic, the 10th (2,200 Hz). Each copy puts its own line there, and lines a few hertz apart beat: they drift in and out of step that many times a second. With 3 copies over 20 cents, they’re 12.7 Hz apart at 2,200 Hz. Detuning is in cents, so the lines are further apart for higher harmonics, which beat faster: that’s the shimmer.

Copies whose phases are unrelated add up by their power, not their amplitude: 7 copies are about $\sqrt 7$ (2.6) times as loud as one, not 7. So each copy is turned down by $1/\sqrt n$ , and the volume stays about the same as you add them.

3 · Oscillators that act on each other

Each voice has two oscillators. Mixed, they make a richer wave. They can also act on each other:

Hard sync. Oscillator 2 restarts its cycle whenever oscillator 1 starts one. Its wave is cut short and starts again, at oscillator 1’s pitch, so the note keeps oscillator 1’s pitch, but its shape (its harmonics) is set by oscillator 2’s. Sweeping oscillator 2’s tuning makes the tearing, vocal sound of a sync lead.
Ring modulation multiplies the two. As the FM page showed for sidebands, a product of two tones has neither tone in it, only their sum and difference. Unless the tunings are simply related, those aren’t harmonics of anything: bells and clangs.
FM. Oscillator 2 pushes oscillator 1’s phase back and forth, as on the FM page, but with any waves, not just sines.

Hard sync makes a new jump in the wave each time oscillator 2 restarts, and as the subtractive page showed, a sharp jump makes harmonics above half the sample rate, which fold back as aliasing. The same PolyBLEP that smooths a saw’s own jump smooths this one too. It just has to work out the size of the jump and exactly when, between samples, it happens.

Top: three cycles of oscillator 1 (A4) alone, oscillator 2 alone, and what you hear. Bottom: its spectrum, with the aliases (lines that aren’t harmonics of A4) marked in pink. With sync on and oscillator 2 7.3 semitones up, smoothing the jumps takes the strongest alias below 8 kHz from −34 dB to −66 dB (switch Jumps to sharp to hear and see the difference). Try ring modulation and FM too, with the same tunings.

4 · Another kind of filter

The ladder filter is four one-pole stages with feedback round them. The state-variable filter (SVF) is the other classic: two integrators in a loop, the second fed by the first, both fed back. Its charm is that it gives low-pass, band-pass and high-pass at once, from different points in the same loop, so one filter can sweep between them. It’s written here in the same zero-delay-feedback form as the subtractive page’s ladder, so its cutoff and resonance stay where they’re put even near the top of the range.

Each filter at the same cutoff and resonance (the one chosen in bold, which the keyboard plays through). With no resonance, the SVF’s low-pass is 6 dB down at the cutoff and the four-pole ladder 12 dB: the ladder’s slope is steeper (24 dB an octave, to the SVF’s 12), and it starts falling sooner. Turn the resonance up and both grow a peak at the cutoff.

5 · The modulation matrix, and your hands

The subtractive synth had fixed routes: one LFO to the pitch, the filter or the loudness. A modulation matrix lets anything move anything. Each row of the matrix is a source (an envelope, an LFO, the key’s velocity, the mod wheel…), a destination (pitch, cutoff, loudness, pan, another LFO’s rate…) and an amount, which can be negative. This synth has six rows, two LFOs (LFO 1, one for each voice, and LFO 2, one for the whole synth) and a third envelope, just for the matrix.

Most of this is worked out every 16 samples (a third of a millisecond), not every sample: control rate. Nothing a hand or an envelope does changes faster than that, and it saves a lot of sums. The oscillators, the filter and the loudness envelope still run every sample.

The hands get their own sources. A MIDI keyboard sends:

Pitch bend: a spring-loaded wheel, sent as a 14-bit number (16,384 steps), so a slow bend doesn’t step. It bends every note by up to its range (2 semitones here, unless you change it).
The mod wheel (controller 1), which stays where it’s left. Usually it brings in vibrato: here, the matrix’s “LFO 1 × mod wheel” source.
Aftertouch: how hard a key is pressed after it’s down. Most keyboards send one value for the whole keyboard (“channel pressure”); some send one per key. Here it’s another source, set to open the filter.

1 → amount 2 → amount 3 → amount LFO 1 rate LFO 1 shape Attack Decay Sustain Pitch bend Mod wheel Aftertouch

What the matrix does to one note, C4, held for 1.5 seconds: its pitch, the filter’s cutoff, and its loudness on each side. As set up, the mod wheel (half way) brings in a vibrato of ±0.3 semitones, the mod envelope opens the filter and lets it fall back, and LFO 2 pans the sound slowly from side to side. Change the rows, move the bend, wheel and aftertouch sliders (or a MIDI keyboard’s), and play.

6 · Drive, and the aliasing it makes

Drive pushes the sound into a curve that squashes its peaks: here, $\tanh$ , which rounds a sine towards a square. That’s distortion, and it adds harmonics, which is the point. But a curve applied to each sample makes harmonics far above half the sample rate, and they fold back as aliases, and unlike a saw’s, they’re not at a harmonic of the note.

The usual cure is to work at a higher sample rate (oversampling). A cheaper one is antiderivative anti-aliasing (ADAA). Instead of the curve’s value at each sample, it outputs the curve’s average over the stretch between this sample and the last: the change in the curve’s antiderivative ( $\ln\cosh$ , for $\tanh$ ), divided by the change in the input. Averaging over a stretch is a gentle low-pass filter applied before the sample is taken, which is what sampling needs.

Top: a sine on A6 (1,760 Hz), driven hard. Bottom: its spectrum: the harmonics, and in pink, the aliases. With ADAA on, the strongest alias below 8 kHz falls from −65 dB to −81 dB. The strongest alias anywhere improves less (from −44 dB to −49 dB): it’s close to half the sample rate, where ADAA’s averaging filters least. On lower notes there’s far less to fold back: on A4, the aliases are below −95 dB either way.

7 · Chorus

A chorus adds a copy of the sound, delayed by a few milliseconds, with the delay slowly swaying. A delayed copy added to the sound makes a comb filter: at some frequencies the copy arrives in step and adds, at others it arrives upside down and cancels. As the delay sways, the notches sweep up and down, and the copy’s pitch rises and falls a touch (a delay that’s shrinking plays the sound slightly faster). Done a little differently on each side, it sounds like several players, and the sound fills the stereo field.

Top: the two sides’ delays, a quarter of a cycle apart, swaying around 8 ms. Bottom: the left side’s comb now: the sound plus its delayed copy, notches where they cancel. Watch them sweep as the delay changes.

8 · Echo

An echo is a longer delay, with feedback: some of what comes out goes back in, so each repeat brings another, quieter one. Real tape echoes lost some top on each trip, so here a low-pass filter in the loop (tone) darkens each repeat a bit more. Ping-pong sends each repeat to the other side.

Each repeat of a click, left (up) and right (down): its energy, on a scale where a bar’s full length is the first repeat and a bar 50 dB shorter would vanish. Each trip round loses the feedback’s share, and the tone filter takes a little more.

9 · Reverb

A room answers a sound with thousands of reflections, arriving closer and closer together until they’re a smooth, decaying wash. A feedback delay network (FDN) imitates that with a handful of delay lines (eight here, of different lengths, so their echoes don’t line up), whose outputs are mixed and fed back into all of their inputs. The mix is a Hadamard matrix: every line feeds every other, and no energy is gained or lost in the mixing. So the echoes multiply, and how fast the sound dies is set by a gain in each line: each trip round a line of length $L$ samples takes $60 L / (T_{60} \cdot f_s)$ decibels off, so the sound falls 60 dB in $T_{60}$ seconds, the reverberation time. A low-pass filter in each line (damping) does what air and soft walls do, taking the top off faster.

Top: the reverb’s answer to a click, its first 400 ms: separate echoes at first, then too many to tell apart. Bottom: how its energy falls (the energy still to come, at each moment, as Schroeder measured rooms), below 1 kHz and above 4 kHz. Asked for 2.5 seconds, with the damping at 6 kHz, the lows take 2.44 s to fall 60 dB and the highs 1.12 s. With the damping off (20 kHz), both take within 0.1 s of what was asked.

The Hadamard matrix turns up a long way from audio. Its smallest form, $\tfrac{1}{\sqrt2}\left(\begin{smallmatrix}1 & 1\\ 1 & -1\end{smallmatrix}\right)$ , is the Hadamard gate of quantum computing, and the 8 × 8 one here is three of those combined (each step doubles it, with the copies in the bottom-right corner turned upside down). In a quantum computer it spreads a single state evenly across all of them; here it spreads a click in one line evenly across all eight, and that’s why the echoes multiply so fast. Both rely on the same property: the matrix loses nothing. A quantum gate has to keep the probabilities adding up to 1, and the reverb’s mixing has to keep the energy, so that only the lines’ gains decide how quickly the sound dies.

There’s a trap in building one: each line has to be read at a whole number of samples. Reading between two samples (interpolating) is a gentle low-pass filter, and in a loop gone round dozens of times a second, it adds up: the highs would die sooner, as if damped, whatever the damping was set to.

10 · The arpeggiator

An arpeggiator plays the keys you hold one at a time, in a pattern, in time: up, down, up and down, at random, or in the order you pressed them, over one or more octaves. The gate is how much of each step the note sounds. Latch keeps it going after you let go, until you play a new chord.

The first bar the arpeggiator plays, for the keys you’re holding (or C, E and G, until you hold some), as a piano roll. At 120 BPM and 4 notes a beat, each step is 125 ms. It runs inside the synth, sample by sample, so its timing doesn’t depend on what the rest of the page is doing.

11 · The synth

Everything at once. Choose a preset or start from Plain, and change anything. The patch bar saves your sounds in this browser, copies them as a code or a link to share, and opens them again. Link knobs (on the keyboard’s bar) ties a MIDI controller’s knobs to any slider, as on the other pages.

Preset

Your hands (a MIDI keyboard’s wheels and pressure move these)Pitch bend Mod wheel Aftertouch

OscillatorsWave 1Wave 2Osc 2 tuningOsc 2 fineOsc 2 mixPulse widthSub oscNoiseJumps

Oscillators togetherHard syncRing modFM (2 → 1)

UnisonCopiesDetuneWidth

FilterKindCutoffResonanceDriveEnvelopeKey follow

Filter envelopeAttackDecaySustainRelease

Loudness envelopeAttackDecaySustainReleaseVelocity

Mod envelopeAttackDecaySustainRelease

LFOsLFO 1 rateLFO 1 shapeRestart on noteLFO 2 rateLFO 2 shape

Modulation matrix1→amount2→amount3→amount4→amount5→amount6→amount

PlayingModeVoicesGlideBend range

Drive and chorusDriveAnti-aliasingChorusChorus rateChorus depth

EchoEchoTimeFeedbackTonePing-pong

ReverbReverbSizeDecayDamping

ArpeggiatorPatternNotes a beatOctavesGateTempoLatch

OutputLevel

Every setting, grouped as above. With a MIDI keyboard, the pitch bend wheel, mod wheel and aftertouch move the “your hands” sliders. Each preset uses the mod wheel for vibrato and aftertouch to open the filter, except where it says otherwise in its matrix.

Summary

An instrument shares out voices between the notes, and steals them back when it runs out; mono and legato play one note at a time, legato gliding between them.
Unison stacks detuned copies on each note: their beating makes the shimmer, and $n$ copies are $\sqrt n$ times as loud.
Two oscillators can act on each other: hard sync, ring modulation and FM. Sync’s jumps alias unless they’re smoothed, like any other.
The state-variable filter gives low-pass, band-pass and high-pass from one loop.
A modulation matrix connects any source to any destination; the hands (bend, mod wheel, aftertouch) are sources too. Most of it runs at control rate.
Drive aliases; ADAA cuts the low aliases cheaply. Chorus is a swaying short delay (a moving comb), echo a long one with feedback, and reverb a network of delays mixed and fed back, decaying in a set time.
The arpeggiator plays held notes in a pattern, in time.