Recorded sounds: samples and wavetables

The first three pages in this series made sounds from nothing: adding sine waves, carving away harmonics, and modulating one oscillator with another. Each could imitate real instruments only so well, and getting close took a lot of skill. This page takes the direct route. To sound like a piano, record a piano. A sampler plays recordings (“samples”) back from the keyboard, at whatever pitch each key asks for.

The idea is older than digital audio. The Mellotron, from the 1960s, had a strip of tape for every key: press a key, and a recording of a real flute or violin played, for about eight seconds before the tape ran out. Digital samplers arrived around 1980, with the Fairlight CMI and then the E-mu Emulator, at prices only studios and stars could afford. By the mid-1980s home computers could do it too: the Amiga’s sound chip played four channels of samples, and that’s how a generation of computer music was made, which a later page is about.

The second half of the page is about a cousin of sampling: the wavetable, where instead of a whole recording, each note plays one tiny cycle of a wave, chosen from a table of many.

It leans on:

Pitch shifting and time stretching, which explains how a sampler plays a recording faster or slower, and what goes wrong.
Carving away harmonics, for aliasing and envelopes.
The first page’s instructions for playing the figures: the keyboard, the computer’s keys, MIDI, and linking knobs.

1 · A sample is a recording

A sample plays at its own speed on one key, its root note. To play it on other keys, the sampler plays it faster or slower: twice as fast is an octave up, half as fast an octave down, and each semitone is $2^{1/12}$ times faster than the one below. Between the original samples, the sampler has to work out values that weren’t recorded, which is interpolation; the pitch-shifting page compares the ways of doing it.

Choose a sound, or make your own: Record one from your microphone (up to 4 seconds: sing a note, whistle, clap, anything), or open a sound file. The figure finds a pitched sound’s root note itself, by pitch detection; you can change it. Then play it from the keyboard. Recordings stay in your browser; nothing is uploaded.

The built-in sounds are all synthesised on this page, so there’s nothing to download: a sung “ah” (made the way the voice coding page makes vowels), a plucked string, a bell (made by FM), and a drum. Notice how each note plays for a different length: an octave up, the sample plays twice as fast, so it lasts half as long.

Play the voice over a few octaves and it goes wrong quickly. High up it turns into a chipmunk; low down, a monster. Speeding a recording up moves everything up: not just the pitch but also the voice’s formants, the resonances of the singer’s throat and mouth that make an “ah” an “ah”. A real singer’s formants stay put whatever note they sing.

2 · One sample, many notes

Sample libraries get round this by brute force: multisampling. They record the instrument on many notes, every few semitones, and each key plays the recording nearest to it, so no sample is stretched far. A good piano library records every note, at several loudnesses (since a piano played hard doesn’t just sound louder, but brighter), and even several takes of each, so repeated notes don’t sound mechanical.

Top: the keymap: which recording each key plays (the dots mark where each was made). Bottom: the harmonics of the chosen key’s note, as the sampler plays it (blue), against where a singer’s “ah” formants are (dashed). When the sample is stretched, its formants move with it (orange). Play the keyboard in either mode.

With one sample, sung on A3, E4 plays it 7 semitones faster, and its formants move up by ×1.50; at the top of the keyboard they’re more than two octaves out. With a sample every half-octave, no key is more than 3 semitones from its sample: E4 plays the D♯4 sample 1 semitone faster, and its formants move by only ×1.06.

The cost is memory, which was the limit for decades: an early sampler might have a few hundred kilobytes, and a good piano library today runs to many gigabytes.

3 · Holding a note: loops

A recording lasts as long as it lasts. To let a note sustain for as long as the key is held, a sampler loops part of it: when playback reaches the loop’s end, it jumps back to the loop’s start, again and again. The trouble is the jump. Unless the sound just before the end matches what leads into the start, the wave breaks at the join, and the loop clicks or buzzes, once per trip round it.

A sung “ah” on A3, 1.2 s long. Top: the sample, and its loop (shaded). Bottom: the join, four cycles either side of the jump: what the loop plays (orange) against what leads into the loop’s start in the recording (grey). Where they differ, the loop jumps. The score is the difference over the last cycle, against the sound’s own level: 0 is seamless, and about 1.4 is no better than joining two random cycles. Hold a note plays it for 3 s; the keyboard plays it too.

As set at first, the loop is 66.66 cycles long, so it jumps part-way through a cycle: it scores 1.29, and it buzzes. Whole cycles trims it to exactly 67, which helps (0.87), but not entirely: a real voice’s pitch wanders a little, as this one’s does, so even a whole number of cycles doesn’t quite meet. A 50 ms crossfade fixes it: the end of the loop fades into a copy of what leads into the start, so the jump lands where the sound already is, scoring 0.09.

Crossfades are a sampler’s standard trick. They work on sounds whose pitch drifts, and on sounds with no pitch at all, such as a breathy flute or a string section, where there’s no cycle to line up.

4 · Wavetables

Shrink a loop all the way down to a single cycle of a wave, and it’s no longer a recording: it’s an oscillator with whatever wave shape you like. A wavetable synthesiser keeps a table of many such cycles, one after another, usually changing gradually from one to the next, and each note plays from a position in the table. Move the position during the note, and the tone changes: smoothly, if the sampler blends neighbouring waves.

Wolfgang Palm’s PPG Wave (1981) made this famous: digital wavetable oscillators feeding analogue filters, with a sound somewhere between a sampler’s and a classic synthesiser’s. Most software synthesisers today include wavetables, often with hundreds of waves per table, drawn by hand or cut from recordings.

Top: the table, its waves stacked from the first (front) to the last (back), with the wave at the position in orange. Bottom: that wave’s harmonics. Choose how each note moves through the table, and play.

Each table has 32 waves of 2,048 samples, and the sound blends neighbouring waves as the position moves. “Sweeps through the table” moves the position from the first wave to the last over two seconds, which on the resonant-peak table sounds like a filter sweep, and on the pulse table like pulse-width modulation. Unlike a filter, a table can move between any shapes at all.

5 · Wavetables alias too

A wave stored in a table has a fixed set of harmonics: a sawtooth cycle of 2,048 samples holds up to 1,023 of them. Played on a low note, that’s fine. On a high note, most of those harmonics land above half the sample rate, and fold back as aliases, exactly as the naive sawtooth did on the subtractive page.

The fix is to keep several copies of each wave, each with fewer harmonics: one keeping 1,023 harmonics, one keeping 512, one 256, and so on, halving each time: one copy per octave. Each note reads the fullest copy whose harmonics all fit below half the sample rate. (Graphics programmers know the same trick for textures, as mipmaps.) It costs a little memory, and nothing at all while playing.

The spectrum of a note played from the table, at 44.1 kHz. Blue dots mark the note’s real harmonics; everything else is aliasing. The shaded band, below 5 kHz, is where the ear is most sensitive. Tick band-limited copies to switch from the full table to the copy for the note. Sweep up plays the table gliding from C3 to C8.

On C7, with the position two-thirds of the way through the first table (the sawtooth), the full table’s loudest alias below 5 kHz is 25.5 dB below the note: as bad as the naive sawtooth on the subtractive page. Reading the band-limited copy (level 7, which keeps harmonics up to the 8th), nothing rises above the analysis’s own floor, 92.7 dB down. On the resonant-peak table, whose waves are mostly high harmonics, the full table’s aliases can be louder than the note itself.

6 · Samples on home computers

Samples reached millions of people through home computers and games. Two machines shaped the sound:

The Amiga (1985) had a sound chip, Paula, that played four channels of 8-bit samples straight from memory, each at its own speed. It changed pitch by changing how fast it fetched the samples, with no interpolation at all, and with half a megabyte or so of memory for everything, samples had to be short and coarse. That gritty sound is part of the music made with trackers, which the next page in this series takes apart.
The Gravis UltraSound (1992), a PC sound card, had its own memory for samples (up to a megabyte) and could play up to 32 of them at once, interpolating between samples in its own hardware. It sounded far smoother than FM cards like the AdLib, and it was a favourite of the demoscene, the community of programmers and musicians making real-time audio-visual “demos”.

After that, sound cards stopped needing a sampler at all: computers became fast enough to mix samples themselves, and send the result out as plain digital audio.

7 · A whole sampler

Here’s a sampler with everything on this page. Use a built-in sound, record your own, or open a file; set its root note, a loop with a crossfade, the interpolation, and the attack and release. It plays up to 16 notes at once.

The sample, with its loop shaded. Try recording a sung or whistled note, looping its steady middle with whole cycles and a crossfade, and playing chords with it. Interpolation chooses how the sampler works out values between samples: none (drop-sample, as on the Amiga), a straight line (linear, as on the Gravis), or a smooth curve through four samples (cubic). The difference is clearest on a bright sound played well below its root. Download the sample saves the sound itself as a WAV file.

What samples can’t do is respond: a recording of a string plucked once sounds the same every time, however you play it. The next page in this series makes the instrument itself, in arithmetic: physical modelling, where the string vibrates.

Summary

A sampler plays recordings from the keyboard, faster or slower to change the pitch: each semitone is $2^{1/12}$ times faster. Speeding up shortens the sound and moves its formants, which is why one sample can’t cover a keyboard.
Multisampling records the instrument every few semitones (and at several loudnesses), so no recording is stretched far: with one every half-octave, nothing moves more than 3 semitones.
Loops sustain a note. A loop that jumps part-way through a cycle buzzes; whole cycles help, and a crossfade fixes even sounds whose pitch wanders.
A wavetable is a table of single cycles; moving through it changes the tone, like a filter but between any shapes.
Tables alias on high notes; keeping band-limited copies, one per octave, removes the aliasing at almost no cost.