Inside an MP3 encoder

Record a few seconds of sound, or pick a demo signal. Then follow it through every stage of an MPEG-1 Layer III encoder’s front end, from raw samples to the quantised numbers that get packed into an MP3 file. Everything runs in your browser, in plain TypeScript written to be read.

1 · Choose some audio

Load a file… Normalise to −1 dBFS

2 · Encoder settings

Bitrate

Blocks

Psychoacoustic noise shaping

3 · Listen

Loading…

The big picture

An MP3 encoder is a lossy encoder. It doesn’t try to keep the waveform. It tries to keep what you can hear of it. It has two parallel paths:

The signal path turns the audio into a frequency-domain representation suited to coding: 576 real numbers per granule.
The psychoacoustic path analyses the same audio separately and works out, for each frequency band, how much noise could be added without anyone noticing.
Quantisation is where they meet. The 576 numbers are rounded to integers, as coarsely as the bit budget demands, and the rounding noise is steered into the bands where the psychoacoustic model says it will be masked.

Each section below covers one block, using your audio. Use the bar above to choose which granule (576-sample slice) you’re looking at.

1 · Frames and granules

MP3 works on frames of 1152 samples (26 ms at 44.1 kHz). Each frame splits into two granules of 576 samples. The granule is the real unit of work: block switching, the MDCT, psychoacoustic analysis and quantisation all happen once per granule. The frame is mostly a packaging unit: it shares one header and one block of side information.

The transforms overlap, though, so “the samples in granule g” is an approximation. The MDCT for a granule covers 1152 input samples, half of them shared with each neighbour. The polyphase filter in front also looks 512 samples into the past. Together these produce a fixed encoder delay of 528 samples: the spectrum of granule g is centred 528 samples earlier than the granule’s nominal middle. Every MP3 encoder has this delay. It’s why MP3 files carry a little silence at the start, and why gapless playback needs metadata to trim it.

The selected granule’s frame. Thin lines mark granule boundaries and thick lines mark frame boundaries. The shaded hump is the MDCT window for the selected granule, which is where its 576 coefficients actually come from. Notice how it overlaps both neighbours.

2 · The polyphase filterbank

First stop: a bank of 32 band-pass filters, each 689 Hz wide, inherited from MP3’s older siblings, Layers I and II. All 32 filters are frequency-shifted copies of one 512-tap low-pass prototype. Each copy is made by multiplying the prototype by a cosine at the band’s centre frequency.

After filtering, each subband only contains 689 Hz of bandwidth, so it can be decimated by 32: keep every 32nd sample. That makes the transform critically sampled: 32 input samples in, 32 subband samples out (one per band). Nothing is gained or lost in count, it’s just reorganised by frequency. Per granule, each subband gets 576 / 32 = 18 samples.

The prototype: one ordinary FIR low-pass

Each subband filter is a plain FIR filter, $y[n] = \sum\nolimits_m h[m]\,x[n-m]$ , with 512 coefficients (“taps”). They all derive from one prototype $p[n]$ : a low-pass filter centred on 0 Hz. Its coefficients are real, so its response at −f mirrors its response at +f. The passband is therefore a single bump from −344.5 Hz to +344.5 Hz, 689 Hz wide counting both sides. That’s exactly one subband.

The coefficients are symmetric about tap 256, which makes the filter linear-phase: its phase falls in a straight line with frequency. A straight phase slope is a pure delay. Every frequency comes out exactly 256 samples (5.8 ms) late, so the waveform’s shape isn’t distorted.

Top: the 512 prototype coefficients (its impulse response): a windowed-sinc-like shape, symmetric about tap 256. Middle: its magnitude response, zoomed to ±3 kHz and including negative frequencies. Flat through the middle, exactly −3 dB at ±344.5 Hz (dashed: half power, so neighbouring bands’ powers add up to a flat total where they cross), and below −90 dB by ±689 Hz. Bottom: its phase response across the passband: a straight line with slope −256 samples. That’s a constant delay, the definition of linear phase.

From one low-pass to 32 band-passes: modulation

To make the filter for band k, multiply the prototype’s coefficients by a cosine:

h_k[n] = p[n]\,\cos\big(\omega_k\,(n - 16)\big), \qquad \omega_k = \left(k + \tfrac{1}{2}\right)\frac{\pi}{32} \;\Longleftrightarrow\; f_k = \left(k + \tfrac{1}{2}\right) \times 689\ \text{Hz}

Why does that move the passband? Write the cosine as two complex exponentials (Euler):

\cos(\omega_k n) = \tfrac{1}{2}\,e^{+j\omega_k n} + \tfrac{1}{2}\,e^{-j\omega_k n}

Multiplying anything by $e^{j\omega_k n}$ turns each of its frequency components $e^{j\omega n}$ into $e^{j(\omega + \omega_k)n}$ . The whole spectrum slides up by $\omega_k$ . A cosine is half a shift up plus half a shift down, so the prototype’s bump at 0 Hz becomes two half-height copies at $+f_k$ and $-f_k$ . A bump centred on $\pm f_k$ is a band-pass filter for band k. AM radio uses the same trick to move audio up to a carrier frequency. (The ”− 16” only changes the cosine’s phase, not its frequency. It’s chosen so that aliasing between neighbouring bands cancels in the decoder.)

Top: the band-k filter’s coefficients: the prototype (dashed envelope) times a cosine. Higher bands mean a faster cosine. At k = 31 the “beating” you see is real: that cosine equals band 0’s slow cosine with every other sample negated. Multiplying by $(-1)^n$ shifts a spectrum by $f_s/2$ , so band 31’s filter is band 0’s filter moved to the top of the spectrum. Bottom: magnitude responses (linear scale) across the full range −22 kHz to +22 kHz. Faint: the prototype’s bump at 0 Hz. Solid: the modulated filter, whose two half-height copies sit at $\pm f_k$ . Shaded: the band each copy is meant to cover. At k = 0 the two copies meet at 0 Hz and together cover −689 to +689 Hz.

Do that for k = 0…31 and you get the whole filterbank:

Frequency response of all 32 subband filters (three highlighted). Each has a flat passband and steep sides, but neighbours overlap near the band edges. A tone close to an edge leaks into both subbands, and after decimation that leaked energy is aliased: it shows up at the wrong frequency. The decoder’s synthesis filterbank is designed so the aliasing cancels when the bands are recombined (a “pseudo-QMF”), but in the encoder it is still there.

Magnitude of all 32 subband signals over the whole recording (the stronger the colour, the louder; 80 dB range). Time runs left to right, one column per subband sample (32 input samples). This is already a crude spectrogram, with 689 Hz resolution and 0.7 ms time steps.

The actual subband samples feeding the selected granule’s MDCT: 36 per subband (the previous granule’s 18 plus this granule’s 18). Each trace is one subband, lowest at the bottom, scaled individually (subbands more than 40 dB below the loudest are drawn flat). Look at the high subbands of a tone: they are band-pass signals that have been shifted down to baseband by the decimation.

3 · The MDCT: splitting each subband further

689 Hz is too coarse for coding. The psychoacoustic model needs to separate, say, a 200 Hz bass note from its 400 Hz harmonic. So Layer III takes each subband’s samples and applies a modified discrete cosine transform to split it 18 ways. 32 subbands × 18 = 576 frequency lines, each about 38 Hz wide. Two filterbanks in series like this is why MP3’s filterbank is called hybrid. (AAC, MP3’s successor, drops the polyphase stage and uses one big MDCT directly.)

What the MDCT does

The MDCT takes a block of 2N windowed samples and returns N real numbers:

X[k] = \sum_{n=0}^{2N-1} w[n]\,x[n]\,\cos\!\left[\frac{\pi}{N}\left(n + \frac{1}{2} + \frac{N}{2}\right)\left(k + \frac{1}{2}\right)\right]

Blocks overlap by half (each new block advances by N samples), so on average you still get exactly one coefficient per input sample: the MDCT is critically sampled too. You get overlapping windows without redundancy. That’s the MDCT’s real party trick. Plain overlapping FFTs, as in a spectrogram, double the data, which a codec can’t afford.

How can 2N samples survive being squeezed into N numbers? They can’t, on their own. The inverse transform of one block gives back the windowed block plus a time-reversed, mirrored copy of it: time-domain aliasing. But that aliasing has opposite signs in the overlapping halves of neighbouring blocks. When the decoder overlap-adds them, it cancels exactly. This is called TDAC (time-domain aliasing cancellation). It works whenever the window satisfies $w[n]^2 + w[n+N]^2 = 1$ . So the absence of blocking artefacts comes from TDAC, not from the overlap alone.

TDAC on a toy signal (N = 32). Top: the signal and two overlapping sine windows. Middle: the IMDCT output of each block on its own, windowed again. Each looks wrong: it contains a mirrored copy of the signal (the aliasing). Bottom: add them in the overlap region and the aliasing cancels. The sum (dots) lands exactly on the original (line).

Why MDCT coefficients aren’t “frequency bins”

A DFT coefficient is a complex number, so it has a magnitude and a phase. For a steady sine, the magnitude in its bin stays constant from block to block and only the phase rotates. The MDCT is real: there’s no phase to put the rotation in. So the rotation shows up as the coefficients themselves going up and down, even changing sign, as the sine’s phase slides relative to the window. Coefficient $X[k]$ is “how much of a cosine at frequency $(k+\frac12)\,f_s/2N$ , with a particular phase, is in this block”. Try it:

A sine (top) analysed with a 32-coefficient MDCT (middle) and a DFT of the same windowed block (bottom). Drag the phase slider. The DFT magnitudes barely move, but the MDCT coefficients swing between positive and negative and trade energy between neighbours. Each coefficient is a phase-dependent probe of its frequency. Its square still tracks energy on average, and that’s all the codec needs: noise added to coefficient k becomes noise near frequency k in the output.

Long and short blocks

A long MDCT (N = 18 per subband) spans 1152 input samples. Rounding errors in its coefficients spread evenly over that whole span, 26 ms. If a drum hit lands in the second half of the window, the noise smears before the hit, where nothing masks it. You hear this as pre-echo. So when the encoder detects a transient, it switches the granule to three short MDCTs (N = 6 each). That trades frequency resolution (192 lines, about 115 Hz wide) for time resolution (about 4 ms each).

The window can’t jump straight from long to short: their overlapping halves wouldn’t match, and TDAC would break. So the granule before a short one uses a start window (long on the left, short on the right), and the one after uses a stop window.

One more detail: after the MDCT, eight alias-reduction butterflies at each subband boundary mix pairs of lines either side. This partly undoes the leakage between overlapping polyphase filters you saw in section 2, so a tone near a subband edge ends up mostly in one line rather than two mirror images.

Left: the window shape used for the selected granule, over its 36 subband samples. Right: its 576 MDCT coefficients, signed. Faint vertical lines mark the 32 polyphase subbands, 18 lines each. For short granules the three windows are drawn in different colours on the same frequency axis.

An “MDCT-gram”: |coefficient| in dB for every granule (columns) and line (rows, low frequencies at the bottom). The strip along the top shows the block type of each granule: long start short stop. Short granules look blurred vertically and sharp horizontally: that’s the time/frequency trade.

4 · The psychoacoustic model

Now the parallel path. The psychoacoustic model takes its own 1024-point FFT of the audio around the granule. It doesn’t look at the MDCT at all. A model of hearing needs stable magnitudes, and you’ve just seen that MDCT coefficients flicker with phase. The FFT is centred where the granule’s MDCT is centred, so both describe the same moment.

The model (a simplified version of ISO “Model 2”) then:

Groups bins into partitions of ⅓ Bark. The Bark scale models the ear’s critical bands: about 100 Hz wide in the bass, several kHz wide at the top.
Measures tonality. A peak standing well clear of its neighbours is a tone; anything else counts as noise. This matters because a tone is a poor masker (noise must sit 15–30 dB below it to be hidden), while noise masks noise well (only about 6 dB below).
Spreads energy across bands with a spreading function. A loud sound raises your hearing threshold at nearby frequencies, and more so above its own frequency than below.
Subtracts the tonality-dependent offset to get the masking threshold, and never lets it drop below the absolute threshold of hearing. The model assumes full-scale audio plays at 96 dB SPL.
Sums signal energy and threshold over each scalefactor band (next section). Their ratio is the signal-to-mask ratio (SMR): how many dB of signal you must keep above the noise for the noise to stay inaudible.

The selected granule on a Bark-warped frequency axis. Grey: the FFT spectrum. Dots: detected tonal peaks. Orange steps: energy after spreading. Thick blue: the masking threshold. Dashed: the absolute threshold of hearing. Any noise added below the blue line should be inaudible. Try the “masking” demo: the quiet tone just above the loud 1 kHz tone sits under the threshold, but the one at 4 kHz does not.

Per partition: tonality from 0 (noise-like) to 1 (tonal) as bars, and the resulting masking offset in dB (line, right scale). High tonality means a big offset, which means the masker hides less noise.

Deciding when to switch to short blocks

The model also decides the block type. This demo uses LAME-style attack detection: it high-pass filters the signal and compares the energy of 64-sample chunks with the recent past. A jump of more than 10× (10 dB) marks a transient. ISO Model 2 uses perceptual entropy (PE), a rough estimate of how many bits are needed to code the granule transparently, and switches when it spikes. Both are shown below.

Over the whole recording: the attack ratio (log scale) against the switching threshold (dashed), and perceptual entropy (lower panel). Shaded columns are granules coded with short blocks.

5 · Scalefactor bands: where the two paths meet

Here’s the answer to “how does the psychoacoustic model map onto the MDCT domain?”. It doesn’t do so line by line. The 576 MDCT lines are grouped into 22 scalefactor bands, narrow at low frequencies (4 lines) and wide at high ones (up to 158 lines), roughly following critical bands. Short blocks get 13 bands per window. Both paths are summarised onto this shared grid:

The psychoacoustic model delivers one SMR per band, measured with its FFT.
The encoder measures the MDCT energy in the same band.
The allowed noise in the MDCT domain is then $x_\text{min} = E_\text{MDCT} \,/\, \text{SMR}$ .

A ratio means the same thing in both domains, so nothing has to be converted. The FFT and the MDCT never need to agree on absolute levels or on how individual bins line up.

How the 576 lines (or 3 × 192 for short blocks) divide into scalefactor bands, with each band’s upper frequency edge. The last long band (16–22 kHz) has no scalefactor of its own.

The selected granule, per scalefactor band: MDCT energy (bars), allowed noise xmin (blue markers), and the SMR between them (numbers). Where the SMR is negative, the whole band is below the masking threshold. The encoder may zero it completely and nobody will notice.

6 · Quantisation and bit allocation

The decoder will rebuild each line from an integer $ix$ as

xr = \operatorname{sign}(ix)\;|ix|^{4/3}\;2^{(\text{global\_gain} - 210)/4}\;2^{-\frac{1}{2}\,\text{scalefac}[\text{band}]}

so the encoder has three knobs:

global_gain: one step size for the whole granule. +1 makes every step 19% coarser (about +1.5 dB of noise) and the numbers smaller, so cheaper to code.
scalefac[band]: boosts one band by 3 dB per step before rounding. That makes the rounding finer in that band only. This is the noise-shaping knob.
The ¾ power law ( $|x|^{3/4}$ when quantising, $|ix|^{4/3}$ when decoding). It’s built-in companding: large values are rounded relatively more coarsely, so the noise roughly tracks the signal level.

Two nested loops search for the settings:

Inner (rate) loop: find the smallest global_gain whose quantised values fit in the bit budget. Here it’s a binary search.
Outer (distortion) loop: measure the actual noise in each band. Every band whose noise exceeds xmin gets its scalefactor bumped up, and the inner loop runs again. Boosting bands costs bits, so the inner loop coarsens global_gain to pay for it. Noise flows from the bands where it would be audible to the bands where it’s masked. Stop when every band is clean, or the scalefactors run out of range.

Bit costs are estimated. The real encoder picks from 32 Huffman tables, and this demo models them with an entropy-like formula, but keeps their structure. In bitstream order the spectrum splits into big_values (pairs of any size), count1 (quads of 0/±1), and rzero (trailing zeros, which cost nothing). Also omitted: the bit reservoir, which lets hard granules borrow bits from easy ones. Here every granule gets exactly its share.

Per scalefactor band: the quantisation noise actually introduced (bars, red where it exceeds the allowance), the allowed noise xmin (blue markers), and the signal energy (faint bars) for reference. At high bitrates every bar sits under its marker. Lower the bitrate and watch the red appear.

Left: the scalefactors the outer loop settled on: how many 3 dB boosts each band got. Right: the quantised integers in bitstream order, with the big_values / count1 / rzero regions shaded. Most of the spectrum is often exactly zero.

The loop’s trajectory for this granule. Each row is one outer-loop iteration (the inner loop’s global_gain search inside it is not shown). The highlighted row is the result that was kept.

Over the whole recording: bits used per granule (bars) against the budget (line), and how many bands ended up with audible noise (red dots, right scale).

7 · The result

The decoder runs everything in reverse: dequantise, undo alias reduction, IMDCT, overlap-add (TDAC), polyphase synthesis. Use the Listen buttons at the top to compare the original and the decoded audio. Difference plays original − decoded, the noise the encoder added. It’s amplified, because at sensible bitrates it is too quiet to hear clearly otherwise.

Experiments worth trying:

Drop to 32 kbit/s. The high bands empty out first (see below): when bits are short, whole bands get zeroed.
On the masking or noise bursts demo at 64 kbit/s, turn off noise shaping. The SNR barely changes (it may even go up), but “granules with audible noise” jumps from zero to most of them. The same amount of noise is still there. It just isn’t hidden under the masking threshold any more. This is the whole idea of perceptual coding: SNR is the wrong measure.
Force always long on the click demo at a low bitrate and listen for pre-echo before the click.

The MDCT-gram after quantisation (what the decoder receives). Compare it with section 3: dark areas are lines that were quantised to zero.

8 · What’s left: the bitstream

A real encoder would now Huffman-code the integers, choosing among 32 tables region by region. It would write the scalefactors, and pack everything into frames: a 32-bit header (sync word, bitrate, sample rate…), side information (block types, global_gain, table choices, region boundaries, and a pointer into the bit reservoir), then the main data. None of that changes what you hear. It’s lossless packaging of the numbers above.