Colour and the eye

The hearing pages are about what the ear does with sound: which frequencies it hears, how loud it judges them, what it can’t tell apart. This page does the same for the eye and light. It turns out to be a signal-processing story too, and a surprising one: the eye throws away almost everything about the light that reaches it, and keeps three numbers. Everything about colour on screens, cameras and image files follows from that.

1 · Light is a spectrum

Light is an electromagnetic wave, and like a sound it can be a mixture of many frequencies. We describe light by wavelength instead of frequency (the two are tied by the speed of light: violet is about 400 nm, red about 700 nm, a nanometre being a millionth of a millimetre). Any light has a spectrum: how much power it carries at each wavelength, exactly as a sound has a spectrum on the Fourier page.

A light’s spectrum, from 380 to 780 nm, the range the eye can see, with each wavelength’s colour along the bottom. Choose a light, or draw your own on the plot. Below, the colour it looks (worked out as the rest of this page explains, at full brightness), and two numbers for it, x and y, from section 4. Sunlight is a smooth hill; a filament bulb is a slope, mostly red; a white LED is a sharp blue peak and a broad yellow hump; a sodium street lamp is almost all one wavelength.

A spectrum is a whole curve of numbers. The surprise is how little of it the eye uses.

2 · Three cones

The retina’s colour sensors are cones, and there are only three kinds. Each kind absorbs light across a broad range of wavelengths, more at some than others: the L cones (long wavelengths) most at about 570 nm, the M cones (medium) at about 545, the S cones (short) at about 445 (in the 5 nm steps of the table used here). Each cone adds up the light reaching it, weighted by its curve, and sends on one number. So a whole spectrum becomes three numbers, L, M and S. In the language of the Fourier page, each cone takes an inner product of the spectrum with its own curve: a filter bank with three very broad, overlapping bands.

That throws a lot away, and it means two quite different spectra can give the cones the same three numbers. They then look exactly the same colour. They’re called metamers.

the light

three lines

The three cones’ sensitivities (Stockman and Sharpe’s measurements), with a light’s spectrum shaded behind. The three black lines, at 450, 540 and 620 nm, are another light: three single wavelengths, with their powers worked out to give the cones exactly what the first light gives them. Below, both lights’ colours, and the cones’ numbers for each: the same. A spectrum that’s a smooth hill and one that’s three spikes, and nobody can tell them apart. (For a single wavelength of 580 nm, the blue line’s power comes out a hair below zero: a pure wavelength is just outside what any three others can make, as section 3 shows.)

Metamers are why a screen works. Its pixels make three narrow-ish spectra, red, green and blue, nothing like the sunlight off a lemon; but mixed in the right amounts they give the cones the same three numbers as the lemon, and the lemon is what we see.

3 · Matching colours with three lights

Before anyone could measure the cones, colour was measured by matching. In experiments in the 1920s that the CIE (the International Commission on Illumination) made into a standard in 1931, people looked at a split field: on one side a single wavelength, on the other a mixture of three fixed lights, red, green and blue, and turned three knobs until the two halves matched.

the target (and anything added to it)

the three lights

The 1931 experiment. The target is one wavelength; the three lights are the CIE’s: 700, 546.1 and 435.8 nm. Move the sliders until the two halves match (the difference, ΔE, is how different they look: under about 1 is a match), or Show the match. For many targets one amount has to be negative: there’s no way to match with the three lights on one side, but adding that light to the target’s side does. The curves are the CIE’s answers for every wavelength: its r̄, ḡ and b̄ (worked out here from the CIE’s tables, and checked against its own transform). The screen can’t show some of these colours fully, so the match you see is the best it can do.

Red has to go on the target’s side for every wavelength from 436 to 546 nm: the curve dips below zero. Those spectral greens and blue-greens are more saturated than any mixture of the three lights can be. No three lights make every colour.

The CIE didn’t like negative numbers, so it changed coordinates: three new imaginary “primaries”, X, Y and Z, chosen so every real colour has positive amounts of all three, and so that Y is exactly brightness (how bright a light looks, the luminance). The three curves x̄, ȳ and z̄ that came out are still the standard way to turn a spectrum into numbers: they’re what this page uses, from the published tables.

4 · A map of colours

A colour’s three numbers mix up two things: how bright it is, and what colour it is. Taking the brightness out leaves two: x = X / (X + Y + Z) and y = Y / (X + Y + Z), the chromaticity. Every colour, whatever its brightness, is a point on a chart of x and y.

The CIE 1931 chromaticity chart. The curved edge is the single wavelengths (in nm), from violet round to red; the straight line across the bottom joins the two ends, the purples, which aren’t any single wavelength. Every colour there is lies inside. The dashed triangle is what a screen’s red, green and blue (sRGB’s) can make by mixing: about a third of the chart. The curve through the middle is the colours of black bodies, from 1,500 K to 25,000 K (section 6). Mark a light to see where it falls. Colours outside the triangle are drawn as the nearest a screen can show.

Mixing two lights gives a colour on the straight line between their points; three lights, anything inside their triangle. That’s why a screen’s range of colours, its gamut, is a triangle, and why it can’t reach the spectral colours at the edge: a pure 520 nm green is far outside sRGB’s corner. Wider-gamut screens put their corners further out; none reaches the edge.

sRGB, the standard for the web and most screens, is a set of choices on this chart: three corners (its red, green and blue), and a white point, D65, a standard daylight at about 6,500 K. Its matrix turns XYZ into how much of each primary to use; every colour on this page is drawn that way.

5 · Numbers and light: gamma

A pixel’s number in an image file is not an amount of light. The sRGB standard says the stored number is the light raised to about the power 1 / 2.2 (exactly, a short straight piece near black, then a power of 1 / 2.4). Half the number is about a fifth of the light: 0.5 means 0.214.

That’s deliberate, and it’s the same trick as μ-law for telephone voices: spend the bits where the senses are most sensitive. The eye notices small differences between dim lights far more than between bright ones (roughly, it sees ratios, not differences). Store light evenly in 8 bits, and the steps near black are coarse enough to see, while near white there are levels to spare.

Top: from black to white in a few levels, spaced evenly in light, then spaced evenly in sRGB’s numbers. With few levels, the even-light strip jumps visibly near black and wastes its steps near white; sRGB’s spacing looks even. Below: mixing two colours. Averaging the numbers is what software often does by mistake; averaging the light is what really happens when two lights mix (or when a camera’s sensor blurs them). Red and green averaged as numbers give a dark, muddy yellow (128, 128, 0); as light, a bright one (188, 188, 0).

So anything that adds, blurs, scales or mixes images (resizing, blurring, anti-aliased edges, blending two pictures) should turn the numbers back into light first, do the arithmetic, and convert back. Doing it on the numbers darkens and muddies the mixes, and a lot of software still does.

6 · The colour of white: temperature and white balance

Heat anything enough and it glows: first dull red, then orange, yellow, white, and bluish-white. A perfect glower, a black body, gives a spectrum set entirely by its temperature (Planck’s law), and its colour is named by that temperature: a candle about 1,900 K, a filament bulb 2,700 K, sunlight about 5,800 K. This is a light’s colour temperature. Light that isn’t a black body (daylight, LEDs) is given the temperature of the black body it looks most like.

What we call white is whatever lights the scene. A sheet of paper under a filament bulb sends the eye far more red than blue, and we still see it as white: the eye adapts, turning each cone’s sensitivity down in proportion to how much the light excites it (chromatic adaptation). A camera has to do this on purpose: white balance.

Twelve surfaces (each a made-up but smooth reflectance curve, like real paints and leaves), lit by a black body of the chosen temperature. Top row: under daylight (D65), for reference. Middle: under the chosen light, as a camera would record them without adjusting. Bottom: white-balanced: each of three cone-like responses scaled so the light’s white comes out as daylight’s (a von Kries adaptation, with Bradford’s matrix). At 2,700 K the surfaces as lit differ from daylight by ΔE 45 on average; white-balanced, by 6.6. The dashed curve is D65’s spectrum.

White balance brings the colours close, but not back exactly: the green, for one, is still ΔE 14.5 off at 2,700 K. Three numbers can be rescaled, but the light’s whole spectrum changed, and different surfaces’ spectra respond to that change differently. It’s metamerism again, from the other side: two paints that match in daylight may not match under a bulb.

7 · The eye sees brightness more sharply than colour

One more property of the eye, and the biggest saving in every photo and video file. The eye sees fine detail in brightness far better than fine detail in colour. JPEG (and every video format) uses it: it splits each pixel into brightness (Y′, mostly green, since the eye’s brightness sense is mostly M and L cones) and two colour differences (Cb and Cr, how much bluer and redder than grey), then keeps the colour at half the resolution each way, or less.

A picture with its colour (Cb and Cr) kept at a lower resolution, averaged over blocks and spread back, the brightness untouched; or the other way round. Keeping the colour at a quarter of the resolution each way barely shows in most of the picture (look closely at the edges of the red writing); doing the same to the brightness turns it to mush. The thin red and blue stripes are the exception: they differ in colour but hardly in brightness, so blurring the colour turns them purple. Real pictures have little detail like that. Or choose a picture of your own: it’s processed in the page and goes nowhere.

JPEG’s usual setting, called 4:2:0, keeps colour at half the resolution each way: two numbers in every four pixels instead of eight, so half the numbers overall, before any other compression. It’s the same idea as an MP3 encoder dropping what’s masked: throw away what the senses won’t miss.

Summary

Light is a spectrum; the eye’s three cones each reduce it to one number, so colour is three numbers.
Different spectra that give the cones the same numbers are metamers: the same colour.
The 1931 experiment matched each wavelength with three lights; some needed negative amounts. The CIE’s XYZ turned the matches into positive numbers, with Y the brightness.
The chromaticity chart shows every colour; a screen’s three primaries make a triangle inside it.
Stored numbers aren’t light: gamma spends the bits where the eye looks. Mix in light, not in numbers.
A light’s colour temperature sets what white is; white balance scales the three responses back.
The eye sees brightness more sharply than colour, and image files keep less colour.