CNG 483

Introduction to Computer Vision

Color
Asst. Prof. Dr. Meryem Erbilek

Slide partially based on Stanford U. CS131

METU Lecture 1 - 1
 Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing

Slide partially based on Stanford U. CS131

METU Lecture 1 - 2
 What is color?
• The result of interaction
between physical light in the
environment and our visual
system.

• A psychological property of our

visual experiences when we
look at objects and lights,
not a physical property of
those objects or lights.

Slide partially based on Stanford U. CS131 Slide credit: Lana Lazebnik

METU Lecture 1 - 3
 Color and light
• When a beam of sunlight passes through a glass
prism, the emerging beam of light is not white but
consists instead of a continuous spectrum of colors
ranging from violet at one end to red at the other.
• White light: composed of almost equal energy in all
wavelengths of the visible spectrum.

Slide partially based on Stanford U. CS131

METU Lecture 1 - 4
 Electromagnetic Spectrum

Human Luminance Sensitivity Function

Slide partially based on Stanford U. CS131 http://www.yorku.ca/eye/photopik.htm

METU Lecture 1 - 5
 Visible Light
•Estimation of the wavelengths of electromagnetic
radiation emitted by a star is based on surface
temperature.
•For instance, since the surface of the sun is around
5800K, the peak of the sun’s emitted light lies in the
visible region. The Sun is the dominant source for visible-
light waves our eyes receive.
•A corona (outer-most layer of the Sun) is most easily https://sunearthday.nasa.gov/2007/multimedia/gal_007.php
seen during a total solar eclipse, since it is so faint and
the bright photosphere overwhelms it.
•As objects grow hotter, they radiate energy dominated
by shorter wavelengths, changing color. A flame on a
blow torch shifts from reddish to bluish in color as it is
adjusted to burn hotter. In the same way, the color of
stars tells scientists about their temperature.

Slide partially based on Stanford U. CS131

METU Lecture 1 - 6
 Visible Light
Plank’s law for Blackbody radiation
Surface of the sun: ~5800K

Why do we see light of these wavelengths?

…because the
peak of the sun’s
emitted light lies in
the visible region

Slide partially based on Stanford U. CS131 © Stephen E. Palmer, 2002

METU Lecture 1 - 7
 The Physics of Light

Any source of light can be completely described

physically by its spectrum: the amount of energy emitted
(per time unit) at each wavelength 400 - 700 nm.

88
© Stephen E. Palmer, 2002
 The Physics of Light

Some examples of the spectra of light sources

Rel. power
Rel. power

Rel. power
Rel. power

99
© Stephen E. Palmer, 2002
 The Physics of Light

Some examples of the reflectance spectra of surfaces

Yellow Blue Purple

% Light Reflected

Red

400 700 400 700 400 700 400 700

1010
Wavelength (nm) © Stephen E. Palmer, 2002
 Interaction of light and surfaces
• Reflected color is the result of
interaction of light source
spectrum with surface
reflectance.
• Spectral radiometry
– All definitions and units are now
“per unit wavelength”
– All terms are now “spectral”

Slide partially based on Stanford U. CS131 From Foundation of Vision by Brian Wandell, Sinauer Associates, 1995

METU Lecture 1 - 11
Collects and processes information
from across the electromagnetic
spectrum.

1212
 Interaction of light and surfaces
• What is the observed color of any surface under
monochromatic light (visible light of a narrow band
of wavelengths)?

https://www.youtube.com/watch?v=hd077pa-5CI
Slide partially based on Stanford U. CS131 Olafur Eliasson, Room for one color
METU Lecture 1 - 13 Slide by S. Lazebnik
James Turrell, a Californian artist, used light and color to
completely immerse people in a world where there was no
depth perception.

Slide partially based on Stanford U. CS131 James Turrell, Breathing Light

METU Lecture 1 - 14
 Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing

Slide partially based on Stanford U. CS131

METU Lecture 1 - 16
Two types of light-sensitive receptors

•When we look at a scene, light first enters our eyes through

the pupil and then the retina.

•The retina is primarily composed of two types of light-

sensitive cells: rods and cones, named for their appearance
under a microscope.

Cones Rods
cone-shaped rod-shaped
less sensitive highly sensitive
operate in high light operate at night
color vision gray-scale vision

17
© Stephen E. Palmer, 2002
 Rod / Cone sensitivity

The famous sock-matching problem… 18

Color perception
M L
Power

Wavelength

Rods and cones act as filters on the spectrum

• To get the output of a filter, multiply its response curve by the
spectrum, integrate over all wavelengths
– Each cone yields one number

• Q: How can we represent an entire spectrum with 3 numbers?

• A: We can’t! Most of the information is lost.
– As a result, two different spectra may appear indistinguishable
» such spectra are known as metamers 19
Slide by Steve Seitz
Spectra of some real-world surfaces

metamers

20
Standardizing color experience
• We would like to understand which spectra
produce the same color sensation in people
under similar viewing conditions
• Color matching experiments

21
Foundations of Vision, by Brian Wandell, Sinauer Assoc., 1995
Color matching experiment 1

22
Source: W. Freeman
Color matching experiment 1

p1 p2 p3
23
Source: W. Freeman
Color matching experiment 1

p1 p2 p3
24
Source: W. Freeman
Color matching experiment 1

The primary color

amounts needed for a
match

p1 p2 p3
25
Source: W. Freeman
Additive color mixing

Colors combine by
adding color spectra

Light adds to
existing black.
26
Source: W. Freeman
Color mixing

27
Source: W. Freeman
Examples of additive color systems

CRT phosphors

multiple projectors
http://www.jegsworks.com
28
http://www.crtprojectors.co.uk/
Color matching experiment 2

29
Source: W. Freeman
Color matching experiment 2

p1 p2 p3
30
Source: W. Freeman
Color matching experiment 2

p1 p2 p3
31
Source: W. Freeman
 Color matching experiment 2

The primary color

We say a “negative” amounts needed for a
amount of p2 was match:
needed to make the
match, because we
added it to the test
color’s side.

p1 p2 p3

p1 p2 p3 p1 p2 p3
32
Source: W. Freeman
Subtractive color mixing
Colors combine by
multiplying color
spectra.

Pigments remove
color from incident
light (white).

33
Source: W. Freeman
Examples of subtractive color systems
• Printing on paper
• Crayons
• Photographic film

34
 Trichromacy
• In color matching experiments, most people can
match any given light with three primaries
– Primaries must be independent
• For the same light and same primaries, most
people select the same weights
– Exception: color blindness
• Trichromatic color theory
– Three numbers seem to be sufficient for encoding
color
– Dates back to 18th century (Thomas Young)
Slide partially based on Stanford U. CS131

METU Lecture 1 - 35
 Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing

Slide partially based on Stanford U. CS131

METU Lecture 1 - 36
 Linear color spaces
• Defined by a choice of three primaries
• The coordinates of a color are given by the
weights of the primaries used to match it

mixing two lights produces mixing three lights produces

colors that lie along a straight colors that lie within the triangle
line in color space they define in color space
Slide partially based on Stanford U. CS131

METU Lecture 1 - 37
 How to compute the weights of the primaries
to match any spectral signal

p1 p2 p3

• Matching functions: the amount of each primary

needed to match a monochromatic light source at
each wavelength

38
Source: W. Freeman
 Linear color spaces: RGB space
• Primaries are monochromatic lights (for monitors,
they correspond to the three types of phosphors)
• Subtractive matching required for some
wavelengths
RGB primaries RGB matching functions

39
Nonlinear color spaces: HSV

• Perceptually meaningful dimensions:

Hue, Saturation, Value (Intensity)

40
 Overview of Color
• Physics of color
• Human encoding of color
• Color spaces
• White balancing

Slide partially based on Stanford U. CS131

METU Lecture 1 - 41
 White balance
• When looking at a picture on screen or print, we adapt to
the illuminant of the room, not to the scene in the picture!
• When the white balance is not correct, the picture will
have an unnatural color “cast”
incorrect white balance correct white balance

42
http://www.cambridgeincolour.com/tutorials/white-balance.htm
 White balance

• Digital cameras:
• Automatic white balance
• White balance settings corresponding to
several common illuminants
• Custom white balance using a reference
object

• Film cameras:
• Different types of film or different filters for different
illumination conditions

43
http://www.cambridgeincolour.com/tutorials/white-balance.htm Slide: F. Durand
 White balance
• Von Kries adaptation

• The von Kries coefficient law in color adaptation

describes the relationship between the illuminant
and the human visual system sensitivity.

• Multiply each channel by a gain factor to match the

appearance of a gray neutral object.

44
Slide: F. Durand
 White balance
• Von Kries adaptation
• Multiply each channel by a gain factor.
• Best way: gray card
• Take a picture of a neutral object (white or gray)
• Deduce the weight of each channel
– If the object is recoded as rw, gw, bw
use weights 1/rw, 1/gw, 1/bw

45
Slide: F. Durand
White balance
• Without gray cards: we need to “guess” which
pixels correspond to white objects
• Gray world assumption
• The image average rave, gave, bave is gray
• Use weights 1/rave, 1/gave, 1/bave
• Brightest pixel assumption (non-staurated)
• Highlights usually have the color of the light source
• Use weights inversely proportional to the values of the
brightest pixels
• Gamut mapping
• Gamut: convex hull of all pixel colors in an image
• Find the transformation that matches the gamut of the image
to the gamut of a “typical” image under white light
• Use image statistics, learning techniques
46
Slide: F. Durand
Uses of color in computer vision
Color histograms for indexing and retrieval

47
Swain and Ballard, Color Indexing, IJCV 1991.
Uses of color in computer vision
Skin detection

M. Jones and J. Rehg, Statistical Color Models with

48
Application to Skin Detection, IJCV 2002. Source: S. Lazebnik
 Uses of color in computer vision
Image segmentation and retrieval

C. Carson, S. Belongie, H. Greenspan, and Ji. Malik, Blobworld:

Image segmentation using Expectation-Maximization and its 49
application to image querying, ICVIS 1999. Source: S. Lazebnik
 Uses of color in computer vision
Building appearance models for tracking

D. Ramanan, D. Forsyth, and A. Zisserman. Tracking People by Learning their 50

Appearance. PAMI 2007. Source: S. Lazebnik
Credits
• Most slides are mainly by Juan Carlos
Niebles and Ranjay Krishna from Stanford AI
Lab

51