Lecturer 7:

Camera Calibration

Source credit : Svetlana Lazebnik

All Slides Credit to Svetlana Lazebnik

Dandhi Kuswardhana, Ph.D: EL462/ Visi Komputer

 Our goal: Recovery of 3D structure

J. Vermeer, Music Lesson, 1662

A. Criminisi, M. Kemp, and A. Zisserman,Bringing Pictorial Space to Life: computer techniques for the
analysis of paintings, Proc. Computers and the History of Art, 2002
Things aren’t always as they appear…

http://en.wikipedia.org/wiki/Ames_room
Single-view ambiguity

X?
X?
X?

x
Single-view ambiguity
Single-view ambiguity

Rashad Alakbarov shadow sculptures

Anamorphic perspective

Image source
Anamorphic perspective

H. Holbein The Younger, The Ambassadors, 1533

https://en.wikipedia.org/wiki/Anamorphosis
 Our goal: Recovery of 3D structure
• When certain assumptions • In general, we need
hold, we can recover multi-view geometry
structure from a single view

Image
source

• But first, we need to understand the geometry of a

single camera…
 Camera calibration
world coordinate system

• Normalized (camera) coordinate system: camera

center is at the origin, the principal axis is the z-axis,
x and y axes of the image plane are parallel to x and y
axes of the world

• Camera calibration: figuring out transformation from

world coordinate system to image coordinate system
Review: Pinhole camera model

( X ,Y , Z )  ( f X / Z , f Y / Z )

X X
   f X f 0  
Y      Y  x = PX
Z  fY = f 0  
Z
   Z   1 0 
1
  1
 Principal point

• Principal point (p): point where principal axis intersects the

image plane
• Normalized coordinate system: origin of the image is at the
principal point
• Image coordinate system: origin is in the corner
Principal point offset

We want the principal

py point to map to (px, py)
instead of (0,0)

px

( X , Y , Z )  ( f X / Z + px , f Y / Z + p y )

X X
   f X + Z px   f px 0  
Y      Y 
 Z    f Y + Z py  =  f py 0  
Z
   Z  
  1 0 
1
  1
Principal point offset

py principal point: ( px , p y )

px
X
f px 0  
Y 
=  f py 
0  
Z
 1 0 
1
 Principal point offset

py principal point: ( px , p y )

px
X X
px   f p x  1 0    f px 0  
     Y    Y 
py  =  f py  1 0   =  f py 0  
  Z Z
  1   1 0   1 0 
1
calibration matrix projection matrix
1
K [I | 0]

P = KI | 0
Pixel coordinates

1 1
Pixel size: 
mx m y

mx pixels per meter in horizontal direction,

my pixels per meter in vertical direction

mx  f p x   x x 
K =  my 
 f p y  =  y  y 
 1  1   1 
pixels/m m pixels
 Camera rotation and translation

• In general, the camera

coordinate frame will be
related to the world
coordinate frame by a
camera rotation and a translation
coordinate system world coordinate
system

• Conversion from world to camera coordinate system

(in non-homogeneous coordinates):

~
(
~ ~
X cam = R X − C )
coords. of point coords. of camera center
in camera frame in world frame
coords. of a point
in world frame
Camera rotation and translation

~
( ~ ~
X cam = R X − C )
3D transformation
matrix (4 x 4)
Camera rotation and translation

~
( ~ ~
X cam = R X − C )
3D transformation
matrix (4 x 4)
Camera rotation and translation

2D 3D transformation
transformation matrix (4 x 4)
perspective
matrix (3 x 3) projection
matrix (3 x 4)
Camera rotation and translation
Camera rotation and translation

~
t = − RC
 Camera parameters P = K R t 
• Intrinsic parameters
• Principal point coordinates
mx  f p x   x x 
• Focal length K =  my  f p y  =  y  y 

• Pixel magnification factors  1  1   1 
• Skew (non-rectangular pixels)
• Radial distortion
 Camera parameters P = K R t 
• Intrinsic parameters
• Principal point coordinates
• Focal length
• Pixel magnification factors
• Skew (non-rectangular pixels)
• Radial distortion

 
• Extrinsic parameters
~
• Rotation and translation relative
to world coordinate system
P = K R − RC
• What is the projection of the coords. of
camera center
camera center?
in world frame
~
 ~ C

PC = K R − RC   = 0  The camera center is the
null space of the
1 projection matrix!
Camera calibration

x = K R t  X
X 
  x  * * * * Y 
 y  = * * * *  
    Z 
   * * * *  
1
Source: D. Hoiem
Camera calibration
• Given n points with known 3D coordinates Xi
and known image projections xi, estimate the
camera parameters

Xi

xi

P?
Camera calibration: Linear method
 xi  P1 X i 
T

 y   P T X  = 0
 x i = PX i x i  PX i = 0  i  2 i
 1  P3T X i 

 0 − XTi yi XTi  P1 
 T
 
 X T
i 0 − xi X i  P2  = 0
− yi XTi

T
xi X i 0  P3 

Two linearly independent equations

Camera calibration: Linear method
 0T XT
1 − y1X T
1
 T 
 X1 0T
− x1X  P1 
T
1
 
    P2  = 0 Ap = 0
 T T 
0 XTn − yn X n  P3 
 XT 0T − xn XTn 
 n
• P has 11 degrees of freedom
• One 2D/3D correspondence gives us two linearly
independent equations
• 6 correspondences needed for a minimal solution
• Homogeneous least squares: find p minimizing ||Ap||2
• Solution given by eigenvector of ATA with smallest eigenvalue
Camera calibration: Linear method
 0T XT
1 − y1X 
T
1
 T 
 X1 0T
− x1X  P1 
T
1
 
    P2  = 0 Ap = 0
 T T 
0 XTn − yn X n  P3 
 XT 0T − xn XTn 
 n
• Note: for coplanar points that satisfy ΠTX=0,
we will get degenerate solutions (Π,0,0),
(0,Π,0), or (0,0,Π)
 Camera calibration: Linear vs. nonlinear
• Linear calibration is easy to formulate and solve,
but it doesn’t directly tell us the camera parameters
X 
  x  * * * *
x = K R t  X
Y 
 y  = * * * *  
    Z  vs.
   * * * *  
1
• In practice, non-linear methods are preferred
• Write down objective function in terms of intrinsic and extrinsic
parameters
• Define error as sum of squared distances between measured 2D
points and estimated projections of 3D points
• Minimize error using Newton’s method or other non-linear
optimization
• Can model radial distortion and impose constraints such as known
focal length and orthogonality
A taste of multi-view geometry: Triangulation
• Given projections of a 3D point in two or more
images (with known camera matrices), find
the coordinates of the point
Triangulation
• Given projections of a 3D point in two or more
images (with known camera matrices), find
the coordinates of the point

X?

x2
x1

O1 O2
Triangulation
• We want to intersect the two visual rays
corresponding to x1 and x2, but because of
noise and numerical errors, they don’t meet
exactly

X?

x2
x1

O1 O2
Triangulation: Geometric approach
• Find shortest segment connecting the two
viewing rays and let X be the midpoint of that
segment

x2
x1

O1 O2
Triangulation: Nonlinear approach
Find X that minimizes

d (x1 , P1 X) + d (x 2 , P2 X)
2 2

X?

P1X
x2
x1
P2X

O1 O2
Triangulation: Linear approach

1 x1 = P1 X x1  P1 X = 0 [x1 ]P1 X = 0
2 x 2 = P2 X x 2  P2 X = 0 [x 2 ]P2 X = 0

Cross product as matrix multiplication:

 0 − az a y  bx 
  
a  b =  az 0 − a x  by  = [a ]b
− a y ax 0  bz 

Triangulation: Linear approach

1 x1 = P1 X x1  P1 X = 0 [x1 ]P1 X = 0
2 x 2 = P2 X x 2  P2 X = 0 [x 2 ]P2 X = 0

Two independent equations each in terms of

three unknown entries of X
 Preview: Structure from motion
Figure credit:
Noah Snavely

Camera 1 Camera 3
Camera 2
R1,t1 R2,t2 R3,t3
• Given 2D point correspondences between multiple images, compute
the camera parameters and the 3D points
 Preview: Structure from motion
?

Camera 1 Camera 3
Camera 2
R1,t1 R2,t2 R3,t3
• Structure: Given known cameras and projections of the same 3D
point in two or more images, compute the 3D coordinates of that point
• Triangulation!
 Preview: Structure from motion

Camera 1 Camera 3
R1,t1 ? Camera 2
R2,t2 ? ? R3,t3
• Motion: Given a set of known 3D points seen by a camera, compute
the camera parameters
• Calibration!
Useful reference