Robert Collins

CSE486, Penn State

Lecture 12:
Camera Projection
Reading: T&V Section 2.4
Robert Collins
CSE486, Penn State
Imaging Geometry

W
Object of Interest
in World Coordinate
System (U,V,W)
V
U
Robert Collins
CSE486, Penn State
Imaging Geometry

Camera Coordinate
Y System (X,Y,Z).

X Z
• Z is optic axis
f • Image plane located f units
out along optic axis
• f is called focal length
Robert Collins
CSE486, Penn State
Imaging Geometry

W
Y y
X Z x

V
U

Forward Projection onto image plane.

3D (X,Y,Z) projected to 2D (x,y)
Robert Collins
CSE486, Penn State
Imaging Geometry

W
Y y
X Z x

V
u U

v Our image gets digitized

into pixel coordinates (u,v)
Robert Collins
CSE486, Penn State
Imaging Geometry
Camera Image (film) World
Coordinates Coordinates W
Coordinates
Y y
X Z x

V
u U

v Pixel
Coordinates
Robert Collins
CSE486, Penn State
Forward Projection

World Camera Film Pixel

Coords Coords Coords Coords
U X x u
V Y y v
W Z

We want a mathematical model to describe

how 3D World points get projected into 2D
Pixel coordinates.

Our goal: describe this sequence of

transformations by a big matrix equation!
Robert Collins
CSE486, Penn State
Backward Projection

World Camera Film Pixel

Coords Coords Coords Coords
U X x u
V Y y v
W Z

Note, much of vision concerns trying to

derive backward projection equations to
recover 3D scene structure from images
(via stereo or motion)

But first, we have to understand forward projection…

Robert Collins
CSE486, Penn State
Forward Projection

World Camera Film Pixel

Coords Coords Coords Coords
U X x u
V Y y v
W Z

3D-to-2D Projection
• perspective projection

We will start here in the middle, since we’ve already

talked about this when discussing stereo.
Robert Collins
CSE486, Penn State
Basic Perspective Projection
Scene Point Perspective Projection Eqns
P = (X,Y,Z)
X
Image Point y x f
Y
p = (x,y,f) X Y
Z
x Y
Z
y
y f
x
X
f Z
Z
O

O.Camps, PSU
Robert Collins
CSE486, Penn State
Basic Perspective Projection
Scene Point Perspective Projection Eqns
P = (X,Y,Z)
X
Image Point y x f
Y
p = (x,y,f) X Y
Z
x Y
Z
y
y f
x
X
f Z
Z derived via similar
O
triangles rule

x X

f
Z
O.Camps, PSU
Robert Collins
CSE486, Penn State
Basic Perspective Projection
Scene Point Perspective Projection Eqns
P = (X,Y,Z)
X
Image Point y x f
Y
p = (x,y,f) X Y
Z
x Y
Z
y
y f
x
X
f Z
Z derived via similar
O
triangles rule
X Y
y
x
f f
Z Z
O.Camps, PSU
Robert Collins
CSE486, Penn State
Basic Perspective Projection
Scene Point Perspective Projection Eqns
P = (X,Y,Z)
X
Image Point y x f
Y
p = (x,y,f) X Y
Z
x Y
Z
y
y f
x
X
f Z
Z
O

So how do we represent this as a matrix equation?

We need to introduce homogeneous coordinates.

O.Camps, PSU
Robert Collins
CSE486, Penn State
Homogeneous Coordinates
Represent a 2D point (x,y) by a 3D point (x’,y’,z’) by
adding a “fictitious” third coordinate.

By convention, we specify that given (x’,y’,z’) we can

recover the 2D point (x,y) as
x' y'
x y
z' z'
Note: (x,y) = (x,y,1) = (2x, 2y, 2) = (k x, ky, k)
for any nonzero k (can be negative as well as positive)
Robert Collins
CSE486, Penn State

Perspective Matrix Equation

(in Camera Coordinates)

X X 
x f  x'  f 0 0 0  
 y '   0  Y
Z
   f 0 0  
Y Z 
y f  z '   0 0 1 0  
Z 1
Robert Collins
CSE486, Penn State
Forward Projection

World Camera Film Pixel

Coords Coords Coords Coords
U X x u
V Y y v
W Z

Rigid Transformation (rotation+translation)

between world and camera coordinate systems
Robert Collins

World to Camera Transformation

CSE486, Penn State

PC PW
W

X
U
Z V

Avoid confusion: Pw and Pc are not two different

points. They are the same physical point, described
in two different coordinate systems.
Robert Collins

World to Camera Transformation

CSE486, Penn State

PC PW
W

X
U
R Z V
C
Rotate to Translate by - C
align axes (align origins)

P C = R ( PW - C )
Robert Collins

Matrix Form, Homogeneous Coords

CSE486, Penn State

P C = R ( PW - C )

X r11 r12 r13  1 0 0  cx U

Y r21 r22 r23  0 1 0  cy V
Z r31 r32 r33  0 0 1  cz W
1 0 0 0 1 0 0 0 1 1
Robert Collins
CSE486, Penn State
Example: Simple Stereo System
Y Z
(X,Y,Z)
z
left y
camera
located at ( , )
z
(0,0,0) y ( , )
x right
camera
Tx located at
x (Tx,0,0)

X
Left camera located at world origin (0,0,0)
and camera axes aligned with world coord axes.
Robert Collins
CSE486, Penn State
Simple Stereo, Left Camera

X 1r11 r012 r013  1 0 0 0cx U

Y 0r21 r122 r023  0 1 0 0cy V
Z 0r31
r032 r133  0 0 1 0cz W
1 0 0 0 1 0 0 0 1 1

camera axes aligned located at world

with world axes position (0,0,0)

=
Robert Collins

Simple Stereo Projection Equations

CSE486, Penn State

Left camera
Robert Collins
CSE486, Penn State
Example: Simple Stereo System
Y Z
(X,Y,Z)
z
left y
camera
located at ( , )
z
(0,0,0) y ( , )
x right
camera
Tx located at
x (Tx,0,0)

X
Right camera located at world location (Tx,0,0)
and camera axes aligned with world coord axes.
Robert Collins
CSE486, Penn State
Simple Stereo, Right Camera

X 1r11 r012 r013  1 0 0 -T

 cxx U
Y 0r21 r122 r023  0 1 0 0cy V
Z 0r31
r032 r133  0 0 1 0cz W
1 0 0 0 1 0 0 0 1 1

camera axes aligned located at world

with world axes position (Tx,0,0)

=
Robert Collins

Simple Stereo Projection Equations

CSE486, Penn State

Left camera

Right camera
Robert Collins
CSE486, Penn State
Bob’s sure-fire way(s) to
figure out the rotation
X r11 r12 r13  1 0 0  cx U
Y r21 r22 r23  1 about
0forget 0 thiscy V
while thinking
Z r31 r32 r33  0 rotations
0about 1  cz W
1 0 0 0 1 0 0 0 1 1

PC = R P W
This equation says how vectors in the world coordinate
system (including the coordinate axes) get transformed
into the camera coordinate system.
Robert Collins
CSE486, Penn State
Figuring out Rotations
X r11 r12 r13  U
Y r21 r22 r23  V PC = R P W
Z r31 r32 r33  W
1 0 0 0 1 1
what if world x axis (1,0,0) corresponds to camera axis (a,b,c)?

X
a r11 r12 r13  U
1 X
a ra11 r12 r13  U
1
Y
b r21 r22 r23  V
0 Y
b rb21 r22 r23  V
0
Z
c r31 r32 r33  0
W Z
c rc31 r32 r33  0
W
1 0 0 0 1 1 1 0 0 0 1 1
we can immediately write down the first column of R!
Robert Collins
CSE486, Penn State
Figuring out Rotations
and likewise with world Y axis and world Z axis...

same axis in camera coords axis is world coords

X r11 r12 r13  U

Y r21 r22 r23  V
Z r31 r32 r33  W
1 0 0 0 1 1

world X axis (1,0,0) world Z axis (0,0,1)

in camera coords in camera coords
world Y axis (0,1,0)
in camera coords
Robert Collins
CSE486, Penn State
Figuring out Rotations
Alternative approach: sometimes it is easier to specify
what camera X,Y,or Z axis is in world coordinates. Then
do rearrange the equation as follows.

PC = R P W R-1PC = PW RTPC = PW

r11 r21 r31  X U

r12 r22 r32  Y V
r13 r23 r33  Z W
0 0 0 1 1 1
Robert Collins
CSE486, Penn State
Figuring out Rotations
r11 r21 r31  X U
r12 r22 r32  Y V RTPC = PW
r13 r23 r33  Z W
0 0 0 1 1 1
what if camera X axis (1,0,0) corresponds to world axis (a,b,c)?

r11 r21 r31  X

1 U
a ra11 r21 r31  X
1 U
a
r12 r22 r32  Y
0 V
b rb12 r22 r32  Y
0 V
b
r13 r23 r33  Z
0 W
c 0
rc13 r23 r33  Z W
c
0 0 0 1 1 1 0 0 0 1 1 1
we can immediately write down the first column of RT,
(which is the first row of R).
Robert Collins
CSE486, Penn State
Figuring out Rotations
and likewise with camera Y axis and camera Z axis...

same axis in camera coords axis is world coords

X r11 r12 r13  U

Y r21 r22 r23  V
Z r31 r32 r33  W
1 0 0 0 1 1
camera X axis (1,0,0)
in world coords

camera Z axis (0,0,1)

camera Y axis (0,1,0) in world coords
in world coords
Robert Collins
CSE486, Penn State
Example

y
z
x

0 0 1 0 -1 0
Rtrain 0 -1 0 Rfly 0 0 1
1 0 0 -1 0 0
Robert Collins
CSE486, Penn State Note: External Parameters
also often written as R,T
X r11 r12 r13  1 0 0  cx U
Y r21 r22 r23  0 1 0  cy V
Z r31 r32 r33  0 0 1  cz W
1 0 0 0 1 0 0 0 1 1

R ( PW - C ) r11 r12 r13 tx

r21 r22 r23 ty
= R PW - R C
r31 r32 r33 tz
= R PW + T 0 0 0 1
Robert Collins
CSE486, Penn State
Summary

World Camera Film Pixel

Coords Coords Coords Coords
U X x u
V Y y v
W Z
We now know how to transform 3D world
coordinate points into camera coords, and
then do perspective project to get 2D points
in the film plane.

Next time: pixel coordinates