Scribd
Upload
7 views

PAS 7 Epipolar Geometry 2

Uploaded by

feolivos
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
7 views

PAS 7 Epipolar Geometry 2

Uploaded by

feolivos
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 29

Perception for Autonomous Systems 31392:

Epipolar Geometry and


the Fundamental Matrix

Lecturer: Evangelos Boukas—PhD

10 Feb. 2020 DTU Electrical Engineering 2


Epipolar Geometry: General Case

• Assuming:
– 2 Camera Views
– A ray passing through
the camera center

10 Feb. 2020 DTU Electrical Engineering 3


Epipolar Geometry: General Case

• Assuming:
– 2 Camera Views
– A ray passing through
the camera center
• We can find the:
– baseline and
– the epipolar lines

10 Feb. 2020 DTU Electrical Engineering 3


Epipolar Geometry: General Case

• Assuming:
– 2 Camera Views
– A ray passing through
the camera center
• We can find the:
– baseline and
– the epipolar lines
– Through the epipolar plane

10 Feb. 2020 DTU Electrical Engineering 3


Epipolar Geometry

• What about this case?

10 Feb. 2020 DTU Electrical Engineering 6


Epipolar Geometry

• What about this case?


• What’s the epipolar plane?

10 Feb. 2020 DTU Electrical Engineering 6


Epipolar Geometry

• What about this case?


• What’s the epipolar plane?
• Can we find the epipole of I1?

10 Feb. 2020 DTU Electrical Engineering 6


Epipolar Geometry

• What about this case?


• What’s the epipolar plane?
• Can we find the epipole of I1?
• What about the epipole of I2?

10 Feb. 2020 DTU Electrical Engineering 6


Epipolar Geometry:

• How about this case?

10 Feb. 2020 DTU Electrical Engineering 10


Epipolar Geometry:

• How about this case?


• Where are the epipoles?

10 Feb. 2020 DTU Electrical Engineering 10


Epipolar Geometry:

• How about this case?


• Where are the epipoles?

• How would such a case be


usefull
– The scanline is used in
Disparity calculation

10 Feb. 2020 DTU Electrical Engineering 10


Example of Epipolar Lines “in the wild”

• Let’s think of this


example:

• Where should the epipole be?

10 Feb. 2020 DTU Electrical Engineering 13


Example of Epipolar Lines “in the wild”

• Let’s think of this


example:

• Where should the epipole be?

10 Feb. 2020 DTU Electrical Engineering 13


Example of Epipolar Lines “in the wild”

• What about this:

• How should the epipolar Lines


look like?

10 Feb. 2020 DTU Electrical Engineering 15


Example of Epipolar Lines “in the wild”

• What about this:

• How should the epipolar Lines


look like?

10 Feb. 2020 DTU Electrical Engineering 15


Essential Matrix

• Assuming two calibrated stereo pairs:


• We can express the points x, y
from the image plane to homogeneous
෡ using the inverse
coordinates 𝑥ො and 𝑥′
of the camera matrix
xˆ = K −1 x = X
xˆ  = K −1 x  = X 

10 Feb. 2020 DTU Electrical Engineering 17


Essential Matrix

• We can express the left homogeneous point


to the right one:
• 𝑥ො = R ∗ 𝑥෡′ + 𝑇, where R is the Rotation Matrix
and T the translation vector
• The we can prove that there is a Matrix
connecting the two points 𝑥, ො 𝑥෡′
• Trying to eliminate the left side by
Applying cross product and then dot product
T 𝐱 𝑥ො = T 𝐱 R ∗ 𝑥෡′ + 𝑇 𝐱 𝑇
𝑥ො . T 𝐱 𝑥ො = 𝑥ො . T 𝐱 R ∗ 𝑥෡′ + 0
0 = 𝑥ො . T 𝐱 R ∗ 𝑥෡′

• Or we can write it in matrix form: xˆ E xˆ  = 0


T
with E = t  R , where [t]x is the skew symmetric matrix

10 Feb. 2020 DTU Electrical Engineering 18


Essential Matrix

xˆ T E
• The essential with E = t  R is a 3x3 matrix,
xˆ  = 0 matrix
for which:
– E x’ is the epipolar line associated with x’ (l = E x’)
– ETx is the epipolar line associated with x (l’ = ETx)
– E e’ = 0 and ETe = 0
– E is singular (rank two)
– E has five degrees of freedom

10 Feb. 2020 DTU Electrical Engineering 19


Fundamental Matrix

• We know how to get from a homogeneous point


in one camera to another
• How can we get directly from one image to another?

ˆxT E xˆ = 0
xˆ = K −1 x xT F x = 0 with F = K −T E K −1
xˆ  = K −1 x 
Which is the fundamental matrix

10 Feb. 2020 DTU Electrical Engineering 20


Fundamental Matrix

xT F x = 0 matrix
• The fundamental with F = K −T E K −1 is a 3x3 matrix,
for which:
– F x’ is the epipolar line associated with x’
– FTx is the epipolar line associated with x
– F e’ = 0 and FTe = 0
– F is singular (rank two): det(F)=0
– F has seven degrees of freedom:

10 Feb. 2020 DTU Electrical Engineering 21


How to compute the ___________
Homography Fundamental Matrix

• Similar to DLT method as before


It’s called the 8 point algorithm as we need 8 points to solve it

xT F x = 0
𝑥𝑥 ′ 𝑓11 + 𝑥𝑦 ′ 𝑓12 + 𝑥𝑓13 + 𝑦𝑥 ′ 𝑓21 + 𝑦𝑦 ′ 𝑓22 + 𝑦𝑓23 + 𝑥 ′ 𝑓31 + 𝑦 ′ 𝑓32 + 𝑓33 = 0

𝑓11
𝑓12
𝑥1 𝑥1 ′ 𝑥1 𝑦1 ′ 𝑥1 𝑦1 𝑥1 ′ 𝑦1 𝑦1 ′ 𝑦1 𝑥1 ′ 𝑦1 ′ 1
𝑓
A𝒇 = ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ ⋮ 13 =0
𝑓
𝑥𝑛 𝑦𝑣′ 𝑥𝑛 𝑦𝑛 ′ 𝑥𝑛 𝑦𝑛 𝑥𝑛 ′ 𝑦𝑛 𝑦𝑛 ′ 𝑦𝑛 𝑥𝑛 ′ 𝑦𝑛 ′ 1 21
⋮
𝑓33

10 Feb. 2020 DTU Electrical Engineering 22


How to compute the Fundamental Matrix

• Homography (No Translation) • Fundamental Matrix (Translation)

• Correspondence Relation • Correspondence Relation


x' = Hx  x'Hx = 0 xT Fx = 0
1. Normalize image coordinates 1. Normalize image coordinates
x = Tx ~
~ x  = Tx x = Tx ~
~ x  = Tx
2. RANSAC with 4 points 2. RANSAC with 8 points
– Solution via SVD – Initial solution via SVD
3. De-normalize:
~
()
– Enforce det F = 0 by SVD
~ 3. De-normalize:
H = T−1HT ~
F = T T FT

10 Feb. 2020 DTU Electrical Engineering 23


Epipolar Geometry to Rectified Epipolar Geometry

• To finally go full circle:


– To be able to use the stereo in a
block matching algorithm to produce
3D points we must first
convert to rectified stereo
– How would you do that?

10 Feb. 2020 DTU Electrical Engineering 24


Rectified Epipolar Geometry

10 Feb. 2020 DTU Electrical Engineering 25


An Example

• Assume these two images

10 Feb. 2020 DTU Electrical Engineering 26


An Example

• First step is to match some points

• Next, calculate the fundamental matrix

10 Feb. 2020 DTU Electrical Engineering 27


An Example

• Finally calculate the rectified images

• Notice how the images are now scan line

10 Feb. 2020 DTU Electrical Engineering 29


Perception for Autonomous Systems 31392:

Epipolar Geometry and


the Fundamental Matrix

Lecturer: Evangelos Boukas—PhD

10 Feb. 2020 DTU Electrical Engineering 29

You might also like