![This poster presents two methods applicable to the calibration problem of
two cameras with non-overlapping field-of-view. The linear method is then
implemented to find the relative pose of two Kinect cameras pointing in
opposite directions. The result is promising but undetermined error still exists.
Future work needs to be done to modify the rotation averaging algorithm. The
current algorithm works on synthetic data but diverges on real data.
Additional simulation model could be generated to precisely mimic the real
world situation. The system could also be improved by providing extrinsic
matrix with less noise.
Camera Network Calibration with Non-overlapping Views
Baiwu Zhang1 and K.H. Wong2
1 Department of Electrical & Computer Engineering, University of Toronto, Canada
2 Department of Computer Science and Engineering, Chinese University of Hong Kong, Hong Kong
Introduction
Camera calibration plays an important role in various fields, including
computer vision, robotics, autonomous vehicle and virtual reality. Methods for
direct calibration of a single camera has been fully developed, but how to
calibrate multiple cameras of one system is still an open problem.
This poster discusses the calibration of a camera network using mirrors.
Multiple cameras pointing into different directions such that their fields of
view are no longer overlapping can not be calibrated using conventional
checkerboard-based method. We implemented two algorithms to tackle this
problem. Results on both calibrating synthetic and real data are demonstrated
as well as the implementation on a non-overlapping pair of Kinect cameras on
top of a robot.
Problem Setup
Let R be the extrinsic matrix of a camera from direct calibration, and 𝑹 be the
mirror reflection of R. We use normal n and scalar distance d to represent a
mirror plane. Each planar motion is shown as T and the rigid transformation is
expressed in terms of 𝜽, 𝝎 and t.
Linear Method [1]
In this method, camera extrinsic matrix R from direct calibration are
transformed into virtual cameras with a symmetry transformation
S =
𝐼 − 2𝑛𝑛𝑇 2𝑑𝑛
0 1
.
Then equation (1) can be constructed for each set of pictures.
ti x + tan 𝜃𝑖
2
𝜔𝑖 𝑡𝑖
𝑇 𝑛 − 2 tan 𝜃𝑖
2
𝜔𝑖 𝑑0 = 0 1
The system of equations can be written in the following form
𝐵1 𝑏1 0
𝐵2 0 𝑏2
… 0
… 0
⋮ ⋮ ⋮
B 𝑁 − 1 0 0
⋱ ⋮
… 𝑏 𝑁 − 1
𝑛0
𝑑0
𝑑1
𝑑2
⋮
d 𝑁 − 1
= 0 (2)
as Bi been 4x4 sub-matrices
Bi =
ti
T −2
ti x 0
and bi vectors with dimension 4
bi =
2 cos 𝜃𝑖
2
−2 sin 𝜃𝑖
2
𝜔𝑖
With N≥3, equation (2) can be build to find a least square solution using
SVD. Then R can be recovered.
Rotation Averaging Method [2]
Experiments Result
As shown in Figure 5, a double-
sided checkerboard is placed in the
middle of two Kinect cameras such
that they can observe the
checkerboard only via mirror
reflections. 15 images are captured
by each camera for individual
mirror angles. Each camera pose
(shown in Figure 6) is recovered
from MATLAB Calibration toolbox.
Conclusion & Future Work
Acknowledgment
I am grateful to Professor K.H. Wong and Calvin Kam for guiding me
throughout this research. I also want to thank my lab partners and all other
friends I met here for an incredible experience in Hong Kong.
References
Figure 1. The geometry of the mirror-based
camera pose estimation problem [1].
Figure 2. The virtual camera 𝐶 and the real
camera C are symmetric to each other with
respect to the mirror Π [1].
Two steps are involved in rotation averaging method. First step is to try
minimized the equation (3),
𝐸 𝑅, 𝑛𝑖 =
𝑖=1
𝑛
𝑅𝑖 𝐼 − 2𝑛𝑖𝑛𝑖𝑇 − 𝑅 𝐹
2
3
which aims to find the R. Equation (3) describes the error between the R
that we want to find, and all the virtual camera models recovered from
direct calibration. Minimization of (3) turns out to be equivalent to
maximizing E2,
𝐸2 =
𝑖=1
𝑛
𝑅𝑖 𝑛𝑖 𝑛𝑖
𝑇, 𝑅 =
𝑖=1
𝑛
𝑛𝑖
𝑇 𝑅𝑖
𝑇 𝑅𝑛𝑖
which is also equivalent to finding the rotation R that is closest to G =
𝑖=1
𝑛
𝑅𝑖. A closed-form solution can be obtained by SVD.
The second step utilizes the result from the first step as the initial value, and
then applies gradient descent to the projection error between the current R
and all other virtual cameras models. Details of the algorithm is shown
below.
Figure 3. Rotation averaging algorithm step 2 [2]. Figure 4. Error comparison of step 1 and 2 on synthetic
data.
Checkerboard
Kinect 2Kinect 1
Mirror
Figure 5. Calibration of two Kinect cameras with a mirror.
Figure 6. Direct calibration result of one camera from
MATALB.
Figure 7. 3D model generated from the linear
method.
We apply the linear algorithm on the result generated from the previous
steps, and plot the final 3D model of the two non-overlapping cameras in
Figure 7.
Based on our model, the two Kinect cameras are rotated to develop a point
cloud of their surroundings. The accuracy of the mapping of two point cloud
is increased compared to the estimated model. Results are shown below.
Figure 8. Point cloud with estimated model. Figure 9. Point cloud with the model in Figure 7.
[1] R. Rodrigues, P. Barreto, and U. Nunes. Camera pose estimation using
images of planar mirror reflections. In Proceedings of the European Conference
on Computer Vision (ECCV), Crete, Greece, 2010.
[2] G. Long, L. Kneip, X. Li, X. Zhang, and Q. Yu. Simplified Mirror-Based Camera
Pose Computation via Rotation Averaging. International Conference on
Computer Vision(ICCV), Santiago, Chile, 2015.](https://image.slidesharecdn.com/75ada283-e061-458b-93e2-e5070e412424-160811020342/85/poster-1-320.jpg)
poster
- 1. This poster presents two methods applicable to the calibration problem of two cameras with non-overlapping field-of-view. The linear method is then implemented to find the relative pose of two Kinect cameras pointing in opposite directions. The result is promising but undetermined error still exists. Future work needs to be done to modify the rotation averaging algorithm. The current algorithm works on synthetic data but diverges on real data. Additional simulation model could be generated to precisely mimic the real world situation. The system could also be improved by providing extrinsic matrix with less noise. Camera Network Calibration with Non-overlapping Views Baiwu Zhang1 and K.H. Wong2 1 Department of Electrical & Computer Engineering, University of Toronto, Canada 2 Department of Computer Science and Engineering, Chinese University of Hong Kong, Hong Kong Introduction Camera calibration plays an important role in various fields, including computer vision, robotics, autonomous vehicle and virtual reality. Methods for direct calibration of a single camera has been fully developed, but how to calibrate multiple cameras of one system is still an open problem. This poster discusses the calibration of a camera network using mirrors. Multiple cameras pointing into different directions such that their fields of view are no longer overlapping can not be calibrated using conventional checkerboard-based method. We implemented two algorithms to tackle this problem. Results on both calibrating synthetic and real data are demonstrated as well as the implementation on a non-overlapping pair of Kinect cameras on top of a robot. Problem Setup Let R be the extrinsic matrix of a camera from direct calibration, and 𝑹 be the mirror reflection of R. We use normal n and scalar distance d to represent a mirror plane. Each planar motion is shown as T and the rigid transformation is expressed in terms of 𝜽, 𝝎 and t. Linear Method [1] In this method, camera extrinsic matrix R from direct calibration are transformed into virtual cameras with a symmetry transformation S = 𝐼 − 2𝑛𝑛𝑇 2𝑑𝑛 0 1 . Then equation (1) can be constructed for each set of pictures. ti x + tan 𝜃𝑖 2 𝜔𝑖 𝑡𝑖 𝑇 𝑛 − 2 tan 𝜃𝑖 2 𝜔𝑖 𝑑0 = 0 1 The system of equations can be written in the following form 𝐵1 𝑏1 0 𝐵2 0 𝑏2 … 0 … 0 ⋮ ⋮ ⋮ B 𝑁 − 1 0 0 ⋱ ⋮ … 𝑏 𝑁 − 1 𝑛0 𝑑0 𝑑1 𝑑2 ⋮ d 𝑁 − 1 = 0 (2) as Bi been 4x4 sub-matrices Bi = ti T −2 ti x 0 and bi vectors with dimension 4 bi = 2 cos 𝜃𝑖 2 −2 sin 𝜃𝑖 2 𝜔𝑖 With N≥3, equation (2) can be build to find a least square solution using SVD. Then R can be recovered. Rotation Averaging Method [2] Experiments Result As shown in Figure 5, a double- sided checkerboard is placed in the middle of two Kinect cameras such that they can observe the checkerboard only via mirror reflections. 15 images are captured by each camera for individual mirror angles. Each camera pose (shown in Figure 6) is recovered from MATLAB Calibration toolbox. Conclusion & Future Work Acknowledgment I am grateful to Professor K.H. Wong and Calvin Kam for guiding me throughout this research. I also want to thank my lab partners and all other friends I met here for an incredible experience in Hong Kong. References Figure 1. The geometry of the mirror-based camera pose estimation problem [1]. Figure 2. The virtual camera 𝐶 and the real camera C are symmetric to each other with respect to the mirror Π [1]. Two steps are involved in rotation averaging method. First step is to try minimized the equation (3), 𝐸 𝑅, 𝑛𝑖 = 𝑖=1 𝑛 𝑅𝑖 𝐼 − 2𝑛𝑖𝑛𝑖𝑇 − 𝑅 𝐹 2 3 which aims to find the R. Equation (3) describes the error between the R that we want to find, and all the virtual camera models recovered from direct calibration. Minimization of (3) turns out to be equivalent to maximizing E2, 𝐸2 = 𝑖=1 𝑛 𝑅𝑖 𝑛𝑖 𝑛𝑖 𝑇, 𝑅 = 𝑖=1 𝑛 𝑛𝑖 𝑇 𝑅𝑖 𝑇 𝑅𝑛𝑖 which is also equivalent to finding the rotation R that is closest to G = 𝑖=1 𝑛 𝑅𝑖. A closed-form solution can be obtained by SVD. The second step utilizes the result from the first step as the initial value, and then applies gradient descent to the projection error between the current R and all other virtual cameras models. Details of the algorithm is shown below. Figure 3. Rotation averaging algorithm step 2 [2]. Figure 4. Error comparison of step 1 and 2 on synthetic data. Checkerboard Kinect 2Kinect 1 Mirror Figure 5. Calibration of two Kinect cameras with a mirror. Figure 6. Direct calibration result of one camera from MATALB. Figure 7. 3D model generated from the linear method. We apply the linear algorithm on the result generated from the previous steps, and plot the final 3D model of the two non-overlapping cameras in Figure 7. Based on our model, the two Kinect cameras are rotated to develop a point cloud of their surroundings. The accuracy of the mapping of two point cloud is increased compared to the estimated model. Results are shown below. Figure 8. Point cloud with estimated model. Figure 9. Point cloud with the model in Figure 7. [1] R. Rodrigues, P. Barreto, and U. Nunes. Camera pose estimation using images of planar mirror reflections. In Proceedings of the European Conference on Computer Vision (ECCV), Crete, Greece, 2010. [2] G. Long, L. Kneip, X. Li, X. Zhang, and Q. Yu. Simplified Mirror-Based Camera Pose Computation via Rotation Averaging. International Conference on Computer Vision(ICCV), Santiago, Chile, 2015.