2017 3rd International Conference on Control, Automation and Robotics

Increasing the Accuracy for Computer Vision Systems by Removing the Noise

Zhenzhou Wang
State Key Lab for Robotics, Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
e-mail: [email protected]

Abstract-Robots rely on the computer vision systems to obtain known markers on the designed pattern are used to calculate
the environmental information. As a result, the accuracy of the the system parameters. Before they can be used for
computer vision systems is essential for the control of the calibration, the two-dimensional coordinates of these points
robots. Many computer vision systems make use of markers of in the camera view need to be computed, where the noise is
the well-designed patterns to calculate the system parameters.
introduced when the coordinates of the points are computed
Undesirably, the noise exists universally, which decreases the
as the Gaussian mean of all the comer or bright pixels.
calibration accuracy and consequently decreases the accuracy
Consequently, calibration error is caused by the noise. To
of the computer vision systems. In this paper, we propose a
pattern modeling method to remove the noise by decreasing
remove the noise caused error, we propose a pattern
the degree of freedom of the total calibration markers to one.
modeling method in this paper to reduce or eliminate the
The theorem is proposed and proved. The proposed method noise by reducing the degree of freedom of all the
can be readily adopted by different computer vision systems, calibration markers to one. Experimental results show that
e.g. structured light based computer vision systems and stereo the proposed method is effective in removing the noise in
vision based systems. the developed structured light computer vision systems [1-4],
[8], [9]. The accuracy improvement is significant in the
Keywords-degree of freedom; noise; pattern modeling; sense of mean squared errors. We use the proposed method
computer vision to remove the noise for camera calibration [10] and its
effectiveness is also verified by the experimental results.
Since camera calibration is fundamental to most computer
I. INT RODUCTION
vision applications, the proposed method has the potential to
Computer vision systems are becoming more and more benefit them greatly.
popular in robotics. It is believed that the next generation
robots should mainly rely on the computer vision systems to II. THE P ROPOSED P ATTERN MODE L IN G METHOD
control their behaviors just as human beings do. Many
computer vision systems [1]-[9] utilize some well-designed The pattern modeling method is implemented based on
pattern for calibration. Structured light based computer the distances between the center point and the other points
vison systems project the designed patterns onto the surface and contains the following steps:
Step 1: Model the rays with points in the designed
of the object to be measured. From the distorted pattern, the
surface's profile could be calculated based on the system's pattern and the projection center C(xc,yc,zc)'
The unit of
parameters calculated from a set of calibrated markers. Thus, the modeled points can be chosen as convenient as pixel or
the calibration accuracy is critical for the measurement as rum depending on the convenience of the application.
accuracy of the structured light based computer vision The modeled rays are formulated by Eq. 1.
systems. Traditionally, these computer vision systems use as
many calibration markers as they can until the calibration XI1I-Xc =yl1l_yc =zl1l-zc =tm (1)
accuracy improvement reached zero growth. The camera Xi -Xc Yi - Yc Zi - Zc I

calibration [10] makes use of multiple views of the well­

designed pattern to calculate intrinsic and extrinsic where (Xi' Yi, Zi) is the i th point in the designed pattern.
parameters. Similarly, it also uses as many views of the
designed pattern as it can until the calibration accuracy Step 2: Use the plane ax +by +cz =1 to intercept the
could not be improved any more. Unfortunately, none of modeled rays. Then compute the distances between the
them could avoid the annoying noise in both calibration center points and a set of points around it.
stage and measurement stage even if they might reduce the
noise to a low level by averaging more markers or views of
the designed pattern. The noise induced error during the
calibration stage may affect the measurement accuracy of Step 3: For the captured pattern, compute the distances
the computer vision systems greatly for some techniques [6] between the center point and the same set of points as those
and limits their applications undesirably. used in Step 2 by the following equation.
During camera calibration or the structured light based
3D scanning computer vision system calibration, a set of d; =�(x;-xtr +(y;-ytr +(z;-ztf (3)

978-1-5090-6088-7117/$31.00 ©2017 IEEE 371

 Step 4: Compute the total difference of all the distances the total mean squared error between the modeled points
by the following equation. and the original points.

",d = L",d = L i d"' -dp l

I I I
(4)

Step 5: Find the optimal interception plane P(a,b,c)

that makes ",d minimum.

P =arg min",d (5)

I'

The modeled points are computed in a virtual coordinate

system instead of the world coordinate system. A
registration is thus needed between the modeled points and
the practically intercepted points to convert the coordinates
correctly. We register the two set of points based on the
least square errors by finding the transformation matrix A
that makes the sum of square errors, dr minimum.
G

[
- I'
Xi
"
al2 al3 al4 X�n
I

-I'
Yi
=llI
a21 a22 a23 a2 4 y;" (6)
Zi
-I' a31 a32 a33 a3 4 Zi
11/

a41 a42 a4 3 a44

Figure 1. Tllustration of noise; (a) Designed pattern; (b) Captured pattern; (c)
plot of the X coordinate differences in pixels; (d) plot of the y coordinate
N
differences in pixels. (The y axis label is mm and x axis label is index
dr = 2: ( X;p -x{'f +(y{' - y{'f +(=;1' -z{'f (7) number for (c) and (d))
i=1

A =arg mindr (8)

A

[�,
al2 a13 "

j
a21 a22 a23 a2 4
A= (9)
a31 a32 a33 a3 4
a4 1 a4 2 a43 a44
(a) (b)
where III is a constant and N denotes the totoal number of
Figure 2. Differences after modeling: (a) X coordinate differences; (b) y
points in the registered pattern. After pattern modeling, all
coordinate differences.
the computed points in the captured pattern are replaced
with the modeled points in the ideal pattern. Then they are
used for system calibration.

III. IMP LEMENT AnON OF THE P ROPOSED METHOD

For the practical implementation, the searching range to

find the optimal parameters (a,b,c) is limited since it will be
intractable to search all the possible values thoroughly. In
our conducted experiments, we choose the searching ranges 2000 5 10 152S 20
JO JS 40 45 lS00!-+---:'�O �"----:;"
:---:''' ----:;:-
03 --:'53 ----;'�O ---!".
as aE[-50,50], bE[-50,50] and cE[-50,50] respectively. (a) (b)
Figure 3. Results of modeling the pattern in the camera view (Fig. 1 (b))
The complexity of the searching is 1000000 and it takes less with the searched projection center (a) Modeled and original X coordinates;
than one minute in MATLAB. (b) Modeled and original y coordinates.
Since the searching range is fixed, the center used to
We use the projected laser dot pattern to demonstrate the
model the rays will affect the equations of the modeled rays,
process of removing the noise by the proposed method. Fig.
which might in turn affect the [mal pattern modeling
1 (a) shows the designed pattern and it is projected by a Pico
accuracy significantly. Thus, an additional search around the
Laser Projector onto a horizontal diffusive plane. The
arbitrarily assigned center is performed to find a center that
brightest point in the center denotes the center marker. The
could yield more accurate registration results by minimizing

372
camera captures the projected pattern as shown in Fig. 1 (b). IV. EXPE R IMENT A L RE SULT S
These laser dots are segmented by the method proposed in
[11-12]. The coordinates of the bright laser dots are Firstly, we model a camera calibration pattern [10] in
calculated based on the Gaussian mean of the bright pixels. Fig. 4. We see obvious mismatches between the modeled y
To demonstrate the noise, we select 44 points around the coordinates and the original y coordinates. The mean
center laser dot and plot the differences of the x coordinates squared errors are l.8155 and 2.8142 respectively. We then
and y coordinates between adjacent laser dots respectively. search around the projection center within the range [-5, 5]
Fig. I (c) and (d) show the calculated x coordinate in three dimensions. We found the new center that could
differences and y coordinate differences respectively. Based yield the minimum MSEs, which are reduced to 1.074 and
l.6262 respectively. The new modeling results are shown in
on the designed pattern, the differences should be a constant
Figure 5. It is seen that the modeled points and original ones
without noise while noise adds the random variations. These
match significantly better than those in Fig. 4.
random variations are the introduced noise. We plot the
differences of the x and y coordinate for these 45 points
before modeling and after modeling in Fig. 2 (a) and (b)
respectively. As can be seen, the noise (random variation) is
eliminated successfully. For the modeled results shown in
Fig. 2, the computed mean squared errors (MSE) are 2.8061
and 2.0916 for the x and y coordinate respectively. We
search a new projection center in a small range [-5, 5] in
three dimensions and find the center that yields the
minimum MSE. The MSEs are reduced to 2.2204 and 5 10 15 � � » � � 5 10 15 " "

1.6357 respectively. Fig. 3 shows the modeled coordinates (a) (b)

after pattern modeling (in red) versus the original original
coordinates (in blue). The modeled y coordinates with the
new center match better than the modeled y coordinates os

with the original center, which indicates that searching the

optimal parameters is a challenging engineering problem .5
that needs great effort. With the proposed pattern modeling
.,
method, the accuracy of the computer vision system 600
.., ..,
proposed in [9] could be improved from millimeter to
femtometre [13].
(c) (d)

Figure 6. Illustration of camera calibration pattern modeling with another

found center: (a) Modeled and original x coordinates; (b) Modeled and
original y coordinates; (c)-(d) Modeled points against the original points.

Secondly, we model the calibration grid for the

computer vision system in [1]-[4]. We model the points on
the two planes (the blue points on the back plane and the red
points on the front plane as shown in Fig. 6) independently.
(a) (b) The modeled points versus the two sets of computed points
on the two planes (Fig. 6 (b)) are shown in Fig. 7 (a) and (b)
respectively. As can be seen, they match well, which
indicates that it is correct to apply the proposed pattern
modeling method in phase shift profilometry 3D imaging
system [1]-[4]. The measurement accuracy with and without
pattern modeling are 0.65 mm and 0.95 mm respectively for
measuring the object with the length about 1000 mm. The
measurement accuracy percentages are thus 0.00065 and
0.00095 respectively.
Please note that the proposed pattern modeling method
(c) (d)
has the upper limit in improving the accuracy of some
Figure 4. Illustration of camera calibration pattern modeling: (a) Captured computer vision systems. In other words, the system noise
calibration pattern; (b) Modeled points against the original points; (c)
always exist for some computer vision systems, e.g. the
Modeled and original x coordinates; (d) Modeled and original y
phase shift profilometry [1-4]. Only when the system is
coordinates.
based on analytical solutions, its accuracy could be
improved to be as close as zero [12].

373
 (x/ ,y/ ,y/) i = 1,... ,28 in the world coordinate sytem by the phase shift
profilometry 3D imaging system.

V. CONC LUS ION

In this paper, a pattern modeling method is proposed to

remove the noise by reducing the degree of the freedom of
the all the calibration markers (points) to one. As a result,
the calibration error caused by the noise can be removed
effectively. Both the camera and the projector can be
modeled by the proposed method based on the requirements
of the practical computer vision systems. Experimental
(a) results validated the effectiveness of the proposed method.
In the future, we will apply the proposed method to more
computer vision systems.

AC KNO W LED GMENT

This work was supported by Chinese Academy of

Sciences with grant No. Y5A1270101.

REFERENCE S

[I] Z. Z. Wang, 'Three-dimensional surface imaging by multi-frequency

phase shift profilometry with angle and pattern modeling for system
calibration " Measurement Science and Technology, 27(8):085404,
2016.
(b) [2] K. Liu, Y. Wang, D. L. Lau, Q. Hao and L. G. Hassebrook, " Dual­
Figure 7. Registered points versus computed points: (a) For the points on frequency pattern scheme for high-speed 3-D shape measurement,
the front plane (the red points shown in Fig. 5 (a»; (b) For the points on the "Optics Express, Vol. 18,No. 5,pp. 5229-5244,2010.
back plane (the blue points shown in Fig. 5 (a». (the scale is mm). [3] Z. Z. Wang, "Robust measurement of the diffuse surface by phase
shift profilometry," J Optics. 15(10),105407 (2014).
[4] C. Guan, L. G. Hassebrook, D. L. Lau, andV. G. Yalla, "Improved
composite-pattern structured light profilometry by means of
postprocessing," Opt Eng. Vol. 47, No.9,pp. 097203-11,2008.
[5] H. Kawasaki, H. Masuyama, R. Sagawa, and R. Furukawa, "Single
colour one-shot scan using modified Penrose tiling pattern," iET
Comput. Vis. Vol. 7,No. 5,pp. 293-301, 2013.
[6] L. Huang, C. S. Ng, and A. K. Asundi, "Dynamic three-dimensional
sensing for specular surface with monoscopic fringe reflectometry,"
Opt Express Vol. 19, No. 13,pp. 12809-12814,2011.
[7] W. Jang, C. Je, Y. Seo, and S. W. Lee, "Stuctured-light stereo:
Comparative analysis and integration of structured-light and active
stereo for measuring dynamic shape," Opt Lasers Eng. Vol. 51, No.
(a)
11,pp. 1255-1264,2013.
x
[8] Z. Z. Wang,X. Y. Huang, R. G. Yang and Y. M. Zhang, "Measurement
of mirror surfaces using specular reflection and analytical
computation," MACH ViSiON APPL. Vol. 24, No. 2, pp. 289-304,
2013.
[9] Z. Z. Wang, "A one-shot-projection method for measurement of
specular surfaces," Opt Express. 23(3), 1912-1929 (2015).
[10] Z. Y. Zhang,"A flexible new technique for camera calibration," iEEE
Trans. on PAMi Vol. 22,No. l l,pp. 1330-1334,2000.
[11] Z. Z. Wang, "A new approach for segmentation and quantification of
cells or nanoparticles," iEEE T iND iNFORM. 12 (3), 962-971,
(2016).
[12] Z. Z. Wang, "Automatic segmentation and classification of the
reflected laser dots during analytic measurement of mirror surfaces,"
Optics and Lasers in Engineering. 83 (3),10-22, (2016).
(b)
[13] z.z. Wang, "Improving the accuracy of mirror measurements by
Figure 5. Two sets of points used for calibration: (a) Designed points in the removing noise and lens distortion," Measurement Science and
virtual space (X;w,Y,w, z;w) i = 1,...,28; (b) Computed points Technology. 27 (11),115008, (2016).

374