Performance analysis on color image mosaicing techniques on FPGA

International Journal of Electrical and Computer Engineering (IJECE)
Vol. 9, No. 3, June 2019, pp. 1630~1636
ISSN: 2088-8708, DOI: 10.11591/ijece.v9i3.pp1630-1636  1630
Journal homepage: http://iaescore.com/journals/index.php/IJECE
Performance analysis on color image mosaicing
techniques on FPGA
Jayalaxmi H 1
, S. Ramachandran2
1
Department of Electronics & Communication Engineering, Acharya Institute of Technology, India
2
Department of Electronics & Communication Engineering, SJBIT, India
Article Info ABSTRACT
Article history:
Received Jun 11, 2018
Revised Dec 3, 2018
Accepted Dec 15, 2018
Today, the surveillance systems and other monitoring systems are
considering the capturing of image sequences in a single frame. The captured
images can be combined to get the mosaiced image or combined image
sequence. But the captured image may have quality issues like brightness
issue, alignment issue (correlation issue), resolution issue, manual image
registration issue etc. The existing technique like cross correlation can offer
better image mosaicing but faces brightness issue in mosaicing. Thus, this
paper introduces two different methods for mosaicing i.e., (a) Sliding
Window Module (SWM) based Color Image Mosaicing (CIM) and (b)
Discrete Cosine Transform (DCT) based CIM on Field Programmable Gate
Array (FPGA). The SWM based CIM adopted for corner detection of two
images and perform the automatic image registration while DCT based CIM
aligns both the local as well as global alignment of images by using phase
correlation approach. Finally, these two methods performances are analyzed
by comparing with parameters like PSNR, MSE, device utilization and
execution time. From the analysis it is concluded that the DCT based CIM
can offers significant results than SWM based CIM.
Keywords:
CIM
DCT
Device utilization
Execution time
FPGA
Image mosaicing
MSE
PSNR
SWM Copyright © 2019 Institute of Advanced Engineering and Science.
All rights reserved.
Corresponding Author:
Jayalaxmi H,
Department of Electronics & Communication Engineering,
Acharya Institute of Technology, Bengaluru, India.
Email: jayalaxmiresearchwork@gmail.com
1. INTRODUCTION
The customized advancement of high-resolution image mosaics is a dynamic research domain
ofcomputer graphics, vision and image processing [1]. The image mosaicing is normally used to fabricate the
visual view field by sticking together various video sequences. The camera's view field is always less than
human view field [2]. Also, large objects often can't be caught in a single image as in aerial photography.
Using a convergence lens for broader view is a kind of solution but it will have some sort of distortion in
capturing the image. Also, the capturing the entire scene with less resolution may also induce image quality
issues [3]. The panoramic image mosaics can be created by using some devices like Video Recording,
surround video thatmoves around the camera optical core interest [1]. Also, it has strong imperatives on the
imaging conditions. Thus, to overcome this, image alignment, image saucing and image frames pasting can
be used to get complete view of image [4].
The minimum complex mosaics are produced by using an image sets whose mutual displacement is
a pure image place translation. This is the approximate case which takes place in satellite images translation.
The image translation can either be handled by physically demonstrating corresponding centers or by an
image correlation [5]. Other direct mosaics are generated by rotating the camera at its optical concentration
by a device and generating a panoramic image that indicates the scene projection over a cylinder. But is quite
tough task to have clear image under different light intensity, noise, orientation, alignment etc [5].

Int J Elec & Comp Eng ISSN: 2088-8708 
Performance analysis on color image mosaicing techniques on FPGA (Jayalaxmi H)
1631
The minimum complex mosaics are produced by using an image sets whose mutual displacement is a pure
image place translation. This is the approximate case which takes place in satellite images translation.
The image translation can either be handled by physically demonstrating corresponding centers or by an
image correlation [5]. Other direct mosaics are generated by rotating the camera at its optical concentration
by a device and generating a panoramic image that indicates the scene projection over a cylinder. But is quite
tough task to have clear image under different light intensity, noise, orientation, alignment etc [5].
This section evolves with the discussion on existing works on image mosaicing techniques.
The image mosaicing is a concept of combining two or more image and forms asingle image without any
visible seam lines. A survey work on image mosaicing is performed in Ghosh and Kaabouch [6] that discuss
existing image mosaicing algorithms, classification of mosaicing technique, state of art in current research
trend etc. In order to construct the mosaiced image various algorithms and techniques were introduced [7].
Most of the research methods have employed significant point matching techniques and corner detection
models [8]. An interesting work of Zagrouba et al. [9] have introduced Harris points primitives [10] and
regions based image mosaicing concept which helps to improve the mosaicing performance against the issues
like illumination variation, brightness, noise etc. In the work of Zhu and Ren [11] Scale Invariant Feature
Transform (SIFT) [12] based image mosaicing method is illustrated. Primarily, the method uses operator
from Harris corner detection [13] and detects the key points. Then constructs directed line segments for
rough point matching. The final outcomes of the method suggest its robustness against rotation, scaling,
resolution and lighting issues. The work of Elibol et al. [14] has introduced underwater image mosaicing by
using submapping. This approach utilizes a modified agglomerative clustering (hierarchical) mechanism to
form the submaps based on similarity among the feature matching among the image which reduces
computational cost. The real time image mosaicing is introduced in Kekec et al. [15] for aerial images.
By Jaziri [16] the algorithm has been developed by using a low-cost FPGA. This research study also
presented a capable design methodology which provides benefits of significant design. The major benefits
show the re-configurable hardware modules for the electrical methods. The outcomes showed the model
platform that presents the effectiveness and the profits of the proposed viewpoint. Rajendran and
Devarajan [17] have developed a new approach to design of a high-performance torque control technique.
This study also focused on direct torque control with space vector modulation of three phase introduction
motor by using FPGA. Ismael et al. [18] have presented a view point to present the Bresenham algorithm
byusing separating every line into number of sections. The outcome shows the maximum number of
segments presented, and points are calculated. In this axis theorem is utilized to identify the image
intersection and affine refinement mechanism to extract exact global consistency. The final outcomes of the
method give the promising results in obtaining the better results of mosaicing. From the review analysis it is
found that very rare researches were considered the alignment issues, image quality and resolution issues,
manual registration and also very less real time implementation is performed for mosaicing. The following
section gives the issues which are considered in this research.
Recently, most of the techniques were offered to build the mosaiced image under homography
variations in the image. If the overlapping among the two images is high mosaicing may give good results
(through Levenberg Marquardt approach [19]) but yields sensitive against local minima and causes higher
computational complexity. Similarly, if the overlapping among the two images is low, the hierarchical
matching can bring significant results to avoid local minima. For the images with improper alignment with
cross correlation approach and faces issues with image brightness variation. Also, very rare researches were
observed for image mosaicing with hardware implementation to cover the image overlapping. The mosaiced
images need to be aligned properly with proper mosaicing algorithms. This paper considers the correlation
problem or alignment problem, image quality and resolution issues during image mosaicing.
Also, the manual image registration and real-time implementation of mosaicing algorithm is considered.
In that sense, two different approaches of image mosaicing are introduced which implements Sliding
Window Module and Discrete Cosine Transform separately to perform automatic image registration and
alignment issues respectively.
2. PROPOSED IMAGE MOSAICUNG SYSTEM
This section gives the design of (a) SWM based colour image mosaicing approach for corner
detection to perform automatic image registration (b) DCT based color image mosaicing to tackle image
alignment issues.
2.1. SWM based CIM
This kind of image mosaicing approach considers two color images (1 & 2) and parallelly both the
images will be subjected to SWM technique. Each color images will have red (r), green (g), and blue (b)

 ISSN: 2088-8708
Int J Elec & Comp Eng, Vol. 9, No. 3, June 2019 : 1630 - 1636
1632
components and for these components are subjected with SWM individually. The SWM technique consists of
image preprocessing module and it is used to create its neighboring pixels and forming the 3x3 window.
Once the window is formed convolution method is applied using Gaussian filtering method. The convolution
method includes both the horizontal and vertical matrix components. After that gradient module collects the
gradients from convoluted image and corner detection is performed. Once the corner detection is completed
for the images, the r, g, b components of both the images will be integrated to form the mosaiced image.
The architecture of SWM based CIM is given in Figure 1.
Original Colour
image 1
R G B colour
Component
Sliding window
module-1
(SWM-1)
Convolution
module-1
(CM-1)
Gradient
Module-1
GM-1
Corner
Detection
Original Colour
image 2
R G B colour
Component
Sliding window
module-2
(SWM-2)
Convolution
module-2
(CM-2)
Gradient
Module-2
GM-2
Corner
Detection
Integrate
RGB
components
Image
Mosaic
Figure 1. Novel architecture for the sliding window based image mosaicing
The following Figure 2 shows the internal architecture of the CIM using SWM. The architecture
contains six Main Sliding Window (MSW) and three individual corner detection model to generate
componentsr, g, and b of mosaiced color image. Here, the 8-bit r, g, b components of both the images
(1 and 2) are subjected to MSW module. Further, the outputs of r, g, and b components from MSW module
will be given to corner detection (corner r, corner g and corner b) module as r, g, b components separately.
The corner matching helps to take a small pixel’s region as a window from the detected corner and compare
it for same region witheverycorner features in the other image. The extracted corners from each window are
fed to the Image transformation module. Two sets of corner values in the images have been detected using
corner module. The convoluted derivatives are fed as input to the next module to compute each image corner
values from the two images being mosaiced. The choice of corner for feature detection is stable when corner
is greater than 1. Once the two sets of corner values in the images have been detected, the aim is to match the
corresponding features to align the images.
MSW
r1
MSW
g1
MSW
b1
MSW
r2
MSW
g2
MSW
b2
Corner
r
Corner
g
Corner
b
CLK
rstn
Valid_in
Image1-r
Image1-g
Image1-b
Image2-r
Image2-g
Image2-b
Final-r
Match-r
Final-g
Match-g
Final-b
Match-b
8
8
8
8
8
8
8
8
8
8
8
8
8
8
Figure 2. Internal architecture of CIM using SWM

1633
Sliding window
module
(SWM)
Convolution
module
(CM)
Gradient
module
(GM)
w1 8
w2 8
w3 8
w4 8
w5 8
w6 8
w7 8
w8 8
w9 8
8 g1
8 g2
8 g3
8 g4
8 g5
8 g6
8 g7
8 g8
8 g9
Pixel-out
8
CLK
rstn
Valid_in
Pixel-in
Valid_out
8
Figure 3. Internal Structure of MainSliding window (MSW) module
The internal structure of MSW module is shown in Figure 3, which consists of three modules like
SWM, Convolution Module (CM) and Gradient Module (GM). Once the clock is activated, an asynchronous
reset signal will be high along with valid_signal=1 (high). The 8-bit pixel_in is the input of the color of any
of the rgb components. The 8-bit pixel_in input is coming serially based on the clocks and it will store in the
memory locations. The stored pixels create its neighboring pixels to form a SWM module. Each window
module is having 3x3 matrixes; hence nine different 8bit window modules (W1-W9) are generated.
These nine different window modules are convoluted using Gaussian filtering with horizontal and vertical
matrix. Finally, it generates gradients (g1 to g9). The gradient module collects the all the nine gradient inputs
to sum up and truncate with last 8-bit values to generate final 8-bit pixel value.
2.2. DCT based CIM
The DCT based CIM approach aims to tackle the local and global alignment issues by using phase
correlation technique. The following Figure 4 indicates the block diagram of the proposed DCT based CIM.
This model of CIM also considers two color images (1 and 2) and extracts the RGB components from them.
These RGB components are feed to DCT engine as input. Then DCT engine is performed over two images
where the images will be parted in different frequencies. The DCT engine is used to perform cosine
transformation using memory coefficients. The DCT will perform compressed image output using 2D-DCT
algorithm. Then it will be forwarded to multiplier and divider module. The DCT output composed of
2D-DCT’s which is input to multiplier which generate product of two multipliers. The output of multiplier is
given to divider, which divide the rgb components of image. The main use of multiplier and divider is used to
find the similar overlapped pixels of two images. The DCT approach helps to resolve most of the overlapped
image parts of two images. Later, Then IDCT engine is applied for phase correlation is calculated. From that
image registration is done and then registration values are generated. Image registration values are considered
as a final mosaicing image. The image registrations process helps to check the aligned pixels in corners of
two images. If the aligned pixels are same then only the registration will take place otherwise the pixels are
not aligned. This process takes place till the last pixels of two images.
The above Figure 5, represents the DCT based CIM top module that composed different input 8 bit
components like red (r), green (g) and blue (b) of image 1 and 2, clock (CLK), reset (rst and Start) and
respective outputs (dout_r, dout_g, dout_b. done_r, done_g and done_b). Figure 6 shows internal structures
of multiplier and divider.
The above Figure 7, indicates the Inverse DCT’s internal architecture that composes various blocks
like shift register, adder/substractor, complement check, multiplier block, adder and updation of output in
RAM memory. Here, 8-bit memory coefficient is chosen to store the 1D-DCT constant coefficient value for
multiplication operation. Further on the basis of counter index, the coefficients are allotted to 8-bit memory.
These 8bit input data will be shifted to 8 times by using shift registers. These shifted 8-bit inputs will be
registered in each 8th
block. On the basis of toggling, addition or subtraction is shifted to output values of
8 bit. Further addition or subtraction is subjected for the shifted 8bit output values. The output of MSB bit
will further analyze the addition and subtraction output. Later, multiplication operation is performed where
the memory 1, 2,3,4,5,6,7,8 with adder/subtractor outputs can yield the 8-different products p1, p2...p7.
Finally, all the products will be added to generate final output of adder. The generated adder output is
rounded off to get the output coefficient (1D-DCT). The transpose of memory taken to get the immediate
coefficients that stores the 64 coefficients in 64 location of RAM memory. Later, on the basis of read/write
signals the RAM module is enabled by using the counter. The first DCT generated coefficient will be
appeared in RAM output. At next, the process of 2D-DCT is begins to generates its corresponding output by
repeating the above process.

 ISSN: 2088-8708
1634
Original Image
1(RGB)
RGB Color
Components
DCT
Engine
Original Image
2(RGB)
RGB Color
Components
Multiply
&
Divide
Module
IDCT
Module
Image
Registration
Done
Final_Out
Figure 4. Block model of DCT based CIM
Color-IM using DCT
CLK
rst
8
8
8
Img1-r
Img1-g
Img1-b
8
8
8
Img2-r
Img2-g
Img2-b
Start
1
dout_r
1
dout_g
1
dout_b
1
done_r
1
done_g
1
done_b
Output1
2
Figure 5. Top module of DCT based colour image mosaicing system
Multiplier
1
Multiplier
2
Division
Module
Ready= 1 dout
Remainder
ready
quotient
din 1
din 2
din 1
din 2
12
12
12
12
24
24
16
16
12
[15:4]
[23:8]
[23:8]
Figure 6. Internal structures of multiplier and divider
Shift Reg.
Add/
Subtractor
block
Check the
complement
Multiplier Adder
8- diif Memory
coeff.
Update Output
in RAM
1D- IDCT gain
I/P
1D- IDCT O/P
1-D DCT
2D- IDCT
O/P 8
2
Figure 7. Internal architecture of inverse DCT

1635
3. RESULTS AND DISCUSSIONS
Both the CIM systems using SWM and DCT approaches are designed using Verilog coding.
Further the validation of these designs is performed by considering appropriate tools that supports the design.
Hence, the system execution is performed in Xilinx 14.7 ISE and simulation using Modelsim 6.3f. The
complete design is held over Artix-7 FPGA board of device 7A100T-3 CSG324. Following are the obtained
results after successful execution of the system.
The above Figure 8 indicates the CIM outcomes obtained for pepper image of size 1600x1200 using
SWM and DCT approaches. In this, left image and right image used to create mosaiced image by matching
its features. Further, the comparative analysis of these two approaches SWM and DCT is performed for
parameters like PSNR, MSE, device utilization and execution time. The following table gives the
corresponding comparison values of PSNR, MSE, device utilization and execution time in Tables 1-4
respectively.
Figure 8. CIM for 1600x1200 peppers image
Table 1. PSNR Comparison of SWM
and DCT based CIM
Parameter/Method (Peppers.jpg) PSNR value in
CIM using SWM 31.6844dB
CIM using DCT 36.2882dB
Table 2. MSE Comparison of SWM
and DCT based CIM
Parameter/Method (Peppers.jpg) MSE value
CIM using SWM 43.0884
CIM using DCT 15.2849
Table 3. Device Utilization
Device Utilization Available SWM based CIM DCT based CIM
Number of Slice Registers 126800 116121 66988
Number of Slice LUTs 63400 252660 94612
Number of completely used LUT-FF pairs 97127 115521 64473
Number of bonded IOBs 210 79 70
Number of BUFG/BUFGCTRLs 32 9 2
Table 4. Execution Time Comparison of SWM and DCT based CIM
Approach CIM using SWM CIM using DCT
Execution time 38.42ms 23.02ms
4. CONCLUSION
This paper presented two different image mosaicing methods like a) SWM based color image
mosaicing (CIM) approach for corner detection to perform automatic image registration, (b) DCT based color
image mosaicing to tackle image alignment issues. Both the methods considered the two images as inputs to
form mosaiced image. A comparative analysis is performed among both the methods. By analyzing the tables
of PSNR, MSE, device utilization and execution time of SWM and DCT based CIM, it is found that effective
PSNR value of DCT (36.2882 dB) than SWM (31.6844 dB), least MSE value of DCT (15.2849) than SWM
(43.0884), less device utilization of DCT than SWM and it takes low execution time of DCT (23.02ms) than
SWM (38.42ms). The higher value of PSNR indicates the good quality of mosaiced image. Least MSE value
represents the low error in mosaiced image and utilizes less hardware components. This paper can be
implemented for surveillance system to monitor the abnormal actions. The proposed methods can be
considered in future researches to have further improvement in performance.

 ISSN: 2088-8708
1636
REFERENCES
[1] Szeliski, Richard, and Heung-Yeung Shum., "Creating full view panoramic image mosaics and environment maps",
Proceedings of the 24th annual conference on Computer graphics and interactive techniques. ACM Press/Addison-
Wesley Publishing Co., 1997.
[2] Reinhard, Erik, et al., “High dynamic range imaging: acquisition, display, and image-based lighting,” Morgan
Kaufmann, 2010.
[3] Stern, Adrian, and Bahram Javidi., "Three-dimensional image sensing, visualization, and processing using integral
imaging," Proceedings of the IEEE 94.3: 591-607, 2006.
[4] Bhosle, Udhav, Subhasis Chaudhuri, and Sumantra Dutta Roy., "A fast method for image mosaicing using
geometric hashing," IETE Journal of Research 48.3-4: 317-324, 2002.
[5] Murino, Vittorio, and Andrea Trucco., "Three-dimensional image generation and processing in underwater acoustic
vision," Proceedings of the IEEE 88.12: 1903-1948, 2003.
[6] Ghosh, Debabrata, and Naima Kaabouch., "A survey on image mosaicing techniques," Journal of Visual
Communication and Image Representation 34: 1-11, 2016.
[7] Zitova, Barbara, and Jan Flusser., "Image registration methods: a survey," Image and Vision Computing 21.11:
977-1000, 2003.
[8] Garcia-Fidalgo, E., Ortiz, A., Bonnin-Pascual, F. and Company, J.P., “A mosaicing approach for vessel visual
inspection using a micro-aerial vehicle,” In Intelligent Robots and Systems (IROS), IEEE/RSJ International
Conference on (pp. 104-110), IEEE, September- 2015.
[9] Zagrouba, Ezzeddine, Walid Barhoumi and Slim Amri., “An efficient image-mosaicing method based on
multifeature matching,” Machine Vision and Applications 20: 139-162, 2007.
[10] Abraham, Rintu, and Philomina Simon., "Review on mosaicing techniques in image processing," Advanced
Computing and Communication Technologies (ACCT), 2013 Third International Conference, IEEE, 2013.
[11] Zhu, Jun, and Mingwu Ren., "Image mosaic method based on SIFT features of line segment," Computational and
mathematical methods in medicine, 2014.
[12] Chen, Shuai, et al., "Image mosaic based on SIFT and morphological component analysis,” Image and Signal
Processing, BioMedical Engineering and Informatics (CISP-BMEI), 10th International Congress, IEEE, 2017.
[13] Amaricai, Alexandru, Constantina-Elena Gavriliu, and Oana Boncalo., "An FPGA sliding window-based
architecture harris corner detector," Field Programmable Logic and Applications (FPL), 24th International
Conference. IEEE, 2014.
[14] Elibol, Armagan, et al., "Fast Underwater Image Mosaicing through Submapping," Journal of Intelligent & Robotic
Systems 85.1: 167-187, 2017.
[15] Kekec, Taygun, Alper Yildirim, and Mustafa Unel., "A new approach to real-time mosaicing of aerial images,"
Robotics and Autonomous Systems 62.12: 1755-1767, 2014.
[16] Jaziri, Ibtihel. "A Simplified Speed Control of Induction Motor based on a Low Cost FPGA," International Journal
of Electrical and Computer Engineering (IJECE) Vol 7, no. 4, 1760-1769, 2017.
[17] ]Rajendran, R., and N. Devarajan. "Simulation and implementation of a high performance torque control scheme of
IM utilizing FPGA," International Journal of Electrical and Computer Engineering (IJECE) Vol 2, no. 3,
277-284, 2012.
[18] Ismael, S, O. Tareq, Y.T. Qassim, "Hardware/Software Co-design for a Parallel Three-Dimensional Bresenham’s
Algorithm," International Journal of Electrical and Computer Engineering (IJECE), 9, no.1, Retrieved on 16-Oct,
2018
[19] Szeliski, Richard, "Image mosaicing for tele-reality applications," Applications of Computer Vision, Proceedings of
the Second IEEE Workshop, IEEE, 1994.
BIOGRAPHIES OF AUTHORS
Jayalaxmi H received the M.Tech degree in VLSI and Embedded System from BMSCE, Bangalore,
VTU, India, in 2005. She is currently working in the department of ECE, Acharya Institute of
Technology, Bangalore, India and pursuing PhD in JNTU, Hyderabad. Her current research interests
include image/video processing, VLSI and published papers in IEEE, Elsevier Science Direct, and
ACEEE.
S. Ramachandran, having a Ph. D. in EE/VLSI Design from IIT Madras. He is currently a Professor
in SJB Research Foundation, SJBIT, Bangalore, establishing Hi Tech Research Labs and guiding a
number of Research scholars. He has written a book titled Digital VLSI Systems Design published by
Springer Verlag, Netherlands, which is followed as a text book for BS/MS/Ph. D. in many Universities
of USA, Europe and other parts of the World. Also, a Video Course has been completed at IIT Madras
on Digital VLSI System Design for NPTEL, National telecast on TV.

Performance analysis on color image mosaicing techniques on FPGA

Recommended

More Related Content

What's hot (18)

Similar to Performance analysis on color image mosaicing techniques on FPGA (20)

More from IJECEIAES (20)

Recently uploaded (20)

Performance analysis on color image mosaicing techniques on FPGA