Muti-focus image fusion using wavelet based contourlet transform and region

Ding Li
School of Electronics and Information Engineering
Xian jiaotong University, Xian, China
1
School of Information and Control Engineering
Xian Architecture and Technology University,
Xian , China
2
[email protected]


Han Chongzhao
School of Electronics and Information
Engineering
Xian jiaotong University, Xian, China
[email protected]

AbstractUtilizing the contourlets advantages
of multiscale, localization, directionality and
anisotropy, a New method for multi-focus image
fusion using wavelet based contourlet transform
and region was proposed. In the contourlet
transform, the Laplacian pyramid transform was
replaced by the wavelet transform to obtain non-
redundant wavelet-based contourlet transform. The
source image is decomposed into low-frequency
image and high-frequency images by wavelet based
contourlet.. The fusion rule of Regional variance is
adopted in low-frequency image, and the fusion
rule of local energy are adopted in high-frequents
subbands. Finally the fused image is obtained
through inverse transform. Experimental results
show that the proposed method has better
subjective visual effect. Compared to traditional
gradient pyramid algorithm and wavelet fusion
algorithm, the proposed algorithm in paper has
smallest mean square error.
Keywordswavelet based contourlet; multi-focus;
image fusion
1. INTRODUCTION
In current multi-focus image fusion method are
based on pyramid fusion
[1]
and wavelet fusion
[2]
. The
pyramid fusion is unsteadily because there exist
correlations between each level. The wavelet transform
approach to image fusion overcome the pyramid
fusions shortcoming. Because the data after
orthogonal wavelet decomposition fall into the
orthogonal subspace, and reduce the correlation of the
detail information each level. But the 2-D wavelets
constructed from tensor products of 1-D wavelets have
square-shape support domain, so it is limit to three
directions such as horizon, verticality and diagonal.
The Wavelets transformation can not capture the image
geometry structure. The contourlet transform and
wavelet based contourlet resolve the problem, which
have advantages of multiscale localization,
directionality and anisotropy.
The fusion rules are divided into two types. One is
based on pixel, another is based on region. The paper
propose a New method for multi-focus image fusion
using wavelet based contourlet transform and region.
The different rule of fusion was adopt in low-frequent
subband and high-frequent subbands respectively. The
EMS is the subjective standard. The experimental
result show the algorithm presented by paper has
smallest EMS.
2. WAVELET BASED CONTOURLET
2.1 Countorulet Transform
As the resolution becomes finer, the wavelet use
many fine dots to capture the contour. On the other
hand the Contourlet transform representation image
used basis elements having a variety of elongate
shapes with different aspect ratios, which proposed by
Do M N and Vetterli M in 2002
[3-4]
. The contourlet
transform have tow stages. The first stage employ
Laplacian pyramid decomposition. The second stage
the bandpass images from the LP are fed into a
directional filter bank (DFB). The DFB leads to 2
l
subbands with wedge-shape frequency partitioning via
an l-level binary tree decomposition as shown in
Figure 1. An l-level tree-structured DFB equivalent as
a 2
l
parallel channel filter bank with equivalent filters
and overall sampling matrices as shown in Fig. 2.
Overall sampling matrices are shown as follwing
diagonal forms
2009 International Conference on Information Management and Engineering
978-0-7695-3595-1/09 $25.00 2009 IEEE
DOI 10.1109/ICIME.2009.31
90
2009 International Conference on Information Management and Engineering
978-0-7695-3595-1/09 $25.00 2009 IEEE
DOI 10.1109/ICIME.2009.31
90
Fig. 1 frequency partitioning wher l = 3 and there are
2
3
= 8 real wedge-shaped frequency bands
Figure.2 the multichannel view of an l-level tree-
structured directional filter bank.

D
k
is synthesis filters. The overall sampling matrices
are shown as (1).
1
( )
1 1
(2 , 2) 0 2
(2, 2 ) 2 2
l
l
k
l l
diag for k
S
diag for k

s <

s <

1 l
l

)
(1)
2.2 Wavelet based Contourlet Transform
Due to the redundancy of the Laplacian pyramid, the
contourlet transform has a redundancy factor of 4/3
and hence, Eslami R and Radha H proposed
nonredundant image transforms based on the
contourlet transformwavelet based contourlet
transform(WBCT)
[5]
.The WBCT also have two stages.
The first stage provides subband decomposition using
wavelet transform but not Laplacian pyramid. At each
level (j) we obtain the three highpass bands
corresponding to the LH, HL, and HH bands. The
second state we apply DFB with the same number of
directions to each band in a given level (j). Starting
from the desired maximum number of directions N
d
=
2
l
on the finest level of the wavelet transform J, we
decrease the number of directions at every the dyadic
scale when we proceed through the coarser levels (j<J).
Fig.3 illustrates a schematic plot of the WBCT using 3
wavelet levels and L = 3 directional levels.
Fig.3 A schematic plot of the WBCT using 3 dyadic
wavelet levels and 8 directions at the finest level. the
directional decomposition is overlaid the wavelet
subbands.
3 MUTI-FOCUS IMAGE FUSION USING
WAVELET BASED CONTOURLET TRANSFORM
AND REGION
3.1 The algorithm based countourlet transform and
region
The fusion rules are divided into two types in
process of fusion image. One is based on single pixel
such as max absolution rule, other one is based on
region such as region variance. Because there exist
correlativity between pixel, so it represent image
considering region more better than pixel. We propose
the new algorithm of wavelet based contourlet
transform in conjunction to region in image fusion.
The discrete step as follow:
Step 1: The source image A and B employ wavelet
based contourlet transform and get corresponding
coefficient sets. Generally three level or four level
decomposition is used in fusion. Provided source
image f
A
, f
B
use J levels wavelet decomposition. L(j) is
the number of directions at each level. The
decomposition result are show as equation (2),(3) and
(4)
( ) ( ) ( ) ( )
1 2
( , ) ( , , , ,
A A A A
A J
f x y b b b a " )
J
)
J
,
,
(2)
( ) ( ) ( ) ( )
1 2
( , ) ( , , , ,
B B B B
B J
f x y b b b a " (3)
( ) ( ) ( ) ( )
, ,1 , ,2, , , ( )
( ) ( ) ( )
, ,1 , ,2, , , ( )
( ) ( ) ( )
, ,1 , ,2, , , ( )
{ }
{ }
{ }
,
,
, }
{
X X X X
j j LH j LH j LH l j
X X X
j HL j HL j HL l j
X X X
j HH j HH j HH l j
b d d d
d d d
d d d
= "
"
"
(4)
, X A B =
a
J
is lowpass frequent band, b
j
is directional subband
sets in j level. d
j,LH,k
is k directional high-frequent
subband at LH highpass band of j level after
transformed by wavelet. d
j,HL,k
is k directional high-
frequent subband at HL highpass band of j level. d
j,HH,k
91 91
is k directional high-frequent subband at HH highpass
band of j level.
Step 2: image fusion.
The different fusion rule are adopted in low-frequent
subband and high-frequent subbands respectively.
Step 3: reconstruction.
We get the reconstruction fusion image from the
fusion coefficient get by second step.
3.2 Fusion rules.
3.2.1 High-frequent subbands fusion rule.
The bigger absolute coefficient value in high-
frequent subbands correspond to region having acute
change. That is to say the energy of high-frequent
coefficient of clear image are bigger than fuzzy image.
We select the region enery as fusion rule. region enery
defined by equation (5).
2
( ) ( )
, , , ,
( , ) ( , )
( , ) ( , ) , , ,
X X
j t k j t k
x y mn
E mn d x y t LH HL HH
eO
= =

(5)
d
j,t,k
(x,y) is k directional high-frequent subband
coefficient at t highpass band of j level high-frequent
subbands; (m,n) is rectangle region; Define the
fusion rule as (6):
( ) ( ) ( )
, , , , , ,
( )
, ,
( ) ( ) ( )
, , , , , ,
( , ), ( , ) ( , )
( , )
( , ), ( , ) ( , )
A A B
j t k j t k j t k
F
j t k
B A B
j t k j t k j t k
d m n E m n E m n
d m n
d m n E m n E m n
>

=
`
<

)

(6)
3.2.2 Low-frequent subband fusion rule
Low-frequent subband a
J
is approximation subgragh
of source image, so selecting coefficient of clear image
is best selection. We regard the region variance as the
rule of clear. Partition the low-frequent subband a
J
into 0h1 subdomain a
j
(m,n), then calculate the region
variance.
( )
( ) ( ) 2
( , ) ( , )
( , ) (( ( , )( , ) ( , ))
X
X X
J
J
x y m n
C x y a m m x y a m n
eO
=

(7)
In equation (7)
( )
( , )
X
J a m n
is the mean value of
coefficient in sub region(m,n). the bigger is the
variance ,the bigger is the local contrast and the higher
is the defination. The fusion process is described by
(8).
( ) ( ) ( )
( ) ( ) ( )
( , ), ( , ) ( , )
( , )
( , ), ( , ) ( , )
A A B
J F
J
B A B
J
a m n C m n C m n
a m n
a m n C m n C m n
>

=
`
<

)
(8)
4. EXPERIMENT RESULT
the experiment use source multi-focus Pepsi images.
Three levels decomposition were used and each level
having 8, 4 and 4 directions. The experimental result is
compared to the traditional gradient pyramid algorithm,
wavelet fusion algorithm based on max value and
Contourlet algorithm based region .
(a)Left focus image (b) Right focus image (c)Pyramid algorithm
(e) Contourlet algorithm
based max value
(f)The algorithm proposed by
paper
(d) Wave fusion algorithm
Fig.4 Experimental Result
92 92
From the Fig.4 we can see that the fusion result
using gradient pyramid lost many detail and visual
effect is the best worse. There appear fuzzy edge using
wavelet fusion algorithm. Better improvement is get by
contourlet based on region value. The algorithm
presented by paper have good subjective visual effect.
The background and front are all clear.
For the sake of comparing quantitatively the paper
employ the mean square error (EMS) as measure
standard. The define is show as (9)
1 2
1/ 2
2
1 1
1 2
1
(( ( , ) ( , ))
N N
i j
MS E R i j F
N N
= =

i j

(9)
F(i,j) is the fusion result, R( i, j) is the clear region
from two source image. The samller EMS value
indicate the result is closer to source image. The table
1 show the each EMS value compared to S1 ,S2 region
using four kind of algorithm. The algorithm in paper
get the smallest EMS value.
Table 1 EMS value of different algorithm
EMS
Pyramid wave fusion contourlet the paper
Algorithm algorithm algorithm algorithm
S1 0.01923 0.016499 0.013179 0.013115
S2 0.03202 0029155 0.005674 0.003991
5. CONCLUSION
Wavelet based contourlet transform have stonger
direction and sparse expression than wavelet transform
and contourlet transform, and capture the structure
character and edge information more better.
Considering the local correlation the paper propose the
algorithm of wavelet based contourlet transform and
region on fusion of multi-focus images. At the low-
frequent band and high-frequent bands different fusion
rule are adopted. The experimental result show that the
algorithm in paper are effective.
REFERENCES
[1] Burt P J,Adelson EH. The laplacian pyramid as a
compact image code[J]. IEEE Transactions on
Communications, 1983,31(4):532-540.
[2] Hill P,Canagarajah N,Bull D. Image fusion using
complex wavelets[A]. British Machine Vision Conference[C].
Cardiff,
2002.487-496.
[3] Do M N, Vetterli M. The contourlet transform: an ef-
ficient directional multiresolution image representation
[J]. IEEE Trans Image Processing, 2005, 14(12):
2091-2106.
[4] Do M N, VetterliM. Contourlets: A directional
multiresolution image representation [A]. Proc of IEEE
International Conference on Image Processing [C]. Rochester,
NY, 2002.357-360.
[5] Eslami R,Radha H. Wavelet based contourlet transform
and it application to image coding[A].IEEE International
Conference on Image Processing[C]. Singapore, 2004.3189-
3192.
93 93