Infrared and Visible Image Fusion using a Deep

Learning Framework
Hui Li Xiao-Jun Wu∗ Josef Kittler
Jiangsu Provincial Engineering Jiangsu Provincial Engineering
Laboratory of Pattern Recognition and Laboratory of Pattern Recognition and CVSSP
Computational Intelligence, Computational Intelligence, University of Surrey,
Jiangnan University, Jiangnan University, GU2 7XH, Guildford, UK
Wuxi, China, 214122 Wuxi, China, 214122
Email: hui li [email protected] Email: xiaojun wu [email protected] Email: [email protected]

Abstract—In recent years, deep learning has become a very great attention. Zong et al.[7] proposed a medical image
active research tool which is used in many image processing fields. fusion method based on SR, in which, the Histogram of
arXiv:1804.06992v4 [cs.CV] 18 Dec 2018

In this paper, we propose an effective image fusion method using Oriented Gradients(HOG) features are used to classify the
a deep learning framework to generate a single image which
contains all the features from infrared and visible images. First, image patches and learn several sub-dictionaries. The l1 -norm
the source images are decomposed into base parts and detail and the max selection strategy are used to reconstruct the
content. Then the base parts are fused by weighted-averaging. fused image. In addition, there are many methods based on
For the detail content, we use a deep learning network to extract combining SR and other tools for image fusion, such as pulse
multi-layer features. Using these features, we use l1 -norm and coupled neural network(PCNN)[8] and shearlet transform[9].
weighted-average strategy to generate several candidates of the
fused detail content. Once we get these candidates, the max In the sparse domain, the joint sparse representation[10] and
selection strategy is used to get the final fused detail content. cosparse representation[11] were also applied in the image
Finally, the fused image will be reconstructed by combining fusion field. In the low-rank category, Li et al.[12] proposed a
the fused base part and the detail content. The experimental low-rank representation(LRR)-based fusion method. They use
results demonstrate that our proposed method achieves state- LRR instead of SR to extract features, then l1 -norm and the
of-the-art performance in both objective assessment and visual
quality. The Code of our fusion method is available at https: max selection strategy are used to reconstruct the fused image.
//github.com/hli1221/imagefusion deeplearning. With the rise of deep learning, deep features of the source
images which are also a kind of saliency features are used to
I. I NTRODUCTION reconstruct the fused image. In [13], Yu Liu et al. proposed
The fusion of infrared and visible imaging is an impor- a fusion method based on convolutional sparse representa-
tant and frequently occuring problem. Recently, many fusion tion(CSR). The CSR is different from deep learning methods,
methods have been proposed to combine the features present but the features extracted by CSR are still deep features.
in infrared and visible images into a single image[1]. These In their method, the authors employ CSR to extract multi-
state-of-the-art methods are widely used in many applications, layer features, and then use these features to generate the
like image pre-processing, target recognition and image clas- fused image. In addition, Yu Liu et al.[14] also proposed
sification. a convolutional neural network(CNN)-based fusion method.
The key problem of image fusion is how to extract salient They use image patches which contain different blur versions
features from the source images and how to combine them to of the input image to train the network and use it to get
generate the fused image. a decision map. Finally, the fused image is obtained by
For decades, many signal processing methods have been using the decision map and the source images. Although the
applied in the image fusion field to extract image fea- deep learning-based methods achieve better performance, these
tures, such as discrete wavelet transform(DWT)[2], contourlet methods still have many drawbacks: 1) The method in [14] is
transform[3], shift-invariant shearlet transform[4] and quater- only suitable for multi-focus image fusion; 2) These methods
nion wavelet transform[5] etc. For the infrared and visible just use the result which is calculated by the last layers and
image fusion task, Bavirisetti et al. [6] proposed a two-scale a lot of useful information which is obtained by the middle
decomposition and saliency detection-based fusion method, layers will be lost. The information loss tends to get worse
where by the mean and median filter are used to extract the when the network is deeper.
base layers and detail layers. Then visual saliency is used to In this paper, we propose a novel and effective fusion
obtain weight maps. Finally, the fused image is obtained by method based on a deep learning framework for infrared and
combining these three parts. visible image fusion. The source images are decomposed into
Besides the above methods, the role of sparse represen- base parts and detail content by the image decomposition
tation(SR) and low-rank representation has also attracted approach in [15]. We use a weighted-averaging strategy to
obtain the fused base part. To extract the detail, first, we use Suppose that there are K preregistered source images, in
deep learning network to compute multi-layer features so as our paper, we choose K = 2, but the fusion strategy is the
to preserve as much information as possible. For the features same for K > 2. The source images will be denoted as Ik ,
at each layer, we use soft-max operator to obtain weight maps k ∈ {1, 2}.
and a candidate fused detail content will be obtained. Applying Compared with other image decomposition methods, like
the same operation at multiple layers, we will get several wavelet decomposition and latent low-rank decomposition, the
candidates for the fused detail content. The final fused detail optimization method[15] is more effective and can save time.
image is generated by the max selection strategy. The final So in our paper, we use this method to decompose the source
fused image is reconstructed by fusing the base part with the images.
detail content. For each source image Ik , the base parts Ikb and detail
This paper is structured as follows. In SectionII, the image content Ikd are obtained separated by [15]. The base parts are
style transfer using deep learning framework will be presented. obtained by solving this optimization problem:
In SectionIII, the proposed deep learning based image fusion
method is introducted in detail. The experimental results are Ikb = arg min ||Ik − Ikb ||2F + λ(||gx ∗ Ikb ||2F + ||gy ∗ Ikb ||2F ) (1)
Ikb
shown in SectionIV. Finally, SectionV draws the paper to
conclusion. where gx = [−1 1] and gy = [−1 1]T are the horizontal
II. I MAGE STYLE TRANSFER USING DEEP LEARNING and vertical gradient operators, respectively. The parameter λ
FRAMEWORK is set to 5 in our paper.
After we get the base parts Ikb , the detail content is obtained
As we all know, deep learning achieves the state-of-the-
by Eq.2,
art performance in many image processing tasks, such as
image classification. In addition, deep learning also can be Ikd = I − Ikb (2)
a useful tool for extracting image fearures which contain
different information at each layer. Different applications of The framework of the proposed fusion method is shown in
deep learning received a lot of attention in the last two years. Fig.1.
Hence, we believe deep learning can also be applied to the
image fusion task.
In CVPR 2016, Gatys et al.[16] proposed an image style
average
transfer method based on CNN. They use VGG-network[17] strategy

to extract deep features at diffierent layers from the “content”

image, “style” image and a generated image, respectively. The decomposition

difference of deep features extracted from the generated image

and source images is mimimised by iteration. The generated F
Deep
learning
image will contain the main object from the “content” image framework
= +
and texture features from the “style” image. Although this
method can obtain good stylized image, its speed is extremly
slow even when using GPU.
Due to these drawbacks, in ECCV 2016, Justin Johnson
et al.[18] proposed a feed-forward network to solve the opti- multi-layers
Recons-
VGG19 fusion
truction
mization problem formulated in [16] in real time. But in this strategy

method, each network is tied to a fixed style. To solve this

problem, in ICCV 2017, Xun Huang et al.[19] used VGG-
network and adaptive instance normalization to construct a
Fig. 1. The framework of proposed method.
new style transfer framework. In this framework, the stylized
image can be of arbitrary style and the method is nearly three
As shown in Fig.1, the source images are denoted as I1
orders of magnitude faster than [16].
and I2 . Firstly, the base part Ikb and the detail content Ikd for
These methods have one thing in common. They all use
each source image are obtained by solving Eq.(1) and Eq.(2),
multi-layer network fearures as a constraint condition. Inspired
where k ∈ {1, 2}. Then the base parts are fused by weighted-
by them, multi-layer deep features are extracted by a VGG-
averaging strategy and the detail content is reconstructed by
network in our fusion method. We use the fixed VGG-19[17]
our deep learning framework. Finally, the fused image F will
which is trained on ImageNet to extract the features. The detail
be reconstructed by adding the fused base part Fb and detail
of our proposed fusion method will be introduced in the next
content Fd .
section.
III. T HE P ROPOSED F USION M ETHOD A. Fusion of base parts
The fusion processing of base parts and detail content is The base parts which are extracted from the source images
introduced in the next subsections. contain the common features and redundant information. In
our paper, we choose the weighted-averaging strategy to fuse
these base parts. The fused base part is calculated by Eq.3,
…

Fb (x, y) = α1 I1b (x, y) + α2 I2b (x, y) (3)

,
soft-max
where (x, y) denotes the corresponding position of the image -norm +
average upsampling
intensity in I1b , I2b and Fb . α1 and α2 indicate the weight values
for pixel in I1b and I2b , respectively. To preserve the common
features and reduce the redundant information, in this paper,
…
we choose α1 = 0.5 and α2 = 0.5.
,
B. The fusion of the detail content
upsampling operator
For the detail content I1d and I2d , we propose a novel fusion
strategy which uses deep learning method(VGG-network) to h ℎ×2
w
extract deep features. This procedure is shown in Fig.2. ×2

Fig. 3. The procedure of the fusion strategy for detailed parts.

We then use the block-based average operator to calculate

VGG19
multi-layers
fusion
Recons-
truction
the final activity level map Cˆki in order to make our fusion
strategy
method robust to misregistration.
Pr Pr i
ˆi β=−r θ=−r Ck (x + β, y + θ)
Ck (x, y) = (6)
(2r + 1)2
where r determines the block size. The fusion method will
be more robust to misregistration if the r is larger, but some
Fig. 2. The procedure of detail content fusion.
detail could be lost. Thus, in our strategy r = 1.
In Fig.2, we use VGG-19 to extract deep features. Then the Once we get the activity level map Cˆki , the initial weight
weight maps are obtained by a multi-layer fusion strategy. maps Wki will be calculated by soft-max operator, as shown
Finally, the fused detail content is reconstructed by these in Eq.7,
weight maps and the detail content.
Cˆi (x, y)
Now, we introduce our multi-layer fusion strategy in detail. Wki (x, y) = PK k (7)
ˆi
Consider detail content Ikd . φi,m
k indicates the feature maps n=1 Cn (x, y)
of k-th detail content extracted by the i-th layer and m is the
where K denotes the number of activity level map, which in
channel number of the i-th layer, m ∈ {1, 2, · · · , M },M =
our paper is set to K = 2. Wki (x, y) indicates the initial weight
64 × 2i−1 ,
map value in the range of [0,1].
φi,m = Φi (Ikd ) (4) As we all know, the pooling operator in VGG-network is a
k
kind of subsampling method. Every time this operator resizes
where each Φi (·) denotes a layer in the VGG-network and the feature maps to 1/s times of the original size where s
i ∈ {1, 2, 3, 4} represents the relu 1 1, relu 2 1, relu 3 1 is the stride of the pooling operator. In the VGG-network,
and relu 4 1, respectively. the stride of the pooling operator is 2. So in different layers,
Let φi,1:M
k (x, y) denote the contents of φi,m
k at the position the size of feature maps is 1/2i−1 times the detail content
(x, y) in the feature maps. As we can see, φi,1:M k (x, y) is size, where i ∈ {1, 2, 3, 4} indicates the layers of relu 1 1,
an M -dimensional vector. The procedure of our strategy is relu 2 1, relu 3 1 and relu 4 1, respectively. After we get
presented in Fig.3. each initial weight map Wki , we use an upsampling operator
As shown in Fig.3, after we get the deep features φi,m k , the
to resize the weight map size to the input detail content size.
activity level map Cˆki will be calculated by l1 -norm and block- As shown in Fig.3, with the upsampling operator, we will
based average operator, where k ∈ {1, 2} and i ∈ {1, 2, 3, 4}. get the final weight map Wˆki , the size of which equals the
Inspired by [12], the l1 -norm of φi,1:M (x, y) can be the input detail content size. The final weight map is calculated
k
activity level measure of the source detail content. Thus, the by Eq.8,
initial activity level map Cki is obtained by
Wˆki (x + p, y + q) = Wki (x, y), (8)
Cki (x, y) = ||φi,1:M
k (x, y)||1 (5) p, q ∈ {0, 1, · · · , (2 i−1
− 1)}
 Now we have four pairs of weight maps Wˆki , k ∈ {1, 2} These layers are relu 1 1, relu 2 1, relu 3 1 and relu 4 1,
and i ∈ {1, 2, 3, 4}. For each pair Wˆki , the initial fused detail respectively
content is obtained by Eq.9, For comparison, we selected several recent and classical
PK fusion methods to perform the same experiment, including:
Fdi (x, y) = n=1 Ŵni (x, y) × Ind (x, y), K = 2. (9) cross bilateral filter fusion method(CBF)[21], the joint-sparse
representation model(JSR)[10], the JSR model with saliency
Finally, the fused detail content Fd is obtained by Eq.10
detection fusion method(JSRSD)[22], weighted least square
in which we choose the maximum value from the four initial
optimization-based method(WLS)[20] and the convolutional
fused detail content for each position.
sparse representation model(ConvSR)[13].
Fd (x, y) = max[Fdi (x, y)|i ∈ {1, 2, 3, 4}] (10) All the fusion algorithms are implemented in MATLAB
R2016a on 3.2 GHz Intel(R) Core(TM) CPU with 12 GB
C. Reconstruction RAM.
Once the fused detail content Fd is obtained, we use B. Subjective Evaluation
the fused base part Fb and the fused detail content Fd to
reconstruct the final fused image, as shown in Eq.11, The fused images which are obtained by the five exist-
ing methods and the proposed method are shown in Fig.5
F (x, y) = Fb (x, y) + Fd (x, y) (11) and Fig.6. Due to the space limit, we evaluate the relative
performance of the fusion methods only on a single pair of
D. Summary of the Proposed Fusion Method images(“street” and“people”).
In this section, we summarize the proposed fusion method
based on deep learning as follows:
1) Image decomposition: The source images are decom-
posed by the image decomposition operation[15] to obtain the
base part Ikb and the detail content Ikd , where k ∈ {1, 2}.
2) Fusion of base parts: We choose the weighted-averaging
(a)infrared image (b)visible image
fusion strategy to fuse base parts, with the weight value for
each base part of 0.5.
3) Fusion of detail content: The fused detail content is
obtained by the multi-layer fusion strategy.
4) Reconstruction: Finally, the fused image is given by
Eq.11.
(c)CBF (d)JSR (e)JSRSD
IV. E XPERIMENTAL RESULTS AND ANALYSIS
The aim of the experiment is to validate the proposed
method using subjective and objective criteria and to carry
out a comparison with existing methods.

A. Experimental Settings (f)WLS (g) ConvSR (h)Proposed

In our experiment, the source infrared and visible images
Fig. 5. Results on “street” images. (a) Infrared image; (b) Visible image; (c)
were collected from [20] and [25]. There are 21 pairs of our CBF; (d) JSR; (e) JSRSD; (f) WLS. (g) ConvSR; (h) The proposed method.
source images and they are available at [26]. A sample of these
images is shown in Fig.4. As we can see from Fig.5(c-h), the fused image obtained by
the proposed method preserves more detail information in the
red window and contains less artificial noise. In Fig.6(c-h), the
fused image obtained by the proposed method also contains
less noise in the red box.
In summary, the fused images which are obtained by CBF
have more artificial noise and the salient features are not clear.
The fused images obtained by JSR, JSRSD and WLS, in
addition, contain artificial structures around the salient features
and the image detail is blurred. In contrast, the fused images
Fig. 4. Four pairs of source images. The top row contains infrared images,
obtained by ConvSR and the proposed fusion method contain
and the second row contains visible images. more salient features and preserve more detail information.
Compared with the four existing fusion methods, the fused
In multi-layer fusion strategy, we choose few layers from images obtained by the proposed method look more natural.
a pre-trained VGG-19 network[17] to extract deep features. As there is no visible difference between ConvSR and the
 In Table I, the best values for F M Idct , F M Iw , SSIMa
and Nabf are indicated in bold. As we can see, the proposed
method has all the best average values for these metrics. These
values indicate that the fused images obtained by the proposed
(a)infrared image (b)visible image
method are more natural and contain less artificial noise. From
the objective evaluation, our fusion method has better fusion
performance than the existing methods.
Specifically, we show in Table II all values of Nabf for the
21 pairs produced by the respective methods. The graph plot
of Nabf for all fused images is shown in Fig.7.
(c)CBF (d)JSR (e)JSRSD
TABLE II
T HE Nabf VALUES FOR 21 FUSED IMAGES WHICH OBTAINED BY FUSION
METHODS .

Methods CBF[21] WLS[20] JSR[10] JSRSD[22] ConvSR[13] Proposed

image1 0.23167 0.14494 0.34153 0.34153 0.0149 0.00013
image2 0.48700 0.16997 0.19749 0.19889 0.0220 0.00376
(f)WLS (g) ConvSR (h)Proposed image3 0.54477 0.21469 0.38627 0.38627 0.0207 0.00622
image4 0.45288 0.22866 0.42353 0.42353 0.0238 0.00132
image5 0.43257 0.19188 0.49804 0.49804 0.0099 0.00020
Fig. 6. Results on “people” images. (a) Infrared image; (b) Visible image; (c) image6 0.23932 0.22382 0.36619 0.36509 0.0230 0.00099
CBF; (d) JSR; (e) JSRSD; (f) WLS. (g) ConvSR; (h) The proposed method. image7 0.41779 0.15368 0.52301 0.52220 0.0151 0.00188
image8 0.15233 0.23343 0.21640 0.21536 0.0340 0.00037
image9 0.11741 0.17177 0.30983 0.30761 0.0237 0.00029
image10 0.20090 0.22419 0.34329 0.34271 0.0201 0.00048
image11 0.47632 0.20588 0.33225 0.32941 0.0102 0.00109
proposed fusion method in terms of human sensitivity, we image12 0.25544 0.22335 0.32488 0.32502 0.0154 0.00058
use several objective quality metrics to evaluate the fusion image13 0.36066 0.19607 0.28106 0.28220 0.0189 0.00035
image14 0.18971 0.20332 0.40615 0.40261 0.0204 0.00082
performance in the next section. image15 0.21509 0.20378 0.35106 0.35013 0.0221 0.00060
image16 0.52783 0.30672 0.26907 0.26888 0.0194 0.00090
image17 0.52887 0.31160 0.33544 0.33720 0.0156 0.00122
C. Objective Evaluation image18 0.26649 0.25937 0.55761 0.55732 0.0150 0.00023
image19 0.12582 0.16205 0.27327 0.27302 0.0138 0.00002
For the purpose of quantitative comparison between the image20 0.25892 0.18401 0.16588 0.16541 0.0257 0.00203
image21 0.18091 0.25074 0.38734 0.38546 0.0275 0.00171
proposed method and existing fusion methods, four quality
metrics are utilized. These are: F M Idct and F M Iw [23] which
calculate mutual information (FMI) for the discrete cosine and
0.60000
wavelet features, respectively; Nabf [24] which denotes the rate
of noise or artifacts added to the fused image by the fusion 0.50000

process; and modified structural similarity(SSIMa ). 0.40000

The values of Nabf

In our paper, the SSIMa is calculated by Eq.12, CBF

WLS
0.30000
JSR
SSIMa (F ) = (SSIM (F, I1 ) + SSIM (F, I2 )) × 0.5 (12) JSRSD
0.20000 ConvSR
Proposed
where SSIM (·) represents the structural similarity operation,
0.10000
F is the fused image, and I1 , I2 are the source images. The
value of SSIMa assesses the ability to preserve structural 0.00000
0 5 10 15 20

information. Image1 - 21

The performance improves with the increasing numerical

index of F M Idct , F M Iw and SSIMa . On the contrary, Fig. 7. Plotting Nabf for all fused images obtained by the fusion methods
the fusion performance is better when the value of Nabf is experimentally compared.
small, which means the fused images contain less artificial
From Table II and Fig.7, the values of Nabf produced by
information and noise.
our method are nearly two orders of magnitude batter than
The average values of F M Idct , F M Iw , SSIMa and Nabf
CBF, JSR and JSRSD. Even compared with ConvSR, the Nabf
obtained by teh existing methods and the proposed method for
values of proposed method are extremely small. This indicates
the 21 fused images are shown in Table I.
that the fused images obtained by the proposed method contain
less artificial information and noise.
TABLE I
T HE AVERAGE VALUES OF F M Idct , F M Iw , SSIMa AND Nabf FOR 21 V. C ONCLUSION
FUSED IMAGES .
In this paper, we present a simple and effective fusion
Methods CBF[21] WLS[20] JSR[10] JSRSD[22] ConvSR[13] Proposed
F M Idct [23] 0.26309 0.33103 0.14236 0.14253 0.34640 0.40463 method based on a deep learning framework(VGG-network)
F M Iw [23] 0.32350 0.37662 0.18506 0.18498 0.34640 0.41684
SSIMa 0.59957 0.72360 0.54073 0.54127 0.75335 0.77799 for an infrared and visible image fusion task. Firstly, the source
Nabf [24] 0.31727 0.21257 0.34712 0.34657 0.0196 0.00120 images are decomposed into base parts and detail content.
The former contains low frequency information and the latter [17] Simonyan K, Zisserman A. Very deep convolutional networks for large-
contains texture information. These base parts are fused by the scale image recognition[J]. arXiv preprint arXiv:1409.1556, 2014.
[18] Johnson J, Alahi A, Fei-Fei L. Perceptual losses for real-time style trans-
weight-averaging strategy. For the detail content, we proposed fer and super-resolution[C]//European Conference on Computer Vision.
a novel multi-layer fusion strategy based on a pre-trained Springer, Cham, 2016: 694-711.
VGG-19 network. The deep features of the detail content are [19] Xun Huang, Serge Belongie. Arbitrary Style Transfer in Real-Time
With Adaptive Instance Normalization[C]//2017 The IEEE International
obtained by this fixed VGG-19 network. The l1 -norm and Conference on Computer Vision (ICCV). IEEE, 2017:1501-1510.
block-averaging operator are used to get the initial weight [20] Ma J, Zhou Z, Wang B, et al. Infrared and visible image fusion based on
maps. The final weight maps are obtained by the soft-max visual saliency map and weighted least square optimization[J]. Infrared
Physics & Technology, 2017, 82: 8-17.
operator. The initial fused detail content is generated for each [21] Kumar B K S. Image fusion based on pixel significance using cross
pair of weight maps and the input detail content. The fused bilateral filter[J]. Signal, image and video processing, 2015, 9(5): 1193-
detail content is reconstructed by the max selection operator 1204.
[22] Liu C H, Qi Y, Ding W R. Infrared and visible image fusion method
applied to these initial fused detail content. Finally, the fused based on saliency detection in sparse domain[J]. Infrared Physics &
image is reconstructed by adding the fused base part and the Technology, 2017, 83: 94-102.
fused detail content. We use both subjective and objective [23] Haghighat M, Razian M A. Fast-FMI: non-reference image fusion
metric[C]//Application of Information and Communication Technologies
methods to evaluate the proposed method. The experimental (AICT), 2014 IEEE 8th International Conference on. IEEE, 2014: 1-3.
results show that the proposed method exhibits state-of-the-art [24] Kumar B K S. Multifocus and multispectral image fusion based on pixel
fusion performance. significance using discrete cosine harmonic wavelet transform[J]. Signal,
Image and Video Processing, 2013, 7(6): 1125-1143.
We believe our fusion method and the novel multi-layer [25] Toet A. TNO Image fusion dataset[J]. Figshare. data, 2014. https:
fusion strategy can be applied to other image fusion tasks, //figshare.com/articles/TN Image Fusion Dataset/1008029.
such as medical image fusion, multi-exposure image fusion [26] Li H. CODE: Infrared and Visible Image Fusion using a Deep Learning
Framework. https://github.com/exceptionLi/imagefusion deeplearning/
and multi-focus image fusion. tree/master/IV images

R EFERENCES
[1] Li S, Kang X, Fang L, et al. Pixel-level image fusion: A survey of the
state of the art[J]. Information Fusion, 2017, 33: 100-112.
[2] Ben Hamza A, He Y, Krim H, et al. A multiscale approach to pixel-level
image fusion[J]. Integrated Computer-Aided Engineering, 2005, 12(2):
135-146.
[3] Yang S, Wang M, Jiao L, et al. Image fusion based on a new contourlet
packet[J]. Information Fusion, 2010, 11(2): 78-84.
[4] Wang L, Li B, Tian L F. EGGDD: An explicit dependency model for
multi-modal medical image fusion in shift-invariant shearlet transform
domain[J]. Information Fusion, 2014, 19: 29-37.
[5] Pang H, Zhu M, Guo L. Multifocus color image fusion using quaternion
wavelet transform[C]//Image and Signal Processing (CISP), 2012 5th
International Congress on. IEEE, 2012: 543-546.
[6] Bavirisetti D P, Dhuli R. Two-scale image fusion of visible and infrared
images using saliency detection[J]. Infrared Physics & Technology, 2016,
76: 52-64.
[7] Zong J, Qiu T. Medical image fusion based on sparse representation of
classified image patches[J]. Biomedical Signal Processing and Control,
2017, 34: 195-205.
[8] Lu X, Zhang B, Zhao Y, et al. The infrared and visible image fusion
algorithm based on target separation and sparse representation[J]. Infrared
Physics & Technology, 2014, 67: 397-407.
[9] Yin M, Duan P, Liu W, et al. A novel infrared and visible image fusion
algorithm based on shift-invariant dual-tree complex shearlet transform
and sparse representation[J]. Neurocomputing, 2017, 226: 182-191.
[10] Zhang Q, Fu Y, Li H, et al. Dictionary learning method for joint sparse
representation-based image fusion[J]. Optical Engineering, 2013, 52(5):
057006.
[11] Gao R, Vorobyov S A, Zhao H. Image fusion with cosparse analysis
operator[J]. IEEE Signal Processing Letters, 2017, 24(7): 943-947.
[12] Li H, Wu X J. Multi-focus Image Fusion Using Dictionary Learning
and Low-Rank Representation[C]//International Conference on Image and
Graphics. Springer, Cham, 2017: 675-686.
[13] Liu Y, Chen X, Ward R K, et al. Image fusion with convolutional
sparse representation[J]. IEEE signal processing letters, 2016, 23(12):
1882-1886.
[14] Liu Y, Chen X, Peng H, et al. Multi-focus image fusion with a deep
convolutional neural network[J]. Information Fusion, 2017, 36: 191-207.
[15] Li S, Kang X, Hu J. Image fusion with guided filtering[J]. IEEE
Transactions on Image Processing, 2013, 22(7): 2864-2875.
[16] Gatys L A, Ecker A S, Bethge M. Image style transfer using convo-
lutional neural networks[C]//Computer Vision and Pattern Recognition
(CVPR), 2016 IEEE Conference on. IEEE, 2016: 2414-2423.