Single Image Super Resolution Using Interpolation and Discrete Wavelet Transform
Single Image Super Resolution Using Interpolation and Discrete Wavelet Transform
ABSTRACT
An interpolation-based method, such as bilinear, (LR) images or from a single image. The SR image
bicubic, or nearest neighbor interpolation, is regarded reconstruction is useful in many practical cases where
as a simple way to increase the spatial resolution for multiple frames of the same scene can be obtained;
the LR image. It uses the interpolation kernel to including medical imaging, satellite imaging, and
predict the missing pixel values, which fails to video applications [17].The basic premise for
approximate the underlying image structure and leads increasing the spatial resolution in SR techniques is
to some blurred edges. In this work a super resolution the availability of multiple LR images captured from
technique based on Sparse characteristics of wavelet the same scene. The set of source low resolution (LR)
transform. Hence, we proposed a wavelet based images captures only a finite amount of information
super-resolution technique, which will be of the from a scene; the goal of SR is to extract the
category of interpolative methods, using sparse independent information from each image in that set
property of wavelets. It is based on sparse and combine the information into a single high
representation property of the wavelets. Simulation resolution (HR) image. The requirement is of SR is
results prove that the proposed wavelet based that each LR image must contain some information
interpolation method outperforms all other existing that is unique to that image [3]. The super resolution
methods for single image super resolution. The method is to take more samples of the scene so as to
proposed method has 7.7 dB improvement in PSNR get some extra information which can be used, while
compared with Adaptive sparse representation and merging the samples to get a high resolution image.
self-learning ASR-SL [1] for test image Leaves, 12.92 These samples can be acquired by sub-pixel shifts, by
dB improvement for test image Mountain Lion & 7.15 changing scene , by changing the amount of blur [14]
dB improvement for test image Hat compared with .HR means that pixel density within an image is high,
ASR-SL [1]. Similarly, 12% improvement in SSIM and therefore an HR image can offer more details that
for test image Leaves compared with [1], 29% are important in many applications, The major
improvement in SSIM for test image Mountain Lion advantage of the super resolution approach is that it
compared with [1] & 17% improvement in SSIM for may cost less and the existing LR imaging systems
test image Hat compared with [1]. can be still utilized. Synthetic zooming of region of
interest (ROI) is another important application in
Keyword: Super Resolution, Image Reconstruction, surveillance, forensic, scientific, medical, and satellite
Single Image Resolution Techniques, Resolution imaging [13]. This application is most suitable for
Enhancement, Wavelet transform, Interpolation. magnifying objects in the scene such as the face of a
criminal or the license plate of a car [16].
1. INTRODUCTION
Image super resolution is an image processing
algorithms that produce high quality, high resolution
(HR) images from a set of low quality, low resolution
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 241
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
Although wide variety of super-resolution literature is
available, it is still an open topic to investigate.
Following subsections describe some of the existing
basic image super-resolution schemes.
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 242
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
by orthogonal multiscale grouping with weighted resolution image without any external training set.
Haar lifting to points grouped by association fields. They proposed a framework for both magnification
To preserve the sharpness of edges and textures SR and deblurring using only the original low-resolution
image is synthesised by an adaptive directional image and its blurred version. In This method, each
interpolation using the extracted geometric pixel is predicted by its neighbors through the
information. This method showed improvements over Gaussian process regression.
existing geometrically driven interpolation techniques
on a subjective scale, and in many cases with an They showed that, by using a proper covariance
improvement in psycho visual color difference. function, the Gaussian process regression can perform
soft clustering of pixels based on their local
2.6 Remotely Sensed image by Hope field structures. This algorithm can extract adequate
Neural Network information contained in a single low-resolution
J Tatem Andrew et al. [12] used their idea of super- image to generate a high-resolution image with sharp
resolution for target identification in remotely sensed edges. Compared to other edge-directed and example-
images. Fuzzy classification improves the accuracy of based super-resolution algorithms this algorithm is
land cover target identification make robust and better superior in quality and performance.
for spatial representation of land cover. The Hopfield
neural network converges to a minimum of an energy 2.9 Learning-based SR with a combining of
function, defined as a goal and several constraints. both global and local constraints
The energy minimum represents a best guess map of K. Guoet al. [15] proposed a statistical learning
the spatial distribution of class components in each method for SR with both global and local constraints.
pixel. More specifically, they introduced a mixture model
into maximum a posteriori (MAP) estimation, which
They used two goal functions to make the output of a combines a global parametric constraint with a patch-
neuron similar to that of its neighboring neurons. The based local non-parametric constraint.
first goal function aims to increase the output of
center neuron to 1. The second goal function aims to The global parametric constraint guarantees the super-
decrease the output of the center neuron to 0. They resolved global image to agree with the sparse
showed that, by using a Hopfield neural network, property of natural images, and the local non-
more accurate measures of land cover targets can be parametric constraint is used to infer the residues
obtained compared with those determined using the between the image derived from the global constraint
proportion images alone. and the ground truth high-resolution (HR) image.
2.7 Neural Network based Optimal Recovery They compared it with traditional patch-based
Theory learning methods without the global constraint, and
Yizhen Huang and YangjingLong [13] proposed a showed optimalthatrecoverybased
this method can not only preserve
neural-network Superglobal
Resolution al
Motivated by the idea that back propagation neural image structure, but also restore the local details more
network are capable of learning complex nonlinear effectively.
function they proposed a neural network approach
that produces better results in high-frequency regions. 2.10 Interpolation based SR using Multi surface
They integrated an optimal recovery based approach Fitting
with in a neural network framework and, if so, two Fei Zhou et al. [16] proposed a interpolation-based
different branches of algorithms complement each method of image super-resolution reconstruction.
other to offer a better algorithm. Using this algorithm They used the idea of multi surface fitting to take
in a two-pass way generates visual results that are advantage of spatial structure information. Each site
very similar regardless of the initial interpolation step, of low-resolution pixels is fitted with one surface, and
and more times of iteration only waste the computing the final estimation is made by fusing the
resource but yield negligible performance gain. multisampling values on these surfaces in the
maximum a posteriori fashion. Figure 2 shows the
2.8 Gaussian Process Regression flow chart of interpolation based SR using Multi
Wan-Chi Siu et-al, [14] addressed the problem of surface Fitting.
producing a high-resolution image from a single low-
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 243
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
2.12 SR by Complex Wavelet Transform
Gholamreza Anbarjafari and Hasan Demirel [18]
proposed a technique to enhance to resolution of
satellite images based on interpolation of high-
frequency subband images obtained by dual-tree
complex wavelet transform (DT-CWT). This method
uses DT-CWT to decompose an input low-resolution
satellite image into different subband images and
interpolates input images followed by combining all
these images to generate high-resolution images by
using inverse DT-CWT. Figure 4 shows the diagram
of the method.
3. PROPOSED METHODOLOGY
An adaptive directional image interpolation is
computed by estimating sparse image mixture models
in a wavelet frame. This section describes a fast
orthogonal block matching pursuit implementation.
The detailed process of wavelet based super-
resolution is, first the sub sampled image y(n) for is
Figure 3: Block Diagram of DWT based SR
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 244
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
decomposed with wavelet transform matrix 𝜓 whose For every single angle, application of directional
columns are the vectors of a translation invariant interpolator is performed for the block of wavelet
wavelet frame 𝜓𝑑,𝑚 on a single scale (the finest one). components if the directional regularity factor is less
Then, it is reconstructed with a dual frame type matrix in that block. Such regularization is very much
𝜓, columns of this dual wavelet frames is𝜓𝑑,𝑚 . The helpful, when the eigen values have significantly large
wavelet coefficients are; variation, such that discrimination between regular &
𝑐(𝑑, 𝑚) = {𝑦, 𝜓𝑑,𝑚 } = 𝜓𝑦 (𝑑, 𝑚) non-regular variations in the direction of angle, can be
done. This is obtained by choosing rectangular blocks
The wavelet transform separates a low frequency that are elongated in the direction of angle. Each
image 𝑦𝑙 projected over the low-frequency scaling block in the spatial neighborhood of is chosen to be
filters 𝜓𝑙 (𝑑, 𝑚)& a high-frequency image 𝑦ℎ projected identical in the three directions 1, 2, 3 & so on.
over the finest scale wavelets in three directions Numerical experiments are performed with 20 angles,
with blocks having a width of 2 pixels & a length
𝜓𝑦 (ℎ, 𝑚).
between 6 & 12 pixels depending upon their
𝑦𝑙 = ∑ 𝑐(𝑙, 𝑚) 𝜓𝑙,𝑚 orientation. Each block, thus, includes between 36 &
𝑚∈𝐺 72 wavelet coefficients over the 1, 2, 3 directions.
&𝑦ℎ = ∑3𝑑=1 𝑐(ℎ, 𝑚) 𝜓ℎ,𝑚
An adaptive interpolation estimator by using block
The low frequency image has little aliasing & can, matching is obtained by estimating the mixing
thus, be precisely interpolated with a cubic spline coefficients of a mixture model which minimizes the
interpolator. Detailed process is; errors. The block matching technique performs the
Step 1: Computes a 1-D interpolation in the direction. tasks of maxima finding & then energy update i.e.
We consider all lines of angle that intersect original find the energy of blocks, if this energy is less than
image samples (crosses in Fig. 5) & we compute mid- threshold than eliminate all blocks.
points (circles) between image samples (crosses), with
a cubic spline interpolation. This operation This algorithm stops when there is no sufficiently
oversamples by a factor two either the image rows, or energetic block compared to a precision threshold.
the image columns, or the diagonals of angle. The The minimization is, also interpreted as an optimized
missing coefficients are shown as squares in Fig. 4.1. approximation in orthogonal bundles computed over
adapted blocks of wavelet coefficients. The adaptive
Step 2: Calculation of new samples or dots with a wavelet interpolator is derived by taking IDWT of the
cubic spline interpolation along these oversampled resulting mixing coefficients.
rows, columns or diagonals. This interpolation
introduces little aliasing because of the oversampling PROPOSED ALGORITHM
provided by the previous step. The positions of these
new samples (dots) are chosen so that any missing
coefficient (square) is a mid-point between two dots
on a line of angle.
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 245
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
5.1 Performance Parameters
5. SIMULATION RESULTS
The proposed method is tested on various standard
images Lena; Baboon & Pepper are taken. All the
input low-resolution images interpolated with factor
2. The resolution of input image was 128 × 128.
These input images are interpolated to the size of 512
× 512.For quantitative comparison of results peak
signal to noise ratio (PSNR) & SSIM is used.
FIGURE 8: SIMULATION RESULT FOR TEST
IMAGE “LEAVES”
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 246
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
5.2.2 TEST IMAGE HAT
6. CONCLUSION
The proposed wavelet based interpolation method
outperforms all other existing methods for single
image super resolution. The proposed method has 7.7
FIGURE 12: SIMULATION RESULT FOR TEST dB improvement in PSNR compared with Adaptive
IMAGE “MOUNTAIN LION” sparse representation and self-learning ASR-SL [1]
for test image Leaves, 12.92 dB improvement for test
5.3 SIMULATION RESULT SUMMARY image Mountain Lion & 7.15 dB improvement for test
Table 1 shows the simulation results for various test image Hat compared with ASR-SL [1]. Similarly,
images for PSNR and SSIM values.
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 247
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
12% improvement in SSIM for test image Leaves Process., vol. 21, no. 11, pp. 4544–4556, Nov.
compared with [1], 29% improvement in SSIM for 2012.
test image Mountain Lion compared with [1] & 17% 10. G. Yu, G. Sapiro, and S. Mallat, “Solving inverse
improvement in SSIM for test image Hat compared
problems with piecewise linear estimators: From
with [1]. Gaussian mixture models to structured sparsity,”
IEEE Trans. Image Process., vol. 21, no. 5, pp.
7. REFERENCES 2481 –2499, May 2012.
1. Jianwei Zhao et-al, “Image super-resolution via
adaptive sparse representation and self-learning”, 11. W. Dong, L. Zhang, G. Shi, and X. Wu, “Image
Published in IEEE IET Computer Vision, deblurring and super-resolution by adaptive sparse
Volume: 12, Issue 5, 2018. domain selection and adaptive regularization,”
IEEE Trans. Image Process., vol. 20, no. 7, pp.
2. Wuzhen Shi et-al, “Single Image Super- 1838–1857, Jul. 2011.
Resolution with Dilated Convolution based Multi-
Scale Information Learning Inception Module”, 12. L. Zhang, L. Zhang, X. Mou, and D. Zhang,
IEEE International Conference on Image “FSIM: A feature similarity index for image
Processing (ICIP), 2017. quality assessment,” IEEE Trans. Image Process.,
vol. 20, no. 8, pp. 2378–2386, Aug. 2011.
3. Y. Sun, G. Gu, X. Sui, Y. Liu, and C. Yang,
“Compressive Super-resolution Imaging Based on 13. J. Yang, J. Wright, T. S. Huang, and Y. Ma,
Local and Nonlocal Regularizations,” IEEE “Image super-resolution via sparse
Photonics Journal, Volume 8, No. 1, 2016. representation,” IEEE Trans. Image Process., vol.
19, no. 11, pp. 2861 –2873, Nov. 2010.
4. Y. Sun, G. Gu, X. Sui, Y. Liu, and C. Yang,
“Single image super-resolution using compressive 14. X. Zhang, M. Burger, X. Bresson, and S. Osher,
sensing with a redundant dictionary,” IEEE “Bregmanized nonlocal regularization for
Photonics Journal, volume 7, No. 2, 2015. deconvolution and sparse reconstruction,” SIAM
J. Imag. Sci., vol. 3, no. 3, pp. 253–276, Jul. 2010.
5. Mayank Agrawal, Ratnakar Dash, “Image
Resolution Enhancement Using Lifting Wavelet 15. H. Zhang, J. Yang, Y. Zhang, and T. S. Huang,
and Stationary Wavelet Transform”, IEEE “Non-local kernel regression for image and video
International Conference on Electronic Systems, restoration,” in Proc. Eur. Conf. Comput. Vis.,
Signal Processing and Computing Technologies 2010, pp. 566–579.
(ICESC), 2014. 16. J. Mairal, F. Bach, J. Ponce, G. Sapiro, and A.
6. Mayank Agrawal, Ratnakar Dash, “Image Super- Zisserman, “Non-local sparse models for image
resolution by Interpolating High Frequency Sub- restoration,” in Proc. IEEE Conf. Comput. Vis.
bands of Image using Surface Fitting”, IEEE Pattern Recog., 2009, pp. 2272–2279.
International Conference on Signal Processing and 17. M. Protter, M. Elad, H. Takeda, and P. Milanfar,
Communication (ICSC), 2014. “Generalizing the nonlocal-means to super-
7. Do Kyung Shin, Young Shik Moon, “Super- resolution reconstruction,” IEEE Trans. Image
Resolution Image Reconstruction Using Wavelet Process., vol. 18, no. 1, pp. 36–51, Jan. 2009.
Based Patch and Discrete Wavelet Transform”, 18. P. Sen and S. Darabi, “Compressive image super-
Journal of Signal Processing Systems, Springer, resolution,” in Proc. Conf. 43rd Asilomar Conf.
Volume 81, Issue 1, pp 71-81, 2014. Signals, System Computer, pp. 1235–1242, 2009
8. V. Prasath R. Buvanesvari, N. Thilartham, K. 19. M. Li and T. Nguyen, “Markov random field
Nirosha, “Image Super Resolution Reconstruction model-based edge-directed image interpolation,”
Using Wavelet Transform Method”, International IEEE Trans. Image Process., vol. 17, no. 7, pp.
Journal of Innovative Technology and Exploring 1121 –1128, Jul. 2008.
Engineering, Volume-3, Issue-9, 2014.
20. R. Baraniuk, “Compressive sensing,” IEEE Signal
9. K. Zhang, X. Gao, D. Tao, and X. Li, “Single Process. Mag., vol. 24, no. 4, pp. 118–121, Jul.
image super-resolution with non-local means and 2007.
steering kernel regression,” IEEE Trans. Image
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 248
International Journal of Trend in Scientific Research and Development (IJTSRD) ISSN: 2456-6470
21. H. Takeda, S. Farsiu, and P. Milanfar, “Kernel
regression for image processing and
reconstruction,” IEEE Trans. Image Process., vol.
16, no. 2, pp. 349–366, Feb. 2007.
22. L. Zhang and X. Wu, “An edge-guided image
interpolation algorithm via directional filtering
and data fusion,” IEEE Trans. Image Process., vol.
15, no. 8, pp. 2226–2238, Aug. 2006.
23. A. Buades, B. Coll, and J. M. Morel, “A review of
image denoising algorithms, with a new one,”
Multisc. Model Simul., vol. 4, no. 2, pp. 490–530,
2005.
24. J. M. Bioucas and M. A. T. Figueiredo, “A new
TwIST: Two-step iterative shrinkage/thresholding
algorithms for image restoration,” IEEE Trans.
Image Process., vol. 16, no. 12, pp. 2992–3004,
Dec. 2007.
25. Daubechies, M. Defriese, and C. Demol, “An
iterative thresholding algorithm for linear inverse
problems with a sparsity constraint,” Commun.
Pure Appl. Math. vol. 57, no. 11, pp. 1413–1457,
Nov. 2004.
26. Z. Wang, A. C. Bovik, H. R. Sheikh, and E. P.
Simoncelli, “Image quality assessment: From
error measurement to structural similarity,” IEEE
Trans. Image Process., vol. 13, no. 4, pp. 600–
612, Apr. 2004.
27. S. C. Park, M. K. Park, and M. G. Kang, “Super-
resolution image reconstruction: A technical
overview,” IEEE Signal Process. Mag., vol. 20,
no. 3, pp. 21 –36, May 2003.
28. X. Li and M. T. Orchard, “New edge-directed
interpolation,” IEEE Trans. Image Process., vol.
10, no. 10, pp. 1521 – 1527, Oct. 2001.
29. H. Knutsson and C. F. Westin, “Normalized and
differential convolution,” in Proceeding of IEEE
Conference on Computer Visual Pattern
Recognition, pp.515–523, 1993.
@ IJTSRD | Available Online @ www.ijtsrd.com | Volume – 2 | Issue – 6 | Sep-Oct 2018 Page: 249