A survey on full reference image quality assessment algorithms

IJRET: International Journal of Research in Engineering and Technology eISSN: 2319-1163 | pISSN: 2321-7308
__________________________________________________________________________________________
Volume: 02 Issue: 12 | Dec-2013, Available @ http://www.ijret.org 303
A SURVEY ON FULL REFERENCE IMAGE QUALITY ASSESSMENT
ALGORITHMS
Alphy George1
, S. John Livingston2
1
PG Student, Computer Science and Engineering, Karunya University, Tamilnadu, India, alphygeorge91@gmail.com
2
Asst. Professor, School of Computer Science, Karunya University, Tamilnadu, India, johnlivingston@karunya.edu
Abstract
Image quality measurement is very important for various image processing applications such as recognition, retrieval, classification,
compression, restoration and similar fields. The images may contain different types of distortions like blur, noise, contrast change etc.
So it is essential to rate the image quality appropriately. Traditionally subjective rating methods are used to evaluate the quality of the
image, in which humans rated the image quality based on time requirements. This is a costly process and it needs experts for
evaluating image quality. Nowadays many image quality assessment algorithms are available for finding the quality of images. These
are mainly based on the properties of human visual system. These image quality assessment algorithms are the objective methods for
finding quality. Most of the methods are heuristic and limited to specific application environment. Whereas some methods perform
efficiently and having comparable performance with the subjective ratings. Objective methods are easy for integrating into various
image compression techniques and other image processing applications. Based on the availability of the image, image quality
assessment algorithms are classified into full reference, reduced reference and no reference respectively. Full reference algorithms
normally adopt a two stage structure including local quality measurement and pooling to get the quality value. The input for this two
stage structure includes a reference image and a distorted image. This paper presents a survey on the existing image quality
assessment algorithms based on full reference method, in which a reference image will be available for finding the quality of the
distorted image.
Keywords: Image distortion, Image quality assessment, Human visual system
----------------------------------------------------------------------***------------------------------------------------------------------------
1. INTRODUCTION
Image distortion is often present in almost all images.
Different types of distortion are there. For example noise, blur,
contrast change etc. These distortions can degrade the entire
quality of the image. For example in image compression, if the
captured image contains distortions then it would not match
with the original image that is stored in the database. So
finding the quality of the image in those areas is very
necessary.
Traditionally subjective rating methods are used for measuring
the quality of the image. In this subjective rating, humans
rated the image quality. The images are given to the experts.
Based on the time requirements available, they give ratings to
the image. Subjective results can provide accurate results, but
it is time consuming and also a costly process [11]. This leads
to the development of objective image quality assessment
algorithms (IQA) that will predict the quality of the image
automatically. According to the availability of the original
reference image, the objective methods are classified into full
reference, reduced reference and no reference [6]. In full
reference method a complete distortion free image is available
for the comparison. In case of no reference method blind
prediction of the image quality will be done. In the third
method, only a portion of the reference image is available in
the form of some extracted features. That’s why that method is
known as reduced reference method.
Most of the IQA metrics are based on the full reference
method. Traditional predictors for the image quality are MSE
(Mean Squared Error) and PSNR (Peak Signal-to Noise
Ratio). MSE is a simple index based metric for measuring
quality. But these are the poor predictors of image quality. In
case of correlation with the human visual perception of
quality, MSE shows poor performance [2]. Recently many
methods have been developed in the area of image quality
measurement based on the properties of human visual system
(HVS). For the full reference methods, commonly a two stage
structure is adopted. This two stage structure of full reference
image quality assessment [8] is given in fig. 1.
These image quality measures can be implemented in various
image compression algorithms and other image processing
applications. In this way we can find if there is any distortions
present in the image. This can be concluded if there is any
degradation in the image quality by evaluating the quality
metric value. On the basis of this information, we can remove
that distorted image and replace with another good one. For
evaluating the performance of IQA algorithms, many

__________________________________________________________________________________________
databases are publicly available. Some of the image distortions
that are present in images taken from csiq database are shown
in fig. 2 [10].
Fig -1: Full reference image quality assessment calculation
(c) (d)
Fig -2: Images taken from the CSIQ database (a) original
image (b) blurring (c) JPEG 2000 (d) contrast change
2. FULL REFERENCE IMAGE QUALITY
ASSESSMENT ALGORITHMS
There are many methods available for image quality
evaluation proposed by many researchers. This survey
concentrates on various algorithms for the image quality
assessment metrics. Various performance metrics are used to
compare the image quality algorithms. This is also mentioned
in this paper.
2.1 SSIM (Structural Similarity Index Measure)
This algorithm [6] is based on the concept that human visual
system is highly adapted for extracting structural information
from an image. So from the available image information in the
original and distorted image, a quality measure is constructed.
For getting the structural information, the influence of
illumination has to be separated from the image. For that first
calculate the luminance information of the two image signals x
and y in the form of mean intensity and . The luminance
comparison function l(x,y) is then the comparison of and
. Then remove this mean intensity from the image signal.
The resultant signal is in the form . In the second step
signal contrast is found using the standard deviation and
. The contrast comparison c(x, y) is then the comparison of
and . In the third step the two image signals are divided
by its own standard deviation. This will give the resultant
signals and . On these signals the
structure comparison s(x,y) is performed. In the final step the
three components have to be combined. This will give the
resultant similarity measure S(x,y) i.e.,
. This image quality
assessment approach is applied using a sliding window
algorithm. The window moves across the entire image. For
each window the SSIM metric is measured.
2.2 MAD (Most Apparent Distortion)
Most apparent distortion [3] is mainly based on the property of
HVS when judging the image quality. It consists of two
strategies for quantifying the information contained in images.
Detection based strategy and Appearance based strategy. In
detection based strategy there are two steps that have to be
done. First find the locations in which distortions are visible.
This can be done by converting the original and the distorted
images into luminance images and then converting it into
perceived luminance. Applying a contrast sensitivity function
to this will give the visibility map. In the second step combine
this visibility map with the local errors. This detection based
strategy mainly used for high quality images. For low quality
images detection based strategy can be used. A log-Gabor
decomposition method can be adopted for decomposing the
original image into different subbands. Comparing the
computed local subband statistics would give the result of
appearance based strategy. The final prediction can get by
combining the two strategies.
(a) (b)

__________________________________________________________________________________________
2.3 NPID (Normalized Perceptual Information
Distance)
This image distortion analysis method [9] is mainly based on
the theory of kolmogorov complexity which is rarely been
studied in the field of image processing. Another important
theory used here is the normalized information distance.
Normalized information formula is calculated using the
kolmogorov complexity. These two concepts are theoretical
concepts and practically non computable [12]. This image
quality measure is wavelet based in which the original and the
distorted images are decomposed into different subbands using
a laplacian pyramid subband decomposition method.
Assuming the local independence and decorrelation among the
subbands, kolmogorov complexity is approximated as
(1)
Where and are wavelet subbands of the original and the
distorted image respectively and K is the kolmogorov
complexity of the image. Because of the non computable
nature of kolmogorov complexity, that can be approximated
using the Shannon entropy. Because of the varying nature of
subband coefficients, it is approximated using information
content. The mutual information that is shared between the
original and distorted image subbands is taken for the metric
calculation. A gaussian scale mixture model (GSM) [14] is
adopted for modeling the subband coefficients of the image
and for the noise distortions in the HVS a gaussian channel
model is adopted. Using these models, the final quality metric
is derived. Finally, an information content weighting method
is used to get more accurate value for the quality.
2.4 MS-SSIM (Multi Scale Structural Similarity
Index Measure)
The multi scale structural similarity measure [7] is a more
flexible method than the other single scale methods. The
existing SSIM algorithm is a single scale approach. By using
this multi scale method image details with different
resolutions can be incorporated. Low-pass filtering and down
sampling are the two main operations used in this method. The
original and the distorted images are iteratively low-pass
filtered and then down sampling will be done on that by a
factor of 2. For this multi scale operation, the original image is
taken as scale 1. The highest scale is scale M, so a total of M-1
iterations are taken place. Similar to the SSIM method, three
comparisons have been done here i.e., contrast comparison,
luminance comparison and the structure comparison. Only
luminance comparison is performed on scale M. Other two are
performed on the intermediate scales. The final quality
measure is the combination of these three comparisons. This is
a convenient image quality metric than the other single scale
approaches.
2.5 FSIM (Feature Based Structural Similarity
Index)
This feature based similarity index [5] is mainly based on the
low level features of an image. Most of the methods are based
on structural similarity. Two features are used here for the
calculation of similarity index, phase congruency (PC) and
gradient magnitude (GM) where PC is the primary feature and
GM is the secondary feature. PC is a dimensionless measure.
For the computation of FSIM, PC and GM have to be
extracted from the image. The PC extraction can be done by
using a log Gabor filter. The gradient operators can be Sobel,
Prewitt and Schar operator. The input images can be gray
scale or color images. If it is a color image, PC and GM can be
extracted from the luminance. One of the main advantage of
FSIM is that it can support color images. Sequence of steps
involved in FSIM calculation is given in Fig. 3.
Fig -3: FSIM calculation
2.6 Practical Image Quality Index
This full reference quality metric [1] is mainly based on the
CSF (Contrast Sensitivity Function). It is having a comparable
performance advantages with the other existing image quality
measurement algorithms. This algorithm is also based on the
texture masking effect. This is a wavelet based method in
which the image is divided into different subbands using a
wavelet decomposition method. It is based on the assumption
that the local distortion and the subband distortion contribute
the entire distortion of the image. Already existing contrast
sensitivity function (CSF) [13] is extended here and creating a
new metric known as DCTex. CSF is used to model spatial
frequency transforms. The proposed DCTex metric is the
combination of contrast sensitivity function and the texture
.

__________________________________________________________________________________________
masking effect. Global smoothness and local brightness are
the two masking parameters used in the metric. For the metric
calculation, a sliding window is used that divides the original
image and the distorted image into 8×8 non overlapping
blocks. This proposed metric can be easily integrated into the
JPEG compression standard.
2.7 Content Partitioned Structural Similarity Index
The content partitioned similarity index [4] is a modified
version of SSIM and MS-SSIM. It is a four component image
quality metric in which the original and the distorted images
are divided into four component regions. The four regions are
smooth regions, changed edges, preserved edges and texture
regions. If the estimation of gradient magnitude is large then
the two edges that is, preserved edges and changed edges can
be found. If the magnitude is small, then smooth regions can
be found. Between the low estimation and large estimation,
texture regions can be found. The SSIM index has to be
calculated for finding the image quality. A weighting method
is adopted for the SSIM values. The values at the edge region
are treated by giving larger weights because edges are having
more importance in images. For the final calculation of the
metric, a pooling method is adopted so that the average of the
weighted values can be taken. The different steps involved in
the metric calculation are given in Fig. 4.
Fig -4: Steps involved in content partitioned structural
similarity index
3. PERFORMANCE COMPARISON METHODS
Each image quality assessment algorithm has its own
advantages and disadvantages. The accuracy and efficiency of
algorithms can be found by comparing the performance. For
comparing the performance of image quality assessment
algorithms, four performance comparison metrics are
commonly used by most algorithms. These are Pearson linear
correlation coefficient (PLCC), Kendall rank correlation
coefficient (KRCC), Spearman rank correlation coefficient
(SROCC) and Mean squared error (MSE) [8]. If the SROCC,
KRCC, and PLCC values are higher and MSE value is lower,
then that metric can be considered as a good one. The
performance comparison can be done by comparing the mean
opinion score (MOS) that is found by subjective rating and the
objective score that is found by using the image quality
measurement algorithms. Subjective scores are normally
represented by Mean Opinion Score (MOS) or Differential
mean opinion score (DMOS). For the implementation of these
algorithms many databases can be used. These are publicly
available online. This includes LIVE, CSIQ, TID2008 etc.
CONCLUSIONS
This survey shows various full reference image quality
assessment algorithms such as SSIM, MAD, NPID, MS-
SSIM, FSIM, content portioned SSIM and practical image
quality metric. All these IQA algorithms are pixel based or
wavelet based. Some algorithms taking the structural
information where as some are taking the energy features or
other features for finding out the quality of the image. Each
algorithm has its own merits and demerits. The complexity of
some algorithms is more because of the large computational
time and the use of complex mathematical equations. Based
on the accuracy, time and other requirements we can select the
algorithms.
Table -1: Summary of full reference IQA metrics
REFER-
ENCES
METHODS
USED
REMARKS
[6] Structural
similarity
index
(SSIM)
Can handle color images,
predict contrast change and
mean shift, Less effective when
used to rate badly blurred and
noisy images
[3] Most
apparent
distortion
(MAD)
Improved prediction accuracy,
Perform well for both high
quality and low quality images,
Not suitable for Color Images
[9] Normalized
perceptual
information
distance
(NPID)
Comparable prediction accuracy
with others, using the non
computable concept of
kolmogorov complexity,
Computation time is higher
[7] Multi scale
structural
similarity for
image
quality
assessment
(MS-SSIM).
A good approximation of the
perceived image quality. Giving
relative importance between
different scales, the prediction
capacity is higher, Fails in
measuring badly blurred
images.
Original and distorted image
Final quality metric
Non uniform weighting
Component partition
SSIM map
Pooling

__________________________________________________________________________________________
[5] A Feature
Similarity
Index for
Image
Quality
Assessment
(FSIM)
Can apply to color images,
better predictability for quality,
robustness, covers nearly all
distortions, Relatively high
computation time
[1] Practical
image
quality
metric
Easy to optimize, reliable for
non-uniform distortions, fast
and simple, distortion
calculation repeated several
times
[4] Content-
partitioned
structural
similarity
index
Improved performance against
human subjectivity, Overhead
of setting weights,
Computation time is more
ACKNOWLEDGEMENTS
I would like to thank my project guide Mr. S. John Livingston
for helping me in the field of image quality assessment.
REFERENCES
[1]. Zhang. F., Ma. Lin., Li. S. and Ngan. K. N. 2011,
“Practical image quality metric applied to image coding”,
IEEE Trans. On Multimedia. 13(4), 615-624.
[2]. Wang. Z, Bovik. A. C., Mean squared error: love it or
leave it? A new look at signal fidelity measures, IEEE Signal
Process. Mag. 26 (1) (2009) 98-117.
[3]. Larson. E. C. and Chandler. D. M., “Most apparent
distortion: full reference image quality assessment and the role
of strategy”, J. Electron. Imaging, 19, 011006:1–21, 2010.
[4]. Li. C. and Bovik. A. C., “Content partitioned structural
similarity index for image quality assessment Signal
processing: image communication 25, 517-526, 2011.
[5]. Zhang. Lin., Zhang. Lei., Mou. X. and Zhang. D., “FSIM:
A feature similarity index for image quality assessment”,
IEEE Trans. Image Process. 20(8), 2378–2386, 2011.
[6]. Wang. Z., Bovik. A.C., Sheikh, H.R. and Simoncelli. E.P.,
“Image quality assessment: from error visibility to structural
similarity”, IEEE Trans. Image Process. 13(4), 600–612, 2011.
[7]. Wang. Z., Simoncelli. E.P. and Bovik. A.C., “Multi-scale
structural similarity for image quality assessment”, In:
Proceedings of IEEE Asilomar Conference on Signals,
Systems, and Computers (Pacific Grove, CA), pp. 1398–1402,
2003.
[8]. Wang. Z. and Li. Q., “Information content weighting for
perceptual image quality assessment”, IEEE Trans Image
Process 20(5), 1185– 1198, 2011
[9]. Nikvand. N and Wang. Z, “Image distortion analysis
based on normalized perceptual information distance”, SIViP
(2013) 7:403-410.
[10]. Larson. E. C., Chandler. D.M.: Categorial image quality
(CSIQ) database. (Online) Available
http://vision.okstate.edu/csiq
[11]. Sheikh. H.R. and Bovik. A.C., “ Image information and
visual quality”, IEEE Trans. Image Process. 15(2), 430–444,
(2006)
[12]. Li. M., Chen. X., Li. X., Ma. B. and Vitanyi P. M. B.,
“The similarity metric”, IEEE Trans Information Theory
50(12), 2004
[13]. Ponomarenko. N., Silvestri. F., Egiazarian. K., Carli. M.,
Astola. J. and Lukin. V. (2007) “On between-coefficient
contrast masking of dct basis functions”, In: 3rd
International
Workshop on Video Processing and functions. Scottsdale,
Arizona, USA.
[14]. Wainwright. M.J., Simoncelli. E.P., “Scale mixtures of
gaussians and statistics of natural images”, In: Solla, S.A.,
Leen. T.K., Muller. K.R., (eds) Advances in neural
information processing, vol. 12,pp. 855-861. MIT Press,
Cambridge (2000)
BIOGRAPHIES
Alphy George received her B.Tech (Computer
Science and Engineering) degree from SCMS
school of Engineering and Technology affiliated
by Mahatma Gandhi University in the year 2012.
She is pursuing her M. Tech degree in Computer
Science and Engineering at Karunya University.
S. John Livingston received his M.E. degree
from Karunya University. He is currently working
as an Assistant Professor in Karunya University.

A survey on full reference image quality assessment algorithms

Recommended

More Related Content

What's hot (19)

Similar to A survey on full reference image quality assessment algorithms (20)

More from eSAT Journals (20)

Recently uploaded (20)

A survey on full reference image quality assessment algorithms