IMAGE QUALITY MEASURE USING CURVATURE SIMILARITY

Susu Yao, W. Lin, Z.K Lu, E.P. Ong, M.H. Locke and S.Q. Wu

Institute for Infocomm Research, Singapore 119613

E-mail: [email protected]
ABSTRACT human perception for the wavelet coefficients is also taken into
account to compute perceived errors that are integrated with the
This paper proposes a new full-reference objective metric for measure of curvature similarity to give overall image quality index.
image quality assessment. The reference and distorted images are The effectiveness of the proposed method is verified through
decomposed into a number of wavelet subbands, in which mean evaluating an image test database. The experimental results show
curvatures and perceived error of the wavelet coefficients of two that the new method can give a high correlation with the subjective
images are computed and integrated to give overall quality index. scores in terms of prediction accuracy, monotonicity and
Taking structural similarity and error visibility into account, the consistence.
new method can achieve high consistency with subjective This paper is organized as follows. In Section 2, wavelet
evaluation compared with other metrics. Experimental results have decomposition and error sensitivity of wavelet coefficients are
shown the effectiveness of the proposed metric. described. Section 3 gives brief description of surface curvature.
Section 4 presents the proposed image quality assessment method.
Index Terms— Image quality assessment, surface curvature, Experimental results and comparison with other metrics are given
correlation, wavelet transform, error visibility. in Section 5. Finally, conclusions are drawn in Section 6.

2. WAVELETS AND ERROR SENSITIVITY MODEL

1. INTRODUCTION
2.1. Wavelet Decomposition
The quality of a digital image is affected by many factors. In
The wavelet transform is one of the most powerful techniques
practical image processing such as acquisition, compression,
for image processing because of its similarities to the multiple
transmission and reconstruction, the visual quality of image is
channel models of the HVS. In recent years, image quality
degraded in different degrees due to added noise or loss of image
measures have used wavelet transform to perform spatial frequency
information. Usually, we compare original image and processed
decomposition [6][7].
image to evaluate visual quality, namely full-reference quality
Figure 1 shows a two-dimensional frequency space in which
assessment. Subjective evaluation is most accurate but it is time-
four-level hierarchical wavelet subbands are obtained using DWT
consuming and inconvenient. Objective evaluation methods which
9/7 biorthogonal filters. In this space, there are total 13 subbands
can automatically predicate perceived visual quality are desirable
with one low-frequency subband and 12 AC subbands. Each level
in most practical image processing systems such as image and
in the decomposition contains an LH band, an HL band, and an
video coding, dynamic monitoring of image quality. A number of
HH band.
objective image quality assessment methods have been proposed in
past few years [1,2,3], in which many efforts have gone into the ω
investigation of error sensitivity in spatial or frequency domain. LH1 HH1
However, psychological experiments have revealed that the human
visual system (HVS) has more sensitivity to the structural π 2
information variation than to the error visibility. Obvious evidence
LH2 HH2
is that human eyes are less sensitive to the small mean value shift
of an image than to the distortions at image discontinuities. Based
on this fact, Wang [4] proposed an image quality assessment π 4 HL1
method in which one of three factors measures the structural LH3 HH3
similarity using vector correlation. However, the spatial vector HL2
LH4 HH4
correlation cannot distinguish effectively the structural variation. HL3
LL HL4
This paper attempts to further exploit structural similarity in
wavelet bands using differential geometric information because π 4 π 2 ω
human eyes are able to deal with many geometric deformations Figure 1. Titling of the two-dimensional frequency space by four-
quickly and accurately. In 3-D space, a surface model can represent level hierarchical wavelet decomposition.
image regions including flat surfaces, piecewise-smooth curved
surfaces, edges and texture. We use surface curvature similarity 2.2. Wavelet Frequency Error Sensitivity Model
between reference and distorted images as a factor to measure It has been found that human eyes have different sensitivities to
quality degradation. On the other hand, the frequency sensitivity of the different frequency bands. In [5], the base detection thresholds

1-4244-1437-7/07/$20.00 ©2007 IEEE III - 437 ICIP 2007

for the 9/7 DWT are measured using a noise added to the wavelet H = (κ max + κ min ) / 2 [8]. The mean curvature H of a surface
coefficients of a blank image of grey level 128. By fitting the data
S is a measure of curvature that comes from differential geometry
from the psychophysical experimental results in a mathematical
and that locally describes the curvature of an embedded surface.
model for a visually detectable noise threshold y, which is written
These two curvatures are given by:
as follows:
I uu + I vv + I uu I v + I vv I u2 − 2 I u I v I uv
2
log y = log a + k (log f − log gθ f 0 )
2
(1) H= (5)
( )
3

where, θ is the orientation of wavelet subband 2 1 + I u2 + I v2 2

( θ = HH , HL, LH ), f the spatial frequency (cycles/degree) I uu I vv − I uv2

K= (6)
determined by both the display resolution r , viewing distance and
wavelet decomposition level λ . f = r 2− λ . a, k , f 0 and gθ are (1 + I 2
u + I v2 )
2

constants which can be found in [5]. The wavelet error detection

Where, I u , I v , I uu , I uv and I vv are the partial derivatives of I .
threshold is then given by Surface curvature is complementary to edge information, which
§ λ 2· is a useful feature for scene analysis, feature extraction, and object
k ¨ log §¨ 2 f 0 gθ ·
¸ ¸
a ¨ © r
¹ ¸ recognition. It has been widely used for image segmentation and
WTλ ,θ = 10 © ¹ (2)
classification in computer vision. In image coding, visual quality is
Aλ ,θ
usually affected by the appearance of coding artifacts, such as
Where, Aλ ,θ is the amplitude of the DWT 9/7 basis function blockiness, blurring and ringing effects. These coding artifacts
deform the surface structure information and cause perceived
corresponding to level λ and orientation θ .
noisiness. As an example, Fig. 2 shows the original and coded
It is easy to verify from the equation (2) that the threshold for images with their curvature maps, from which we can see that the
HH band is the highest while the threshold for LL band is the curvatures of distorted image at smooth regions such as face and
lowest. In other words, human eyes are more sensitive to the low background are quite different from original curvature at those
frequency band than to the high frequency band. They are also points. Therefore by computing curvature similarity between two
dependent on orientation, being most sensitive in the horizontal images can measure the degradation degree of the image quality.
and vertical directions and least sensitive at oblique angles.
Taking into account of the masking effect [7], which is a
function of the actual values of the wavelet coefficients, the
detection threshold will elevate, which can be given by:
(
Tλ ,θ (u , v ) = max WTλ ,θ , Cλ ,θ (u , v ) (3) )
Where, Cλ ,θ (u , v ) denotes the wavelet coefficients and (u, v )
the coordinate. Due to elevated detection threshold the perceived
error between the wavelet coefficients of original and distorted
images denoted as Cλo,θ (u , v ) and Cλd,θ (u , v ) will be reduced,
which can be written as follows: (a) (b)
(
ΔCλ ,θ (u , v ) = Cλ ,θ (u , v ) − C
o d
λ ,θ (u, v )) Tλ ,θ (u, v ) (4)

3. SURFACE CURVATURE

A two-dimensional surface model can represent an image with

smooth, edge and texture regions. We assume that the surface of an
image can be specified as the height I (u, v ) above the support
plane defined by two coordinates (u, v ) . A 3-D point on the (c) (d)
surface is given by z (u, v ) = (u, v, I (u, v )) . Let p be a point on the Fig. 2. (a) Original image; (b) JPEG coded image; (c) Mean
curvature map of (a); (d) Mean curvature map of (b)
surface S. Consider all curves on S passing through the point p on
the surface. Every such curve ci has an associated curvature K i , 4. THE PROPOSED IMAGE QUALITY METRIC
among them the maximal and minimal values are known as the
principal curvatures of the surface, which are denoted as In this section, a full-reference image quality metric is presented.
Firstly, the reference image and its distorted version are
κ max and κ min . The product of κ max and κ min is called decomposed into four-level structure with total 13 subbands using
Gaussian curvature at p ∈ S , i.e., K = κ max ⋅ κ min . The 9/7 biorthogonal wavelet filter banks. In each subband, mean
surface curvature maps are obtained using equation (5). The
corresponding average is known as the mean curvature at p ∈ S , similarity between two images can be quantified in terms of the
correlation function. We calculate the correlation coefficients

III - 438
between two curvature maps of original and distorted images in the 5.2. Criteria for Metric Performance
wavelet subbands using following formula:
¦ (H − μ o )(H ud,v − μ d )
o Following the performance evaluation methods adopted in the
u ,v VQEG Phase-I test [9], we use three evaluation criteria to give
Corr (H o , H d ) = u ,v (7) quantitative measures on the performance of the proposed method.
¦ (H − μo ) ¦ (H − μd )
o 2 d 2 The first criterion is called non-linear correlation, which measures
u ,v u ,v
u ,v u ,v
the prediction accuracy, i.e., the ability of a metric to predict
subjective ratings. It is given by computing the normalized
Where, H uo, v and H ud, v denote mean curvatures of original image correlation coefficient between subjective MOS and objective
and its distorted image at point (u, v ) on the surface, respectively. rating that is fitted via a four-parameter cubic polynomial to the
corresponding MOS, called non-linear regression analysis. The
μ o and μ d are the corresponding mean values of H uo, v and second criterion is the Spearman rank-order correlation, which
measures the prediction monotonicity of a model, i.e., whether the
H ud, v . increases or decreases in one variable are associated with the
Taking perceived error of wavelet coefficients and structural variation of other variable. The third one is outlier ratio, which
similarity into account, the overall quality measure using curvature calculates the percentage of the number of predictions outside the
similarity, namely QMCS in short, is obtained by summing the range of ± 2 times of the standard deviations, which is used as a
values of quality index in all the subbands, which is written as measure of prediction consistency. Besides, we also use Mean
follows: Absolute Error (MAE) and Root Mean Square Error (RMSE) as
the performance indexes for the purpose of comparison with other
§ ·
¨ ¸ quality assessment metrics.
¨ ¸
¨ ¸ 5.3. Experimental Results and Performance Comparison
¨ 1 ¸ (8)
QMCS = ¦ ¨
For each image in the database, we compute its objective
¸
λ ,θ ¨ (
Corr H λo,θ , H λd,θ ) 0 .5
¸
quality score using the proposed method. Then performance
indexes including non-linear correlation, Spearman rank-order
¨1+ ¸
¦ (ΔC λ θ (u , v ) − μ )
correlation, outlier ratio, MAE and RMSE are calculated. In order
¨ 1 2 ¸ to compare the proposed quality metric with other competitive
¨ N λ ,θ
, ΔCλ ,θ
¸
© u ,v ¹ metrics on the same database, three main quality assessment
methods have been tested, which are named PSNR, Sarnoff [11],
denotes the mean value of ΔCλ ,θ (u , v ) ,
MSSIM [4]. The experimental results are listed in Table I. Figs. 3-
Where, μ ΔCλ θ
, 6 draw the scatter plots of MOS versus PSNR, Sarnoff, MSSIM
λ = 1,⋅ ⋅ ⋅,4 , and N λ ,θ is the number of pixels at level λ and and proposed QMCS, respectively. From the experimental results,
we can see that a significant improvement for the image quality
orientation θ . With this equation, we can
obtain a quality score measure in terms of five performance evaluation indexes has been
for each image. As the difference between two images tends achieved in comparison with MSSIM method that used spatial
towards zero ΔCλ ,θ → 0 , the correlation coefficient tends vector correlation as the measure of structural information. It
should be pointed out that the data in Table I and scatter plots
towards 1, the quality score approaches zero.
(Figs.3-5) for PSNR, Sarnoff, and MSSIM models are from [4] for
the purpose of comparison.
5. EXPERIMENTAL RESULTS
Table I. Performance comparison of image quality assessment
5.1. The Test Image Database models; PCC: correlation coefficient; MAE: mean absolute error;
RMSE: root-mean-square error; OR: outlier ratio; SROCC:
We use the image database [10] developed by the Laboratory spearman rank-order correlation coefficient
of Image and Video Engineering (LIVE), the University of Texas Non-linear Regression Rank-
at Austin to test the performance of the proposed quality metric. order
The database is composed of total 344 images that were obtained
Model PCC MAE RMSE OR SROCC
by compressing twenty-nine high-resolution 24-bits/pixel RGB
color images, including 175 JPEG compressed images and 169 PSNR 0.905 6.53 8.45 0.157 0.901
JPEG 2000 compressed images. The compression bit rates were in Sarnoff 0.956 4.66 5.81 0.064 0.947
the range of 0.150 to 3.336 and 0.028 to 3.150 bits/pixel, MSSIM 0.967 3.95 5.06 0.041 0.963
respectively. QMCS 0.971 3.81 4.75 0.012 0.966
In subjective evaluation procedure, observers are asked to
provide their votes of perceived image quality on a continuous 6. CONCLUSIONS
linear scale that was divided into five-grade description marked
with “Bad”, “Poor”, “Fair”, “Good” and “Excellent”. Each JPEG In this paper, we have presented a new method for image quality
and JPEG 2000 compressed image was viewed by 13~20 subjects evaluation. From the view point of comparing structural
and 25 subjects, respectively. The raw scores given by each subject information variation of a reference image and distorted image, we
were scaled to the full range (1~100). Finally, subjective Mean used mean curvature similarity combined with perceived error of
Opinion Score (MOS) value for each distorted image is obtained the wavelet coefficients. Experiments on JPEG and JPEG 2000
by taking the average of those rating values. compressed image database have shown that the new quality metric

III - 439
can obtain a high correlation with subjective evaluation scores in
100
terms of prediction accuracy, monotonicity and consistency.
JPEG Images
90
Fitting with Logistic Function
JPEG2000 Images
80

70

60

MOS
50

40

30

20

10

0
0 2 4 6 8 10 12
QMCS

Fig. 6. Scatter plot of MOS vs. QMCS

Fig. 3. Scatter plot of MOS vs. PSNR

REFERENCES

[1] S. Winkler, “Quality metric design: a closer look”,

Proceedings of the SPIE Human Vision and Electronic
Imaging, vol. 3959, pp.37-44, San Jose, CA, Jan. 22-28, 2000.
[2] Z. Yu, H.R. Wu, S. Winkler, T. Chen, “Vision model based
impairment metric to evaluate blocking artifacts in digital
video”, Proceedings of IEEE, Vol.90, No.1, pp.154-169, 2002.
[3] P.C. Teo and D.J. Heeger, “Perceptual image distortion,” Proc.
SPIE, vol. 2179, pp.127-141, 1994.
[4] Z. Wang, et.al., “Image quality assessment: from error visibility
to structural similarity”, IEEE Trans. Image Processing,
vol.13, No.4, pp. 600-612, 2004.
[5] A.B. Watson, G.Y. Yang, J.A. Solomon, and J. Villasenor,
“Visibility of wavelet quantization noise,” IEEE Trans. Image
Processing, Vol. 6, No. 8, pp. 1164-1175, 1997.
Fig. 4. Scatter plot of MOS vs. Sarnoff [6] Y.K. Lai and C.-C. J. Kuo, “A Haar wavelet approach to
compressed image quality measurement,” J. Vis. Commun.
Image Represent., Vol. 11, pp.17-40, 2000.
[7] A.P. Bradley, “A wavelet visible difference predictor,” IEEE
Trans. Image Processing, Vol. 8, pp. 717-730, May 1999.
[8] M.do Carmo, “Differential geometry of curves and surface,”
Prentice-Hall, 1976.
[9] VQEG: The Video Quality Expertise Group,
http://www.vqeg.org/.
[10] H.R. Sheikh, Z. Wang, L. Cormack, and A.C. Bovik, “LIVE
image quality assessment database”,
http://live.eve.utexas.edu/research/quality (2003).
[11] Sarnoff Corporation, “JNDmetrix Technology,”
http://www.sarnoff.com/products_services/video_vision/jndme
trix/.

Fig. 5. Scatter plot of MOS vs. MSSIM

III - 440