Volterra Filters For Edge Enhancement: A General Framework For Quadratic
Volterra Filters For Edge Enhancement: A General Framework For Quadratic
6, JUNE 1996
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
THURNHOFER AND MITRA: GENERAL FRAMEWORK FOR QUADRATIC VOLTERRA FILTERS FOR EDGE ENHANCEMENT 95 1
where HtL(w)is a linear highpass filter. The most obvious We can then write
solution of this problem is to multiply H h ( w ) with a linear
local mean estimator (i.e., an averaging filter). The exact y(n) = hz(n,1,n2)* *2(n>1,n2)l
n=n1=n2 (4)
computational expense for such a structure depends on the
particular filter design, of course. For a filter with a region where ** denotes 2-D convolution. For example, consider the
of support of 3 x 3 pixels, we have to expect abjout 10 system given by ~ ( n=) 2z(n - 1)x(n) - 3 ~ ( n ) x~( n + +
multiplications and 16 additions. We will see that using l)z(n - 2 ) . Using the notation for unit-sample sequences
quadratic Volterra filters, a much more efficient solution can
be found that only needs seven mulliplications, of which two
could be implemented as shift operations and four additions.
Among the various forms of nonlinear filters, Volterra filters the Volterra kernel for this system is then hZ(nl,n2) =
represent the most natural extension of linear filters. Volterra 26(nl - 1,1%2) - 36(nl,n2) + 6(TLl + 1,1%2
2). -
filters can be described as essentially a linear filter with higher Even though Volterra systems are nonlinear in the input se-
order polynomial extensions. Thus, reducing a Volterra filter quence, they are linear in the kernels. It is this observation that
to its first-order component yields a linear filter. On the allows us to use frequency domain techniques for the analysis
other hand, this relationship with linear filters is tht. basis and design of these systems. We define the spectrum of the
for some very useful properties of the higher order terms. Volterra kernels as the multidimensional Fourier transform of
For example, we can define the describing parameteIs as a h,2(ni,n2) [SI, 161
generalized impulse response and its multidimensional Fourier
transform (or Laplace transform for the continuous-time case)
as a generalized frequency response. Even though the filter
is not linear with respect to the input signal anymore, it is nl=-w nz=-oo
still linear in the impulse response coefficients, i.e., a linear
combination of filters is equivalent to a filter with the same The symbol .F denotes the Fourier transform in one or more
linear combination of the kernel parameters. Another important dimensions, depending on the context. We call H z ( w l ,w 2 )
observation is that Volterra filters work on products of input the second-order generalized frequency response [6] or,
samples. Both of these simple properties are fundamental for for simplicity, second-order frequency response. Similarly,
the analysis because it allows us to write a Volterra filter as a hz(n1.n ~ is) called the second-order impulse response.
multidimensional convolution operation. Similar to the design problem of linear filters, the frequency
We restrict ourselves to second-order filters only since they domain representation is an important tool for the design of
are the smallest-degree filter that exhibit a nonlinear behavior. Volterra filters. We rewrite (4) using the frequency responses
Higher-order filters can potentially be very expensive in terms of input and system as
of number of computations, and for implementation purposes,
it is desirable to keep the number of arithmetic operations as (8)
V(n) = - T - 1 { H L ( w l . w Z ) x ( w 1 , w 2 ) } 1n=n1=n2
small as possible.
In Section 11, we describe the basic properties and rellation- where X ( w ) = . F { x ( n ) }and X ( w 1 , w z ) = F { % ( n l , n z ) =
}
ships of quadratic Volterra filters. Section I11 defines various X ( w l ) X ( w 2 ) .Unless noted otherwise, we use lowercase let-
subclasses of quadratic Volterra filters, which can be vvritten ters for sequences in time or space and uppercase letters for
as mean-weighted highpass filters in one and two dimensions. their Fourier transforms.
We also introduce a scheme to describe their dependence on Unlike for linear filters, however, the impulse response of
input frequency and orientation. Based on this description, we quadratic Volterra filters does not necessarily define the filter
use a least-squares approximation to design a 2-D filter for uniquely, i.e., there can exist many kernels for the same filter
perceptual edge extraction. Finally, Section IV illustrates the [7]. Only if the kernel is symmetric does it uniquely describe
application to image enhancement. the system, and for a second-order system, this means that [8]
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
952 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5, NO. 6, JUNE 1996
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
THURNHOFER AND MITRA: GENERAL FRAMEWORK FOR QUADRP,TIC VOLTERRA FILTERS FOR EDGE ENHANCEMENT 953
and thus
class I1 filters.
Additionally, we can show that
Lemma 2: A Class I1 filter can be approximated as a mean-
weighted highpass filter.
Proof: Here, we can again simplify S h , and with the
symmetry condition for hz ( k l , k z ) , we obtain To prove this, we write the kemel as
S" = x h ; ( k , - k ) 2 0.
k
k#O
k=l
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
954 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5 , NO. 6, JUNE 1996
ko t
a=l
b>O
b=O
Using (23)-(25), we find the corresponding extension to
Fig. 4. Parameters a and b determine the orientation of the input sinusoid.
(15) to be They are interlinked by (I' +
h' = 1.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
THURNHOFER AND MITRA: GENERAL FRAMEWORK FOR QUADRATIC VOLTERRA FILTERS FOR EDGE ENHANCEMENT 955
way. Only three values play a role in this scheme: the input = 2zy71.1, 712) - z(n1 1.712 1) + +
frequency, the orientation of the input sinusoid, and the level x z(n1 - 1 , 7 1 2 - 1) - z(n1 1,712 - 1) +
of the constant output. Most importantly, we can show the
relationships graphically, which is a fundamental step toward
x x(n1 - 1,712 1) + (37)
intuitive understanding. Our assumptions greatly simplify the H;"@l, w2. wx,w4)
originally rather complex situation. = 2 - cos(w1+ w2 - wj - w4)
We now want to illustrate how to interpret the plots of - cos(w1 - wz - wg wg) + (38)
directional and frequency dependence. We call these graphs
isotropy plots because they indicate to which degree i2 filter Y;c(b) (WO, a)
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
956 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5, NO. 6, JUNE 1996
w
....... ..; .... ....................... . . . . . . ~
3 25
............. ..-.
............................. :. . . .
Fig. 5. Isotropy plots for the example filters. The left graph shows the regular view, and the right one displays it from the side with the a-axis
perpendicular the paper plane.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
THURNHOFER AND MITRA: GENERAL FRAMEWORK FOR QUADRATIC VOLTERRA FILTERS FOR EDGE ENHANCEMENT 957
Fig. 6. Original “ring” image and results of applying the example filters to it.
(44), we find the equivalent expression to (43) as this set. Furthermore, the basis filters have the advantage that
for each of them the sum of their coefficients equals zero, and
Y ( n l > n 2 )= ~ ~ B k l , k Z / B l , k 2 ( ~ l , ~ 2 (45)
) thus, (17) holds by design, and the coefficients i9kl , k 2 can have
kl kz arbitrary values. The relation between the kernel coefficients
and the t?kl,kL is
where the O k l , k z represent the design coefficients. We prove
this equivalence by rewriting (43) as
h z ( O , O , 0,O) = 7; k , k 2 (46)
ki kz
JL2(n1,nz1-n1, -4 = - @ n l , n z (47)
for nl + In21 #0
O<nl<oo -oo<nz<oo
h 2 ( n 1 , n 2 , -RI, -n2) =0 for nl < 0. (48)
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
9% IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5 , NO. 6, JUNE 1996
131,-1(Wo, U ) = 27r(l - cos(2woa - 2wob)) optimal solution 8, which minimizes the error in the least-
squares sense [lS]
B1,o(wo, U) = 2 7 r ( 1 - cos(2w"a))
B1,1(wo, U) = 2n2(1 - COS(2W"U + 2wob)) H^ = [B'B] -'BTz. (52)
B1,2(wo, U) = 2 x 7 1 - cos(2woa + 4wob))
B~,-~(wO, U) = 27r2(1 - cos(4woa - 4wob)) We want to design a 2-D version of the Teager filter. In
B2,-1(wo, U ) = 2 7 4 1 - cos(4woa - 2wob))
[lo], it has been shown that Teager's algorithm responds
B~,o(w U)o=, 27rz(1 cos(4woa)) -
with an output y ( n ) = sin2(wo) if the input is a sinusoid
z ( n ) = sin(won). Now, we need to find a 2-D system that
B ~ , l ( w oa,) = 27r2(1 cos(4woa 2 w o b ) )
- + outputs a constant signal y(n1, na) = sin2(wo) for a sinusoidal
B ~ , ~ (Uw) = o ,2?(1 - c o s ( 4 w o ~ 4wob)) + input and is, of course, independent of the orientation of the
excitation.
where b = d m .Then, we sample these continuous We set z ( w o , u ) = sin2(wo) and convert z ( w o , a ) into the
functions in -1 5 U 5 1 and 0 5 W O 5 %. Since images
column vector z. Using (52), yields the parameter vector QT
tend to bc oversampled, very little information is contained
given in Table I. If we relax the restrictions somewhat and
in the very high frequencies, and for this design, we only
allow the filter output to be scaled, i.e., z(wg; U) = aisin2(wo),
consider frequencies up to 5.
We choose a sample spacing of
we can normalize the parameters such that E, 8 ( i ) = 3. This
Aa = 0.05 and Awo = 5,
which yields 201 . 26 samples.
yields 8, in Table I. Fig. 7(a) shows the isotropy plot for
We order the samples into column vectors in the following
manner. The first 201 elements belong to W O = 0 and from this filter. The design goal of virtually perfectly isotropical
behavior has been achieved up to W O = 5. If we had used the
U = -1 up to a = 1. After that, we take the 201 samples for
entire range from 0 to 7r for the optimization, then we would
W O = & and so on until we reach W O = and a = 1. We
have obtained a less accurate behavior for smaller frequencies.
number the functions B k l . k 2 ( W O , U ) in the order given before
The advantage of normalizing the parameter vector is that
from 1 to 12 and obtain vectors bl through b 1 2 , which we
combine into the matrix B the elements of 8, can easily be approximated by simple
numbers. The right-most column in Table I shows the approx-
B = (bl b p . .' b i z ) . (49) imation 4, using only the coefficients 1 and 0.5. Thus, we
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
THURNHOFER AND MITRA: GENERAL FRAMEWORK FOR QUADRATIC VOLTERRA FILTERS FOR EDGE ENHANCEMENT 959
Fig. 7. Ikotropy plots for the (a) 2-D Teager filter and (b) its approximation.
obtain the input-output relationship for the approximate 2-D the filter output is very small. Accordingly, undesired noise
Teager filter enhancement only takes place in the brighter portions, where
it is not as perceivable due to Weber’s law. The result of
enhancing the image with the unsharp masking algorithm in
Fig. I, and the quadratic Volterra filter in (53) as the edge
extraction filter is shown in Fig. 8(c). This image is noticeably
sharper, and even small details in the hair and feather regions
have been enhanced.
Another example is shown in Fig. 9. Again, the result
of unsharp masking with our Volterra filter in Fig. 9(c) is
noticeably sharper. Details have been extracted proportionally
to the local image brightness. In the darker portions of the
This equation is considerably simpler to implement than the
images (e.g., the tree region), the filter output in Fig. 9(bj is
full description in 6,. At the same time, the behavior of this
relatively small, whereas in the brighter portions, even minute
system as shown in Fig. 7(b) does not deviate too much from
details result in a strong response.
the ideal one. It is still extremely isotropic for almost the entire
In order to compare these results quantitatively with a
range of interest from 0 to 7r/2.
linear enhancement scheme using one of the commonly used
Laplacian filters, we need a criterion for image sharpness. We
I v . IMAGE PROCESSING APPLICATIONS have found that the mean magnitude of the gradient image
In this section, we want to demonstrate the application (using Sobel’s operator) corresponds well with the perceived
of quadratic filters to the processing of real images. The sharpness. In other words, if two images appear to be equally
designs in the previous sections were based on the idealized sharp, their mean gradient magnitude is approximately the
assumption that the input signal is a single sinusoid, which, of same.
course, is not true for a real image. The small region of support The measure that we use to assess the visibility of noise is a
of the filter, however, makes this approximation reasonable if relative one, that is, it needs a reference image for comparison.
we use the instantaneous frequency for the input sinusoid. The It observes the portion of the local grey-level fluctuation that
test image is a portion of “Lena” in Fig. 8(a) and (b) shows is already present in the reference (original) image and the
the output of the filter in (53). Details in the brighter parts part that is due to noise from the enhancement process. It also
of the image have been extracted, but in the darker areas, considers noise masking by the surrounding area of the image.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
960 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5 , NO. 6, JUNE 1996
(c)
Fig. 8. Original image “Lena” (a) and edge image obtained by applying the quadratic Volterra filter in (53) to the image in (a). Note that in darker
image areas, fewer edges are extracted than in the bright ones, which can be explained by the mean-weighted highpass characteristics of the filter. The
result of the unsharp-masking enhancement is shown in (c) with p = 0.002.
In highly active areas, the human visual system perceives by x and the enhanced one by y. Note that a comparison of
noise to a lesser degree than in flat ones. The local average the original image with itself yields k~ = 0. The value of k~
luminance influences the visibility as well. Altogether, we increases with the degree of image degradation and is (at least
combine these effects into a single number theoretically) not bounded.
Since the original “Lena” and “Park” images are very
(54) clean, white Gaussian noise ( p n = 0.0, 02 = 100.0) was
added to both of them. We then enhanced these images
The index 1 stands for local and indicates that the variances using the Volterra filter in (53), the Volterra filter C in (40)
and the mean are dependent on n1 and 712 and must be (which is called Type 1A in [ l l ] ) , and the Laplacian filters,
computed over a local neighborhood (5 x 5 pixels for our such that the resulting overall image sharpness is the same
simulations). The sum is carried out over all pixels in the in all corresponding cases. Table I1 lists the quantitative
image. Even though the human eye cannot adapt to very small comparisons. Adding noise to an image does not increase
portions of the image, we assume that the observer would view pllg1l significantly, which corresponds well with our visual
the image at arbitrary distances, which justifies the use of a observations. We then computed the noise visibility measure
localized measure. The reference (original) image is denoted k,v relative to the noisy image before enhancement. The
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
~
THURNHOFER AND MITRA GENERAL FRAMEWORK FOR QUADRATIC VOLTERRA FILTERS FOR EDGE ENHANCEMENT 96 1
(c)
Fig. 9. (a) Original image “Park” and (b) edge image obtained by applying the quadratic Volterra filter in (53) to the image in (a). The result of the
unsharp-masking enhancement is shown in (c) with p = 0.002.
V. CONCLUSION
We have developed a subclass of 1-D quadratic Volterra
filters that can be written as a product of local mean and linear found before heuristically, we are now able to develop them
highpass. Even though some filters of this class have been in a unified framework. We have also extended this concept
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
962 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 5, NO. 6, JUNE 1996
(c) (d)
Fig. 10. Comparison of noise enhancement using u n s h q masking. The “Park” image with added Gaussian noise is shown in (a). The results of unsharp
masking using the first Laplacian filter in (b) ( p = 0.45),the second Laplacian filter in (c) ( p = 0.17),and the Volterra filter in (53) in (d) ( p = 0.002)
demonstrate that linear filters amplify noise, especially in the dark areas more than our Volterra filters. All the enhanced images have approximately the
same sharpness. The Laplacian I11 filter performs very poorly, and the resulting image in not shown here.
to 2-D systems obtaining the general class of filters for edge quantitative criteria, we have shown that quadratic Volterra
extraction. Describing these systems using the kemel and the filters yield perceptually better results than standard linear
frequency response alone, however, does not enable us to find filters.
the coefficients in a simple and robust way. Thus, we have
developed an approximate description that is more suitable
for the problem at hand by using simple 2-D inputs. This REFERENCES
also allows us to evaluate the orientational dependence of the [ 11 K. R. Boff, L. Kaufman, and J. P. Thomas, Handbook of Perception and
filter. We have designed an optimal filter using basis filters and Human Performance, vol. I: Sensory Processes and Perception. New
York: Wiley, 1986.
a least-squares frequency domain approach. Compared with [2] J. S. Lim, 2 - 0 Signal and Image Processing. Englewood Cliffs, NJ:
other methods [4], [ 6 ] , [9], [16], [17], this technique gives a Prentice-Hall, 1990.
131 A. K. Jain, Fundamentals of Digital Image Processing, Englewood
more intuitive understanding of the filter behavior and does Cliffs, NJ: Prentice-Hall, 1989.
not depend on heuristic assumption for the impulse response. [4] G. L. Sicuranza, “Quadratic filters for signal processing,” Proc. IEEE,
vol. 80, pp. 1263-1285, Aug. 1992.
As an immediate application in image processing, we have [5] I. Pitas and A. N. Venetsanopoulos, Nonlinear Digital Filters. Boston,
considered the unsharp masking enhancement scheme. Using MA: Kluwer, 1990.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.
THURNHOFER AND MITRA: GENERAL FRAMEWORK FOR QUADRATIC VOLTERRA FILTERS FOR EDGE ENHANCEMENT 963
161 S. A. Billings and K. M. Tsang, “Spectral analysis for nonlinear Sanjit K. Mitra (S’59-M’63-SM’69-F‘74) re-
systems, Part I: Parametric nonlinear spectral analysis,” Mech. Syst. ceived the B.Sc. degree (Hons.) degree in physics
Signal Processing, vol. 3, no. 4, pp. 319-339, 1989. in 1953 from Uktal University, Cuttack, India,
[7] L. 0. Chua and C. Y. Ng, “Frequency domain analysis of nonlinear the M.Sc. (Tech.) degree in radio physics and
systems: general theory,” Electron. Circuits Syst., vol. 3, pp. 165-185, electronics in 1956 from Calcutta University,
July 1979. Calcutta, India, and the M.S. and Ph.D. degrees
[XI M. Schetzen, The Volterra and Wiener Theories of Nonlinear Systems. in electrical engineering from the University
New York: Wiley, 1980. of Califomia, Berkeley, in 1960 and 1962,
[9] G. F. Ramponi, G. L. Sicuranza, and W. Ukovich, “A computational respectively.
method for the design of 2-D Volterra filters,” IEEE Trans. Circuits From June 1962 to June 1965 he was with Comell
Syst., vol. 35, pp. 1095-1102, Sept. 1988. University, Ithaca, NY, as a Professor of Electrical
[IO] J. F. Kaiser, “On a simple algorithm to calculate the ‘energy’ of a Engineering. ]He was with AT&T Bell Laboratories, Holmdel, NJ, from June
signal,” in Proc. IEEE Int. Con$ Acoust., Speech Signal Processing, 1965 to Janu.my 1967. He has been with the faculty of the University of
Albuquerque, NM, Apr. 1990, pp. 381-384. California since then, first at the Davis campus and more recently at the Santa
[ l I] S. K. Mitra, H. Li, I. S. Lin, and T.-H. Yu, “A new class of nonlinear Barbara campus as a Professor of Electrical and Computer Engineering,
filters for image enhancement,” in Proc. IEEE Int. Conf: Acoust, Speech where he served as Chairman of the Department from July 1979 to June
Signal Processing, Toronto, Ont., Canada, 1991, pp. 252552528, 1982.
1121 S . Thumhofer, “Quadratic Volterra filters for edge enhancement and Dr. Mitra served as President of the IEEE Circuits and Systems
their applications in image processing,” Ph.D. dissertation, Univ. of Society in 1986. He is currently a member of the editorial boards of the
Califomia, Santa Barbara, 1995. Intemational Joumal on Circuits and Systems and Siganl Pvocessing, the
[I31 S. Thumhofer and S. K. Mitra, “Volterra filters for perceptual edge Intemational Joumal on Multidimensional Systems and Signal Processing,
extraction,” in Proc. 28th Ann. Asilomar Confi Signals, Syst. Comput., Signal Processing, and the Joumal of the Franklin Institute. He received the
Pacific Grove, CA, Nov. 1994, pp. 736-740. 1973 F. E. Terman Award and the 1985 AT&T Foundation Award of the
[14] S. Thurnhofer and S. K. Mitra, “Designing quadratic Volterra filters American Society of Engineering Education, the Education Award of the
for nonlinear edge enhancement,” in Proc. Inl. Con$ Digital Signal
IEEE Circuits and Systems Society in 1989, and the Distinguished Senior
Processing, Limassol, Cyprus, June 1995, pp. 320-325.
U.S. Scientist Award from the Alexander von Humboldt Foundation of West
[ 151 J. M. Mendel, Lessons in Digital Estimation Theory. Englewood Cliffs,
Germany in 1989. In May 1987, he received an Honorary Doctorate of
NJ: Prentice-Hall, 1987.
[ I 61 G. Ramponi, “Bi-impulse response design of isotropic quadratic filters,” Technology degree from Tampere University, Tampere, Finland. He is a
Proc. IEEE, vol. 78, pp. 665-677, Apr. 1990. Fellow of the AAAS and SPIE and is a member of EURASIP and ASEE.
[I71 G. Ramponi, G. L. Sicuranza, and W. Ukovich, “An optimization
approach to the design of nonlinear Volterra filters,” in Proc. EUSIPCO-
86, Signal Processing III: Theories Applications,, The Hague, The
Netherlands, Sept. 1986, pp. 151-154.
Authorized licensed use limited to: INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR. Downloaded on March 8, 2009 at 22:01 from IEEE Xplore. Restrictions apply.