MULTI-SCALE RETINEX FOR COLOR IMAGE ENHANCEMENT

Zia-ur Rahman, Member IEEE

Science and Technology Corporation 101 Research Drive Hampton, VA 23666
ABSTRACT The retinex is a human perception-based image processing algorithm which provides color constancy and dynamic range compression. We have previously reported on a single-scale retinex (SSR) and shown that it can either achieve color/lightness rendition or dynamic range compression, but not both simultaneously. We now present a multi-scale retinex (MSR) which overcomes this limitation for most scenes. Both color rendition and dyinamic range compression are successfully accomplished except for some pathological scenes that have very strong spectral characteristics in a single band.
1. INTRODUCTION

Daniel J. Jobson and Glenn A . Woodell

NASA Langley Research Center MS 473, 8 N Dryden Street

Hampton, Virginia 23681
and tonal rendition, and one can be improved only at the cost of reducing the other. This paper describes our initial research in alleviating some of these trade-offs by using a multi-scale retinex (MSR), i.e. a retinex which combines several SSR outputs to product: a single output image which has both good dynamic range compression and color constancy, and good tonal rendition. The tonal rendition, though, is still scene dependent to a certain extent. We will briefly describe the MSR in Section 2. In section 3 we will provide some of the results of applying the MSR to color images and compare our results with other techniques for image enhancement. Finally, in Section 4 we will discuss the future direction for this research.
2. THE MULTI-SCALE RETINEX

A common problem with color imagery-digital or analog-is that of successful capture of the dynamic range and colors seen through the viewfinder onto the acquired image. More often than not, this image is a poor rendition of the actual observed scene. In 1986, Edwin Land presented the last version of his retinex[l] as a model for human color constancy. Hurlbert[2, 31 showed that there is no mathematical solution to the problem of removing lighting variations. Moore[4, 5 1 implemented a version of the retinex in analog VLSI for real-time dynamic range compression but encountered scene context dependent limitations and hence failed to achieve a generalized implementation. More recently we, inspired by the work of Land, Hurlbert, and Moore decided to delve into this commonly occurring, but surprisingly intractable, problem. Our initial research resulted in the single-scale retinex (SSR) that we have described in detail previouslyE6, 7, 81. The SSR shows exceptional promise for dynamic range compression but does not provide good tonal rendition. In fact, a distinct trade-off controlled by the scale of the surround function exists between dynamic range compression
.This work was performed under a NASA Langley Research Center Contract #NASl-19603

The MSR can be compactly written as

N
n=l

where the subscripts :i E R, G, B represent the three color bands, N is the number of scales being used, and Wn are the weighting factors for the scales. The M n ( z ,y) are the surround functions given by

~ ( x , K~ exp[-(x2 y) =

+y2)/u~],

where the un are the standard deviations of the Gaussian distribution that determine the scale. The magnitude of the scale deterimines the type of information that the retinex provides: smaller scales providing more dynamic range compression, and larger scales providing more color constancy. The I(, are selected so that J F ( x , y) dx d y = 1. Each of the expressions within the summation in Eq. 1 reprfesents an SSR. The SSR has been previously defined[6] to have the following characteristics and properties:

0-7803-3258-X/96/$5.00 0 1996 IEEE

1003

1. The functional form of the surround is a Gaussian.

2. The placement of the log function is AFTER sur-

round formation.

3 . The post-retinex signal processing is a canonical gain-offset rather than an automatic gainoffset. 4. There is a trade-off between dynamic range compression and tonal rendition which is governed by the Gaussian surround space constant. A space constant of 80 pixels was a reasonable compromise between dynamic range compression and rendition.

5. A single scale seemed incapable of simultaneously

providing sufficient dynamic range compression and tonal rendition. 6. Violations of the gray-world assumption led to retinexed images which were either grayed-out locally or globally or, more rarely, suffered from color distortion. The MSR combines the dynamic range compression of the small scale retinex with the tonal rendition of the large scale retinex to produce an output which encompasses both.

As stated above, the MSR still suffers from grayingout of uniform zones much as the SSR did. The advantage that the MSR has over the SSR is in the combination of scales which provide both dynamic range compression and tonal rendition at the same time. The overall result of the application of the MSR is still more saturated than human observation, giving the final image a washed-out appearance, but it preserves most of the detail in the scene. This graying of areas of constant intensity occurs because the retinex processing enhances each color band as a function of its surround. The smaller values in the weaker channels get pushed up strongly, making them approximately equal in magnitude to the dominant channel, leading to a graying out of the overall region. Moore[4] encountered this problem in his implementation of the retinex and attempted to resolve it with using variable gains across the color channels. We do not attempt a solution in this paper but provide a detailed solution elsewhere.[g] However, the MSR produces a much better final image in terms of color, and dynamic range than the SSR. Figure 1 shows a comparison of the SSR and the MSR processing. The differences are easier to see in the original color images (see CD-ROM version of paper), but if one looks around the left side of the

Figure (a) Original (b) Single-scale Retinex (c) Multi-scale Retinex

1004

face and in the area just above the right shoulder of the pictured iman, one sees details for the MSR which are not evident in the SSR. Also the haloing artifacts peculiar to the SSR are eliminated in the MSR.

4. FUTURE RESEARCH

3. RESULTS

Figure 3 shows a comparison of the MSR with image enhancement methods typically used for dynamic range compression. The scenes are selected to show the effects of MSR processing on good images (top row), wide dynamic range compression that is achieved by the MSR (middle row), and color constancy (bottom row). Histogram equalization performs well for the child image, but begins to saturate in both the grass image and the cave image. The logarithmic non-linearity has the poorest performance for all three scenes, though its dynamic range compression capabilities are quite evident in the grass scene. For the MSR processing, the uniform regions in the child scene tend to gray out, but the overall result is still quite good. For the grassy field, the MSR processing compresses the wide dynamic range well anti brings out the colors in both the bright and the dark areas very well. For the cave image, the color of the inside rock, and the outside rock formations are both brought out so they agree with actual observation. The CD-ROM version of the proceedings contains the color postscript figures and the comparisons are much easier to make. The MSR (outputbrings out most of the detail in the black regions but at the cost of enhancing the noise in these regions. This noise is a result of the poor signalto-noise ratio in these areas. The traditional techniques are also able to enhance the dark regions, but not to the same extent as the MSR. In fact, the MSR achieves a balance bei,ween enhancing the darks, yet, at the same time, retaining the colors in the bright regions, as opposed to traditional point non-linearities which tend to enhance the darks at the cost of saturating the brights (Figs. 3(b,c)). Of course, the final rendition in still scene-dependent and can often be grayed-out if the original scene contains large areas of constant intensity (Fig. 3(d)(top row)). The MSR output is different from existing techniques in that the overall effect of processing is scene dependent but the processing itself is not. In other words, though the overall effect adapts itself to the lighting varialions within the scene, the same process, with exactly t8hesame control parameters can be used for any image. This is not true for other adaptive techniques since variations in lighting conditions imply variations in the control parameters.

The main direction of jfurther research is to improve the color rendition of the MSR. Though it produces excellent dynamic range compression, the tonal rendition is scene dependent and can be quite poor. Work is already underway on a newer version of the MSR which combines a post-filter with the MSR to produce an MSR which provides very good color rendition with a very slight loss in overall dynamic range compression.

5. REFERENCES
[l] Edwin Land. Recent advances in retinex theory. Visio~Research, 26(1):7-21, 1986.

[a] Anya

C. Hurlbert. Formal connections between lightness algorithms. Journal of the Optacal Society of America A , 3:1684-1693, 1986.

[3] Anya C. Hurlbert. The Computation of Color. PhD thesis, Massachusetts Institute of Technology, September 1989. [4] Andrew Moore, J . Allman, and R. M. Goodman. A real-time neural system for color constancy. IEEE Transactaons on Neural Networks, 2(2):237-247, March 1991. [5] Andrew Moore, G. Fox, J . Allman, and R. M. Goodman. A VLSI neural network for color constancy. In D. S. Touretzky and R. Lippman, editors, Advances an Neural Informataon Processzng 3, pages 370-376. Morgan Kaufmann, San Mateo, CA, 1991. [6] Daniel J. Jobson, Zia-ur Rahman, and Glenn A. Woodell. Propertieis and performance of a center/surround retinex IEEE Transactaons on Image Processing. Submitted 1995. [7] Zia-ur Rahman. Properties of a center/surround Retinex Part One: Signal processing design. N A S A Contractor Report #198194, 1995. [8] Daniel J . Jobson and Glenn A. Woodell. Properties of a center/surround Retinex Part Two: Surround design. N A S A Technical Memorandum #110188, 1995. [9] Daniel J . Jobson, Ziia-ur Rahman, and Glenn A. Woodell. A multi-scale Retinex for bridging the gap between color images and the human observation of scenes. IEEE Trarilsactaons on Image Processzng. Submitted 1996.

1005

1006