Singh 2015
Singh 2015
PII: S0030-4026(15)00532-X
DOI: http://dx.doi.org/doi:10.1016/j.ijleo.2015.06.060
Reference: IJLEO 55695
To appear in:
Please cite this article as: K. Singh, R. Kapoor, S.Kr. Sinha, Enhancement
of low Exposure Images via Recursive Histogram Equalization Algorithms,
Optik - International Journal for Light and Electron Optics (2015),
http://dx.doi.org/10.1016/j.ijleo.2015.06.060
This is a PDF file of an unedited manuscript that has been accepted for publication.
As a service to our customers we are providing this early version of the manuscript.
The manuscript will undergo copyediting, typesetting, and review of the resulting proof
before it is published in its final form. Please note that during the production process
errors may be discovered which could affect the content, and all legal disclaimers that
apply to the journal pertain.
1 Enhancement of low Exposure Images via Recursive Histogram
2
3
4 Equalization Algorithms
5
6
7
8 Kuldeep Singh a, Rajiv Kapoor b, Sanjeev Kr. Sinha c
9
10 a
Central Research Lab, Bharat Electronics Ltd, Ghaziabad, 201010, India. Phone No: +91-
11
t
12
ip
13 9910101592, email: [email protected]
14
b
15 Department of Electronics & Communication, Delhi Technological University, Delhi,
cr
16
17
18 110042, India. email: [email protected]
us
19
c
20 Department of Computer Science, Indian School of Mines, Dhanbad, 826004,India.
21
22
Abstract:
23
24
25
26
an
This paper proposes two exposure based recursive Histogram equalization methods for image
27 enhancement. The proposed methods are very effective for images acquired in low light
M
28
29
30 condition like underwater sequences or night vision images. The first method is Recursive
31
32
ed
37 predefined threshold. The second method is named as Recursively separated Exposure based
38
39
sub image histogram equalization (RS-ESIHE) that performs the separation of image
ce
40
41
42 histogram recursively ; separate each new histogram further based on their respective
43
44
Ac
exposure thresholds and equalize each sub histogram individually. The experimental results
45
46
47 show that low exposure image enhancement problem was not addressed by earlier HE based
48
49 methods , has been efficiently handled by these new methods. The performance evaluation of
50
51
52 new methods is done in terms of Image information content as well as visual quality
53
54 inspection. The proposed methods outperforms earlier HE based contrast enhancement
55
56
57 algorithms specifically for low light images.
58
59 Keywords:
60
61
62
63
64 Page 1 of 17
65
Recursive Histogram equalization, Image Information content, Image Exposure, Low
1 exposure Imaging
2
3
4 1. Introduction
5
6
7 Although there is a tremendous advancement in image capturing devices, still natural images
8
9 are often subject to low-exposure problems under low light or underwater conditions. Digital
10
11
t
12 cameras have a limited dynamic range as a result photographs acquired in high dynamic range
ip
13
14 scenes often exhibit underexposure artifacts in shadow regions [1]. An image captured in a
15
cr
16
dim light environment encounters low-exposure problem caused by non-ideal camera settings
17
18
of aperture and shutter speed. Exposure in an image determines the brightness or darkness of
us
19
20
21 each element in the image [2]. In the low illumination scenario, post-processing using image
22
23
24
25
26
an
enhancement tools is needed to improve the quality of the acquired image. Many Histogram
equalization based image enhancement methods were proposed to cope with contrast related
27
M
28
29 issues. Histogram equalization (HE) is most extensively utilized contrast enhancement
30
31 technique due to its simplicity and ease of implementation [3]. Histogram Equalization
32
ed
33
34
flattens the probability distribution and stretches the dynamic range of gray levels, which in
35
36 result improves the overall contrast of the image [4]. Applying HE straight away on natural
pt
37
38 images is not suitable for most consumer electronics applications such as TV, Cameras etc., as
39
ce
40
41 it tends to change the mean brightness of the image to the middle level of the gray level range,
42
43 which in turn produces annoying artifacts and intensity saturation effects. Kim [4] was the
44
Ac
45
46 first one to propose an algorithm named brightness preserving bi histogram equalization
47
48 (BBHE) which preserves the mean brightness of the image and improves the contrast. BBHE
49
50
51 bisects the histogram based on the input mean brightness and equalizes the two sub
52
53 histograms independently. Wan et al. [5] proposed an algorithm named dualistic sub image
54
55
histogram equalization (DSIHE) and claimed that it is better than BBHE in terms of
56
57
58 preservation of brightness and information content (entropy) of the image. DSIHE separates
59
60 the histogram based on median value instead of mean, which implies that each histogram
61
62
63
64 Page 2 of 17
65
contains almost equal number of pixels. Chen and Ramli introduced minimum mean
1 brightness error bi-histogram equalization (MMBEBHE) for preserving the mean brightness
2
3
4 “optimally” [6]. MMBEBHE is an extension of BBHE, which iteratively calculates the
5
6 absolute mean brightness error (AMBE) for gray levels 0 to L-1 and bisects the histogram
7
8
9
based on the intensity value , which yields minimum AMBE. Chen and Ramli [7]
10
11 proposed another approach named recursive mean-separate histogram equalization (RMSHE).
t
12
ip
13 This technique iteratively performs the BBHE in which the histogram is divided into two parts
14
15
cr
16 based on the average input brightness and BBHE is performed to each sub histogram
17
18 independently. Sim et al. [8] proposed a similar technique to RMSHE known as recursive sub-
us
19
20
21 image histogram equalization (RSIHE). This technique performs the division of histogram
22
23 based on the median value of brightness instead of mean brightness. Kim and Chung [9]
24
25
26
an
presented a method named Recursively Separated and Weighted Histogram Equalization
27
M
28 (RSWHE) similar to RSIHE and RMSHE methods. In addition, RSWHE modify the
29
30 histogram by weighting process using normalized power law function. These techniques do
31
32
ed
33 not provide a mechanism for adjusting the level of enhancement. Clipped histogram based
34
35 techniques [10-12] were proposed as a solution for controlling the enhancement rate as well as
36
pt
37
38 preserving the original brightness. These methods control maximum value of the histogram by
39
ce
40 clipping histograms higher than the pre specified threshold. Different approaches are proposed
41
42
43 for the determination of clipping threshold or plateau limit. Wadud et. al. [13] proposed a new
44
Ac
45 class of histogram partitioning named as dynamic histogram equalization (DHE). The DHE
46
47
48
partitions the original histogram based on local minima and then, assigns a new dynamic
49
50 range to each sub-histogram. The major drawback of this method is that it remaps the
51
52 histogram peaks by allocating new dynamic range, which significantly changes the mean
53
54
55 brightness. Ibrahim and Kong [14], proposed a method brightness preserving dynamic
56
57 histogram equalization (BPDHE) similar to DHE, which is extension of the DHE. BPDHE
58
59
60 applies Gaussian-smoothing filter before the histogram partitioning process is carried out. The
61
62
63
64 Page 3 of 17
65
BPDHE uses the local maxima as the separating point rather than the local minima used in
1 DHE. Ibrahim and Kang claimed that the local maxima are better for mean sbrightness
2
3
4 preservation. Sheet et. al. [15] proposed a modification of the BPDHE technique named as
5
6 Brightness Preserving Dynamic Fuzzy Histogram Equalization (BPFDHE). This method uses
7
8
Fuzzy histogram computation for smoothing operation of histogram before partitioning of
9
10
11 image into sub histograms. The authors of BPDFHE method proved the superiority of the
t
12
ip
13 algorithm in terms of less computational time and brightness preservation. These dynamic
14
15
cr
16 methods are suitable only for the images having significant peaks in the histograms.
17
18 Quadrants Dynamic Histogram Equalization (QDHE) method was proposed by Ooi and Isa
us
19
20
21 [16] for better contrast enhancement. QDHE partitions the histogram into four sub histograms
22
23 using the median value of intensity and then clips the histogram according to the mean of
24
25
26
an
intensity occurrence of the input image and finally a new dynamic range is assigned to each
27
M
28 sub-histogram before each sub histogram is equalized. Ooi and Isa [17] also proposed
29
30 Adaptive Contrast Enhancement Methods with Brightness Preserving which comprised of two
31
32
ed
33 methods named as dynamic quadrants histogram equalization plateau limit (DQHEPL) and bi-
34
35 histogram equalization median plateau limit (BHEPL-D). DQHEPL is extension of RSIHE,
36
pt
37
38 divides the histogram into four sub histograms, and then assigns a new dynamic range and
39
ce
40 finally implements clipping process. BHEPL-D is the extension of the BHEPL except that it
41
42
43 clips the histogram using the median of the occupied intensity. Chang and Chang [18]
44
Ac
1 [20] proposed Exposure based Sub Image Histogram Equalization (ESIHE) method for
2
3
4 enhancement of low exposure images where the image exposure threshold is used for sub
5
6 dividing image. Low exposure images possess limited dynamic range resulting in poor
7
8
contrast.. Images having histogram bins concentrated towards the lower part or the darker
9
10
11 gray levels possess low intensity exposure whereas images having histogram bins
t
12
ip
13 concentrated towards the higher part or the brighter part possess high intensity exposure and
14
15
cr
16 both category of images exhibit poor contrast. Although various techniques are available to
17
18 cater specific problem of contrast enhancement, but enhancement of low exposure images is
us
19
20
21 still less explored area. In this work, two extensions of the ESIHE are proposed. First, a
22
23 recursive method is proposed as the extension of the ESIHE, called Recursive Exposure based
24
25
26
an
sub image histogram equalization (R-ESIHE). The proposed method iteratively performs
27
M
28 ESIHE on the image until the difference of exposure values between subsequent iterations is
29
30 less than a threshold value. The second method is Recursively separated Exposure based sub
31
32
ed
33 image histogram equalization (RS-ESIHE). Unlike the R-ESIHE, the second algorithm first
34
35 splits the histogram into two or more sub histograms based on the individual exposure
36
pt
37
38 thresholds and then performs histogram equalization of all the sub histograms. Clipping of
39
ce
45
46 gives experimental results, and section 4 concludes the paper.
47
48
49 2. Recursive Histogram Equalization for Low exposure Images
50
51
52 Both the proposed methods undergo two fundamental steps: Exposure thresholds calculation
53
54 and Histogram Clipping. The description of each step is presented in the following
55
56
57 subsections.
58
59
60 2.1 Exposure Threshold calculation
61
62
63
64 Page 5 of 17
65
The categorization of an image into low or high exposure is done based on exposure threshold
1 (Singh et al, 2013). The normalized range of exposure value is [0-1]. The images containing
2
3
4 majority of low exposure regions possess exposure values lesser than 0.5 tending towards zero
5
6 however, the overexposed images have exposure values greater than 0.5 tending towards one.
7
8
Image intensity exposure value can be calculated as eq. (1).
9
10
11
t
12 (1)
ip
13
14
15
cr
16 Where h(k) is histogram of image and L is total number of gray levels. Parameter (as
17
18 calculated in eq. 2) is the gray level boundary value that divides the image into under exposed
us
19
20
21 and over exposed sub images.
22
23
24
25
26 2.2 Histogram Clipping
an (2)
27
M
28
29
30
The idea behind histogram clipping is to prevent over enhancement leading to the natural
31
32 appearance of the image. For limiting the enhancement rate, we need to limit the first
ed
33
34 derivative of histogram or the histogram itself (Ooi et. al., 2009).The histogram bins having
35
36
pt
37 the value greater than the clipping threshold are limited to the threshold (Fig.1).
38
39
ce
40 Number of Pixels
41
42
43 Tc
44
Ac
45
46
47
Over Exposed
48 Under Exposed
49 Intensity
50
51 0 Xa L-1
52
53
54 Fig.1. Process of Histogram Sub Division and Clipping
55
56 The clipping threshold is calculated as an average number of gray level occurrences. The
57
58
59 formula for clipping threshold is presented in (3) and (4) calculates the clipped histogram
60
61
62
63
64 Page 6 of 17
65
(3)
1
2 (4)
3
4
5
6 Where h(k) and hc(k) are the original and clipped histogram respectively. This method of
7
8
9 histogram clipping is computationally efficient and consumes lesser time.
10
11
t
12 2.3 Recursive Exposure based sub-image histogram equalization (R-ESIHE)
ip
13
14
15 The proposed RESIHE method is a recursive variant of ESIHE, which performs ESIHE
cr
16
17 recursively on the given histogram. The number of recursions is dependent on the exposure
18
us
19
20 difference between successive iteration. The fundamental step of the algorithm is histogram
21
22 subdivision and equalization.
23
24
25
26
2.3.1 Histogram Sub Division and Equalization
an
27
M
28
The original histogram is first bisected in two sub images WL and WU ranging from gray level
29
30
31 0 to and to L-1 based on exposure threshold value Xa as calculated in (2).
32
ed
33 and are corresponding PDF of these sub images as defined in eq. (5-6)
34
35
36
pt
37 (5)
38
39
ce
40
41 (6)
42
43
44
Ac
45 Where and are total number of pixels in sub images WL and WU respectively.
46
47
48 and are corresponding CDF of individual sub images and CDFs can be defined as eq.
49
50 (7-8)
51
52
53
54 (7)
55
56
57 (8)
58
59
60
61
62
63
64 Page 7 of 17
65
The next step is to equalize all the four sub histograms individually. The transfer functions for
t
12 final step involves the integration of both sub images into one complete image. The number of
ip
13
14 iterations is decided based on a threshold £ whose value is normally taken very less (here
15
cr
16
17 0.01).
18
us
19 2.3.2 Algorithm of RESIHE
20
21
22 Step 1: Compute the histogram of the image.
23
24
25
26
an
Step 2: Compute the value of exposure and threshold parameter .
33
34
35 Step 5: Apply the histogram equalization on individual sub histograms.
36
pt
40
41
42 threshold £.
43
44 2.4 Recursively Separated Exposure based sub-image histogram equalization (RS-ESIHE)
Ac
45
46
47
Conceptually, RS-ESIHE is a recursive version of ESIHE, which performs recursive
48
49
50 decomposition of the histogram. ESIHE decomposes the input histogram only once based on
51
52 the exposure threshold while RS-ESIHE decomposes it recursively based on exposure
53
54
55 thresholds of individual sub histograms up to a recursion level r, generating sub-
56
57 histograms. The decomposed sub histograms are then equalized individually. For simplicity
58
59
60
61
62
63
64 Page 8 of 17
65
recursion level, r is taken as two. The RS-ESIHE method comprises of Exposure Threshold
t
12
ip
divided based on .
13
14
15
cr
16 (11)
17
18
us
19
20 (12)
21
22
23 2.4.2 Histogram Sub Division and Equalization
24
25
26
an
The original histogram is first bisected based on exposure threshold value as calculated in
27
M
28 (2). These individual sub Histograms are further decomposed into two smaller sub histograms
29
30
31 where the individual exposure threshold and as calculated in (6) and (7) acts as
32
ed
33 separating point of sub histograms. The Histogram Sub Division process results in four sub
34
35
36 images , , and ranging from gray level 0 to , to , to
pt
37
38 , to L-1. , , and are corresponding PDF of these sub
39
ce
40
41
images as defined in equations ( 13-16)
42
43
44
(13)
Ac
45
46
47
48
(14)
49
50
51
52
(15)
53
54
55
56
(16)
57
58
59
60
61
62
63
64 Page 9 of 17
65
and are total number of pixels in sub images , , and
t
12
ip
13 (19)
14
15
(20)
cr
16
17
18
The next step of RS-ESIHE is to equalize all the four sub histograms individually. The
us
19
20
21 transfer functions for Histogram Equalization based on equations (13-20) can be defined as
22
23
24
25
26
equations (21-24) an
27 (21)
M
28
29
30 (22)
31
32
ed
(23)
33
34
35 (24)
36
pt
37 and are the transfer functions used for equalizing the sub histograms
38
39
individually. The RS-ESIHE output image is produced by the combination of all four transfer
ce
40
41
42 functions.
43
44
Ac
1 respectively and divide the sub histograms into further sub histograms using and as
2
3
4 decomposing threshold, resulting in total four sub histograms.
5
6
7 Step 6: Apply the histogram equalization on individual sub histograms and combine the sub
8
9 images into one image for analysis.
10
11
t
12
ip
13
14
15
cr
16
17
18
us
19
20
21
22
23
24
25
26
an
27
M
28
29
30
31
32
ed
33
34
35
36
pt
37
38
39
ce
40
41
42
43
44
Ac
45 Fig.2. Results of Mosque image (a) Original, (b) RSIHE, (c) RMSHE, (d) QDHE, (e) RSWHE,
46
47
48 (f) BHEP-L, (g) ESIHE , (h) RESIHE and (i) RS-ESIHE
49
50
51 3. Experimental Results
52
53
54 In this section, the simulation results of proposed methods are compared with existing
55
56
histogram equalization based methods i.e. RMSHE, RSIHE, QDHE, RSWHE, BHEPL and
57
58
59 ESIHE. In order to analyze and compare the existing methods four low exposure test images:
60
61
62
63
64 Page 11 of 17
65
Fish1, Fish2 Mosque and Couple are taken. Both fish images are underwater sequence while
t
12
ip
13
14
15
cr
16
17
18
us
19
20
21
22
23
24
25
26
an
27
M
28
29
30
31
32
ed
33
34
35
36
Fig.3. Results of Fish2 image (a) Original, (b) RSIHE, (c) RMSHE, (d) QDHE, (e) RSWHE,
pt
37
38
39 (f) BHEP-L, (g) ESIHE , (h) RESIHE and (i) RS-ESIHE
ce
40
41
42 3.1 Performance Assessment based on Visual Quality
43
44
Ac
45 Images acquired in low light conditions including under water sequences are taken to test the
46
47
48 robustness of the proposed method for low exposure imaging.
49
50
51 The analysis of visual results from Figure 2-5 shows the effectiveness of recursive methods
52
53
54
especially on low light conditions. The concrete results in terms of contrast enhancement can
55
56 be clearly observed in Fig. 2 of Mosque image. RSIHE, RMSHE , RSWHE and BHEP-L
57
58 methods are not able to increase the exposure , however both RESIHE and RS-ESIHE has
59
60
61 improved the contrast and the objects are clearly visible. In both the underwater images i.e.
62
63
64 Page 12 of 17
65
Fish1 and Fish2 in Fig. 3 and Fig. 5 respectively the original images are acquired in very dim
t
12
ip
13
14
15
cr
16
17
18
us
19
20
21
22
23
24
25
26
an
27
M
28
29
30
31
32
ed
33
34
35
36
Fig.4. Results of Couple image (a) Original, (b) RSIHE, (c) RMSHE, (d) QDHE, (e) RSWHE,
pt
37
38
39 (f) BHEP-L, (g) ESIHE , (h) RESIHE and (i) RS-ESIHE
ce
40
41
42 The resultant Fish images of proposed methods have the objects clearly distinguishable. The
43
44
Ac
45 proposed algorithms enhance the overall image quality of couple image in Fig. 4. Low-
46
47 intensity regions in the background are properly exposed resulting clear vision.
48
49
50 3.2 Performance Assessment based on Average Information Content
51
52
53 To evaluate the performance of the proposed methods , Average information content is being
54
55
56 used as an image quality measure [21]. Average information content (entropy) is a measure
57
58 of richness of the details of the image and usually measured in units as bits. Eq. (25) defines
59
60
61 Entropy
62
63
64 Page 13 of 17
65
(25)
1
2 Where is probability density function of a given image at intensity level l and L is total
3
4
number of gray levels in the image. An image with higher entropy value have richness in
5
6
7 details and perceived to have better quality.
8
9
10
11
t
12
ip
13
14
15
cr
16
17
18
us
19
20
21
22
23
24
25
26
an
27
M
28
29
30
31
32
ed
33
34
35
36
pt
37
38
39
ce
40
41
42
43
44
Ac
45
Fig.5. Results of Fish1 image (a) Original, (b) RSIHE, (c) RMSHE, (d) QDHE, (e) RSWHE,
46
47 (f) BHEP-L, (g) ESIHE , (h) RESIHE and (i) RS-ESIHE
48
49 The entropy results of various methods for all the four images taken for experimentation
50
51
52 tabulated in Table 1. For optimum performance the entropy should be as close as possible to
53
54 the original image. The proposed new methods produces images with entropy very close to
55
56
57 the original one thus becomes well suited for bringing out information contents of the image.
58
59
60
61
62
63
64 Page 14 of 17
65
Table 1 Average Information Content Results
t
12
ip
13 Average 5.550 5.479 5.517 3.902 5.533 5.482
14
15
cr
16
17
18
4. Conclusion
us
19
20
21
22 Effective recursive histogram equalization techniques are proposed here for enhancement of
23
24
25
26
an
low exposure images. Decomposition of histogram based on exposure based thresholds and
27
individual sub histogram equalization provide very efficient results for low exposure imaging.
M
28
29 The methods can be very effective for contrast enhancement of images acquired in dim light
30
31 conditions i.e. under water or night vision images. Better results in terms of average
32
ed
33
34 information contents make these methods more suitable for bringing out information contents
35
36 of the images. The histogram clipping technique is also combined with histogram equalization
pt
37
38
39 to provide control on over enhancement that leads to natural enhancement. The Visual quality
ce
40
41 of resultant images of recursive methods shows the robustness of the method on low light
42
43
44 images.
Ac
45
46
47 References
48
49
50
[1] C. H. Lee, L.H. Chen, W. K. Wang , Image contrast enhancement using classified virtual
51
52 exposure image fusion, IEEE Trans. Consumer Electronics 58(2012),1253-1261.
53
54 [2] Y. Wang, S. Zhuo, D. Tao, J. Bu, N. Li , Automatic local exposure correction using bright
55
56
57 channel prior for under-exposed images, Signal Processing 93(2013), 3227-3238.
58
59 [3] R. C. Gonzalez and R. E. Woods, Digital Image Processing, Prentice Hall, 2002.
60
61
62
63
64 Page 15 of 17
65
[4] Y.T. Kim, Contrast Enhancement Using Brightness Preserving Bi-Histogram
t
12
ip
13 Histogram Equalization for Scalable Brightness Preservation, IEEE Trans. Consumer
14
15
cr
16 Electronics 49 (2003) 1301-1309.
17
18 [7] S. D. Chen and A. R Ramli, Minimum Mean Brightness Error Bi-Histogram Equalization
us
19
20
21 in Contrast Enhancement, IEEE Trans. Consumer Electronics 49 (2003) 1310-1319.
22
23 [8] K. S. Sim, C. P. Tso and Y. Y. Tan, Recursive Sub-Image Histogram Equalization
24
25
26
an
Applied to Gray Scale Images, Pattern Recognition Letters 28 (2007) 1209-1221.
27
M
28 [9] M. Kim and M. G. Chung, Recursively Separated and Weighted Histogram Equalization
29
30 for Brightness Preservation and Contrast Enhancement, IEEE Trans. Consumer
31
32
ed
37
38 Weighted Thresholded Histogram Equalization, IEEE Trans. Consumer Electronics 53
39
ce
40 (2007) 757-764.
41
42
43 [11] T. Kim and J. Paik, Adaptive Contrast Enhancement Using Gain-Controllable Clipped
44
Ac
t
12
ip
13 [16] C.H. Ooi and N. A. M. Isa, Quadrants Dynamic Histogram Equalization for Contrast
14
15
cr
16 Enhancement, IEEE Trans. Consumer Electronics 56 (2010) 2552 - 2559.
17
18 [17] C.H. Ooi and N. A. M. Isa, Adaptive Contrast Enhancement Methods with Brightness
us
19
20
21 Preserving, IEEE Trans. on Consumer Electronics 56 (2010) 2543 - 2551.
22
23 [18] Y. C. Chang and C. M. Chang, A Simple Histogram Modification Scheme for
24
25
26
an
Contrast Enhancement, IEEE Trans. Consumer Electronics 56 (2010) 737 - 742.
27
M
28 [19] T. L. Tan, K.S. Sim and C.P. Tso, Image enhancement using background brightness
29
30 preserving histogram equalization, Electronic Letters 48 (2012) 155 - 157.
31
32
ed
33 [20] K. Singh, R. Kapoor, Image enhancement using Exposure based Sub Image
34
35 Histogram Equalization, Pattern Recognition Letters 36 (2014) 10 – 14
36
pt
37
38 [21] S. D. Chen, A New Image Quality Measure for Assessment of Histogram
39
ce
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64 Page 17 of 17
65