IMAGE SEGMENTATION USING REPRESENTATIVENESS ANALYSIS AND REGION GROWING Jrgen Bhner1, Thomas Selige2 & Andre Ringeler1

Abteilung Physische Geographie Geographisches Institut der Georg-AugustUniversitt Gttingen Goldschmidtstr. 5 37077 Gttingen 2 Lehrstuhl fr Pflanzenernhrung Department fr Pflanzenwissenschaften der Technischen Universitt Mnchen Am Hochanger 2 85350 Freising Abstract: Image segmentation is a crucial task in the emerging field of object oriented image analysis. This paper contributes to the ongoing debate by presenting a segmentation procedure currently implemented in SAGA. Key feature at the core of the segmentation procedure is the representativeness analysis, performed for each pixel using geostatistical (semi-variogram) analysis measures. The representativeness layer supports conventional region growing algorithm with necessary start seeds, brake of criterions, and additional opportunities for fast performing initial image segmentation. The segmentation procedure aims to create spatially discrete object primitives and homogenous regions from remotely sensed images as the basic entities for further image classification procedures and thematic mapping applications. In a comprehensive evaluation study comparing eCognition, RHSEG and SAGA segmentation procedures, the SAGA approach was tested as robust and fast. SAGA performed at high quality a detailed segmentation of the actual landscape pattern represented by the remotely sensed imagery.

1 Introduction
In the last few decades, the fast spreading and increasing availability of vast amounts of remotely or proximal sensed data sets affected and fostered method developments in all fields of image processing and image analysis. Developing sensor standards and improved surface representations of satellite images seemed to meet perfectly societys increasing demands on (quantitative and qualitative) environmental information. However, compared to mans ability to abstract land use pattern, topographic features or topological elements from a spatial high resolution satellite image, intelligent technologies are hardly able to go beyond the metaphor of a remotely sensed landscape. Since early standards in spectral pattern recognition and particularly pixel based classification algorithms were found to be disadvantageous to inferring semantic information from images, more sophisticated image segmentation procedures and object-oriented image analysis approaches became an important and challenging main research foci in the entire area of image analysis.
GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115 29

In remote sensing, preliminary procedures for image segmentation have already been introduced in the mid 1970ies (NEUBERT & MEINEL 2003), however, it took more than two decades till segmentation algorithms were established as key functional features of contemporary software packages such as eCognition (BAATZ & SCHPE 2000). The design of segmentation procedures was likewise an important issue in the development of SAGAGIS. Preliminary approaches, which had been originally developed to enhance the SAGA terrain analyses options (BHNER et al. 1997; HOFFMANN & BHNER 2000) evolved to a suitable SAGA segmentation module with sufficient performance even on large image data sets, adaptable and applicable to many kinds of raster data types. In this paper we intend to briefly inform on the current state of implementation of the SAGA segmentation procedure, its methodical structure and its components involved. The final evaluation of the applicability and quality of the obtained segmentation results refers to the bachelor thesis of STOCK (2005), who compared segmentation procedures of eCognition, RHSEG and SAGA on the example of a panchromatic Quickbird image of Central Sulawesi (Indonesia), taken on 14th of April 2004.

2 Research Objectives and general Concept

A pixel is the smallest unit in the analysis of remote sensing imagery. The pixel size, however, is determined by external factors, i.e. the sensor specifications and thus does not necessarily correspond with the dimensions of the features to be studied. The generic idea of image segmentation is to firstly infer image primitives and homogenous regions, called image objects (or segments) as the basic entities for all further thematic classification and refinement applications. Since the digital image is no longer considered as a grid of pixels but as a set of image objects, the image segmentation step tackles some typical problems of pixel-based classification procedures (e.g. the mixed pixel problem) but likewise eases the integration of expert knowledge in thematic classification procedures and mapping applications. Moreover, readily inferable context information and topologic relations between the image objects offer extended image analysis options and thus enhance the diagnostic capacities of digital images. To date a series of image segmentation algorithms have been proposed, favoring either edge detection or region based approaches, the latter, however, are dominating in remote sensing (c.f. NEUBERT & MEINEL 2003; MEINEL & NEUBERT 2004; STOCK 2005). Since remote sensing applications often deal with huge raster data sets of varying complexity and resolution, the development of region based approaches particularly emphasized on a generic and computational efficient segmentation algorithm. One of the probably most prominent examples is the multi-scale image segmentation algorithm of eCognition, which is held to be robust and applicable under operational settings. Key features such as a user adaptable scale parameter and advanced homogeneity cost measures for object merging, which consider spectral and, particularly, spatial object properties (shape, smoothness, compactness), support qualitatively convincing image segmentation results (BAATZ & SCHPE 2000; MEINEL & NEUBERT 2004).

30

GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115

The relevance of object feature scales and spatial dimensions in object oriented image analysis is likewise emphasized in the SAGA segmentation approach. However, instead of using spatial object characteristics, we utilize the spatial representativeness as the basic integrative feature for initial image segmentation. The segmentation procedure consists of two major steps: 1. Representativeness analysis: the spatial representativeness quantifies the distance in which one or a set of spectral features of a pixel is representative for (i.e. spatially correlated with) the spectral properties in the neighborhood. 2. Region growing: region growing algorithms merge neighboring pixels to homogenous objects primitives. The algorithms starts on a limited number of single seed pixels, which are determined by its spatial representativeness. An iterative optimization of the initial segmentation results may optionally be achieved by object merging, using simple approaches such as global thresholding. More complex homogeneity cost measures (e.g. TZOTOS & ARGIALS 2006) or local mutual best fitting (BAATZ & SCHPE 2000) are further sophisticated alternatives. The entire procedure is intended to be used as a first (low level) processing part in an object oriented image analysis framework, focusing on thematic classification of land-cover or vegetation, respectively. Necessary clustering techniques implemented in SAGA, combine the iterated minimum-distance method and the hillclimbing method (RUBIN 1967). The first method is a fast cluster algorithm, which computes initial cluster centroids, based on least square estimations. Preliminary cluster results are subsequently forwarded to the hill-climbing algorithm, which continuously establishes new clusters, using a least square optimization scheme. One major advantage of the combination of both algorithms is the comparably fast handling of a great number of segments, which is particularly relevant on large image data sets. Moreover, the combination facilitates the generation of clusters which cover only a very small fraction of the whole image (BHNER et al. 1997). Examples of image segmentation and classification results, performed on LISSIRS-III multi-spectral images from costal areas in Northern Germany are given in DPEL et al. (2004). In the following, we solely concentrate on the formal definition and discussion of the actual segmentation steps.

3 Calculation of Spatial Representativeness

Spectral features of adjacent pixels are not accidental since they are determined by the land-cover and consequently there are specific underlying rules for the spatial distribution of these features. Representativeness analysis is based on the assumption that on a continuous surface, the change of spectral signals from one surface element (as displayed in a pixel) to neighboring elements depends on their distance. In simple words: spectral signals from neighboring pixels (e.g. situated in the center of an agricultural field with a uniform crop type and homogenous in terms of the phonological state) will more likely have comparable values than spectral signals from pixels further apart (e.g. spectral signals from a forest stand). A well established geostatistical approach for describing and analyzing the distance component in the spatial correlation of locally observed or measured features is the semi-variogram analysis. MATHERON (1963, 1973) introduced this
GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115 31

key analysis function in geostatistics as a method to objectively estimate the distance (i.e. the semi-variogram range) up to which point source observations are significantly (statistically) correlated. Moreover, the approximation of an empirical semi-variogram by a fitted semi-variogram function enables an optimized interpolation of irregularly spaced point source information into a regular spaced grid network. In this study, we instead apply semi-variograms to receive information about the spatial representativeness of a known pixel feature. Based on the empirical semi-variogram function vi, computed for each pixel separately, an average semi-variance gradient is calculated as the inverse distance weighted mean gv of all semi-variance gradients in the distances d1 to dm with the weights 1/d1 to 1/d1 each powered by p [01]. In order to enhance the computational efficiency and performance, the image resolution in the circle around the targeted pixel decreases with increasing radius di. The metric representativeness rc of a pixels feature in the spectral channel c is subsequently computed as the distance in which 50 % of the global (image) variance vm is reached [02]. [01]

gv =

d d
vi
i =1 m

i i

d1
i =1 i

[02]

rc =

0.5vm gv

The measure of representativeness can thus be simply described as the mean distance, in which the spectral feature of a pixel is largely similar to the surrounding pixels. Two examples of empirical semi-variogram functions are given in Figure 1. The first is obtained from a central representative position in a lake and the second is obtained from a setting near the lake front. Both positions are marked in the Landsat-7 ETM image from Northern Thuringia. The final result of the entire procedure is a representativeness matrix for the whole image, i.e. a representativeness value for each pixel. As shown in the analysis results of the Landsat image in Figure 2, minima in the matrix of representativeness are often arranged in pixel-lineaments, mostly indicating land use borders, whilst representativeness maxima typically occur in central parts of huge homogenous agricultural fields. The analyses where directly performed on the digital numbers of the Landsat-7 ETM subset (see Fig.1). As far as different spectral channels are separately analyzed, the representativeness layers have to be averaged for further use in region growing initialization. In this case, a geometric mean is suggested as a suitable standard measure in order to ensure an equally weighted integration of all representativeness layers. However, if specific pattern are better represented in certain channels, a weighted mean may be a reasonable alternative. Moreover, the representativeness analysis may likewise be consistently performed on texture layers, particularly if the spectral variance (i.e. the spectral noise) covers characteristic surface pattern (e.g. the spectral noise of natural vegetation) in order to extend the predictive capacity of the image.

32

GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115

Fig.1: Subset of Landsat-7 ETM (Channel 4, Path 195 / Row 24) taken on 4th of September 1999 (above) and selected semi-variogram examples (the empirical semivariogram function is standardized by the maximum variance; x-axis distances are given in percent of the global distance)

GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115

33

4 Region Growing
Region growing algorithms are clustering techniques which merge neighboring pixels, starting on an infinite set of single seed points. The definition of suitable seed points however, is a crucial task which distinctly affects the quality of all subsequently performed segmentation steps. The representativeness analysis tackles this problem in an elegant manner, given that the local maxima of the representativeness already indicate typical (i.e. spatially representative) positions within an image object. However, in cases with strongly irregular textured surfaces (e.g. the noisy pattern of a natural mixed forest stand), the representativeness analysis tends to yield many seed points in short distances. An overestimation of seed points likewise occurs in cases of extremely irregular shaped image objects, even if the spectral features are absolutely homogeneous (e.g. lakes or meandering rivers). To overcome this disadvantage, a SAGA image filter algorithm may optionally be applied to the representativeness layer, which calculates an inverse distance weighted mean for each pixel in a moving circle. The radius sm of the moving circle is simply computed as a function of the representativeness itself, by:
s m = d g ( rc d g ) p

[03]

where dg is the grid size and rc the representativeness. The potency p in equation [03] controls the intensity of the smoothing effect, however, a potency of 0.5 (= square root) proved suitable in most cases (see Fig.3). Region growing and object merging algorithms are commonly performed on the spectral values of an image and thus require a proper definition for image object homogeneity or heterogeneity, respectively. The definition of a homogeneity criterion is a precondition to decide which neighboring pixels (or image objects) merge at each region growing step. Alternative decision heuristics are defined and discussed in BAATZ & SCHPE (2000). In SAGA, two alternative region growing algorithms are implemented. The rather conventional complete linkage algorithm considers the spectral signatures of one or a set of spectral channels. The implementation of the region growing algorithm consists of the following steps. Initialises an array with the position of all positions of the cluster-centroids (seeds) and their property space For each cluster-centroid build a list of border points. A border point can be an element of multiple cluster Calculate and store for any border point the maximum Euclidean attribute distance to its cluster-centroid in the property space. Store its cluster membership and its spatial position. Calculate and store for every cluster the n-dimensional border of the property space. While the list of border points is not empty repeat: (1) find the border point and the cluster with the minimum maximal Euclidean distance in the property space for each cluster, (2) add this border point to this cluster and update the border of the property space of this cluster, (3) delete this border point from the border point list and add the new border points of the grown cluster, (4) for every new border points calculate its cluster membership, its attribute distance to the cluster-centroid and its spatial position.
34 GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115

Fig.2: Spatial representativeness of the Landsat-7 ETM image (see Fig.1) values range from < 30 m (black) to > 400 m (white) The box indicates the subset of Figure 3

Fig.3: Starting seed points and segmentation results of the complete linkage algorithm the image subset is marked in Figure 2
GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115 35

The second fast merging algorithm is predominantly performed on the representativeness layer itself. A pixel merges with an adjacent image object (i.e. a seed pixel or a merged cluster of pixels), if its representativeness is the local maximum of all other neighboring free pixels. Instead of growing only by one best fitting pixel in each pass, the object growing process enables to assimilate a set of neighboring pixels within one pass and is thus of fast performance. The break of criterion is reached, if the remaining free pixels all are bordering to concurring image objects. These contentious pixels are finally assimilated by the best fitting neighboring image object, considering again the best fit criterion already defined in function [04]. As long as only one spectral channel is analyzed, both region growing procedures yield comparable results, thus the fast merging algorithm is favorable owing to its performance. However, the central task of image segmentation as an initial step in object oriented image analysis is particularly to infer image semantics. As long as important semantic information of an image are better covered by different spectral channels, the complete linkage algorithm is clearly advantageous. An example of segmentation results, obtained with the complete linkage algorithm is given in Figure 3.

5 Discussion and Conclusion

The SAGA segmentation approach is a new and modular concept combining represenativeness analysis steps to identify the initial seeds and two alternative following region growing procedures. The procedures are adaptable to any kind and combination of raster data types. The implementation is tested as robust and has produced excellent performance even on large image data sets. There is no limitation on data size other than given by the used computer configuration. STOCK (2005) compared the SAGA segmentation approach, the eCognition software and the RHSEG procedure. The result of the evaluation test is given in Table 1. SAGA identified all areas with maximum represenativeness as seeds and resulted in a clear separation of e.g. field boundaries. The ecognition approach resulted in a less segmented image compared to the RHSEG and SAGA procedure. Tab.1: Evaluation results of the comparison of eCognition, RHSEG and SAGA by STOCK (2005) Handling Segmentation Performance Rating eCognition ++ ++ +++ ++ RHSEG + ++ ++ + SAGA +++ +++ +++ +++ The SAGA procedure to reduce the seed numbers preserved the relevant locations and still resulted in a detailed segment pattern. In regions where raster cell size is close to the image object size all procedures have their limitation. The represenativeness analysis gives SAGA the opportunity also to calculate the minima of representtativeness to enable even single object identification in small textured areas of a raster data set. Further studies will focus on this aspect to better segment textured areas.
36 GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115

Fig.4: Result from the eCognition multi-scale image segmentation algorithm (STOCK 2005)

Fig.5: SAGA segmentation result performed with the complete linkage algorithm (STOCK 2005)

GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115

37

References
BAATZ, M. & SCHPE, A. (2000): Multiresolution Segmentation: an optimization approach for high quality multiscale image segmentation. In: STROBL/BLASCHKE/GRIESEBNER [Hrsg.]: Beitrge zum AGIT-Symposium Salzburg 2000. Angewandte Geographische Informationsverarbeitung XII: pp. 12-23. BHNER, J., KOETHE, R. & TRACHINOW, C. (1997): Weiterentwicklung der automatischen Reliefanalyse auf der Basis von Digitalen Gelndemodellen. Gttinger Geographische Abhandlungen 100: 3-21. DPEL, U., GEROLD, G., BHNER, J. & KICKNER, S. (2004): Handlungsempfehlungen zur effizienten, umweltvertrglichen Planung von Windenergieanlagen. Leitfaden und Ratgeber fr Lnder, Regionen, Landkreise, Stdte, Gemeinden, Planungsbros und Projektentwickler. dpel landschaftsplanung; Geogr. Inst. Univ. Gttingen. HOFFMANN, H. & BHNER, J. (2000): Spatial pattern recognition by means of representativeness measures. Proc. IGARSS'99, Hamburg, Germany. Vol. I: 110-113. MATHERON, G. (1963): Principles of Geostatistics. - Economy Geology, Vol. 58: 1246-1266. MATHERON, G. (1973): The Intrinsic Random Functions and their Applications. - Advances in Appl. Prob. , Vol. 5: 438-468. MEINEL, G. & NEUBERT, M. (2004): A Comparison of segmentation programs for high resolution remote sensing data. International Archives of the ISPRS, Volume XXXV, Part B, Commision 4: 1097-1105. NEUBERT, M. & MEINEL, G. (2003): Vergleich von Segmentierungsprogrammen fr Fernerkundungsdaten. In: STROBL/BLASCHKE/GRIESEBNER [Hrsg.]: Beitrge zum AGITSymposium Salzburg 2003. Angewandte Geographische Informationsverarbeitung XV: pp. 12-23. RUBIN, J. (1967): Optimal classification into groups: an approach for solving the taxonomy problem. Theoretical Biology 15: 103-144. TZOTOS, A. & ARGIALS, D. (2006): MSEG: A generic region-based multi-scale image segmentation algorithm for remote sensing imagery. ASPRS 2006 Annual Conference (May 1-5; 2006) Reno, Nevada. STOCK, I. (2005): Vergleich von Segmentierungsprogrammen fr hochauflsende Satellitendaten am Beispiel einer Quickbird-Aufnahme von Zentral-Sulawesi, Indonesien. Bachelor Arbeit im Studiengang fr Informatik (ISSN 1612-6793 Nummer ZFI-BM2005-12), Gttingen, 64 pp.

38

GTTINGER GEOGRAPHISCHE ABHANDLUNGEN VOL. 115