Forschungsberichte aus der

Industriellen Informationstechnik

Limeng Wang

Image Analysis and Evaluation of

Cylinder Bore Surfaces in Micrographs

Limeng Wang
Image Analysis and Evaluation of Cylinder Bore Surfaces
in Micrographs

Forschungsberichte aus der Industriellen Informationstechnik

Band 9
Institut fr Industrielle Informationstechnik
Karlsruher Institut fr Technologie
Hrsg. Prof. Dr.-Ing. Fernando Puente Len

Prof. Dr.-Ing. habil. Klaus Dostert

Eine bersicht ber alle bisher in dieser Schriftenreihe erschienenen Bnde

finden Sie am Ende des Buchs.

Image Analysis and Evaluation of

Cylinder Bore Surfaces in Micrographs
by
Limeng Wang

Dissertation, Karlsruher Institut fr Technologie (KIT)

Fakultt fr Elektrotechnik und Informationstechnik
Tag der mndlichen Prfung: 24. September 2013
Referenten: Prof. Dr.-Ing. F. Puente Len, Prof. Dr.-Ing. T. Lngle

Impressum

Karlsruher Institut fr Technologie (KIT)

KIT Scientific Publishing
Strae am Forum 2
D-76131 Karlsruhe
KIT Scientific Publishing is a registered trademark of Karlsruhe
Institute of Technology. Reprint using the book cover is not allowed.
www.ksp.kit.edu

This document excluding the cover is licensed under the

Creative Commons Attribution-Share Alike 3.0 DE License
(CC BY-SA 3.0 DE): http://creativecommons.org/licenses/by-sa/3.0/de/
The cover page is licensed under the Creative Commons
Attribution-No Derivatives 3.0 DE License (CC BY-ND 3.0 DE):
http://creativecommons.org/licenses/by-nd/3.0/de/
Print on Demand 2014
ISSN 2190-6629
ISBN 978-3-7315-0239-5
DOI 10.5445/KSP/1000041878

Image Analysis and Evaluation of

Cylinder Bore Surfaces in Micrographs
Zur Erlangung des akademischen Grades eines

DOKTOR-INGENIEURS

von der Fakultat f

ur
Elektrotechnik und Informationstechnik
des Karlsruher Instituts f
ur Technologie (KIT)
genehmigte

DISSERTATION

von
M.Sc. Limeng Wang
geb. in Luoyang, China

Tag der m
undl. Pr
ufung:

24.09.2013

Hauptreferent:

Prof. Dr.-Ing. F. Puente Le

on, KIT

Korreferent:

Prof. Dr.-Ing. T. L
angle, Fraunhofer IOSB

Acknowledgement
The completion of my dissertation and subsequent Dr.-Ing. has been a
long journey. During this process I have experienced confusion, hesitation
and depression. The success of my doctoral study would not be achieved
without the support, the help and the encouragement of the people by my
side. Except the extended knowledge I also acquired the valuable life experience during the stay at the institute, and in Germany. That will greatly
benet my future life. Therefore, I would like to gratefully and sincerely
express my gratitude.
First of all, I extremely appreciate my mentor, Prof. Fernando Puente
He provided me this research thesis, when I was still in China. His
Leon.
valuable guidance and scholarly inputs opened my mind and directed me
to the correct road at various phases of this research. He has always made
himself available to clarify my doubts despite his busy schedules. I thank
him for the unconditional support and the facilities provided to carry out
the research work at the institute. Special thanks also to Prof. Thomas
Langle, who was willing to review my dissertation as the co-referent. His
concise comments has great help for the revising of this paper.
Everything is hard at the beginning. It is true not only for the research
but also for the integration into the new environment. Luckily, I received
generous kind help from the secretary and other working colleagues at
the institute. My gratitude is hereby extended to Mrs. Kofer. Since my
enrollment at the faculty and up until my doctoral examination, she has
always contributed to the organization issues for me. I owe also a very
important debt to Dr. Konrad Christ, Dr. Andreas Sandmair, Matthias
Michelsburg, Mario Lietz and others, who did great effort for the organization of institute events. In particular, I acknowledge Dr. Wenqing Liu,
for being the discussant for the feeling and experience during the doctoral study. I am very grateful for the convenient working atmosphere at
the institute.
I would like to offer my special thanks to Deutsche Akademische Austauschdienst (DAAD) for the scholarship, which was used to nance my
living cost during the doctoral study. I owe a lot to Prof. Klaus Dostert
for his recommendation to prolong the scholarship every year. I will also

remember the life at Chinesisch-deusches Hochschulkolleg (CDHK) of

the Tongji-University in Shanghai, where I gained the master degree as

well as the chance of the doctoral study in Germany. Prof. Elmar Schrufer
of the Technical University of Munich is a warm-hearted tutor for each
overseas student of CDHK. Without his encouragement, this dissertation
would not have materialized.
Of course, no acknowledgement would be complete without giving
thanks to my parents. They have taught me about hard work and selfrespect, about persistence and about how to be independent. I must give
the tremendous and deep appreciation to my wife, Qingqing. Through
her love, patience, support and unwavering belief in me, I have been able
to complete this dissertation journey.

Lenting, in May 2014

Limeng Wang

Contents
1 Introduction
1.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.2 Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . .
1.3 Thesis organization . . . . . . . . . . . . . . . . . . . . . . .

1
1
3
5

2 Honed surfaces
2.1 Cylinder manufacturing . . . . . . . . . . . . . . . . . . . .
2.2 Surface function . . . . . . . . . . . . . . . . . . . . . . . . .

7
7
9

3 Image acquisition
3.1 Light optical microscopy . . . . . . . . . . . . . . . . . . . .
3.2 Scanning electron microscopy . . . . . . . . . . . . . . . . .

13
13
15

4 Defect inspection
4.1 Previous work . . . . . . . . . . . . . . . . . . . .
4.1.1 Approach using signal processing . . . .
4.1.2 Approach based on surface measurement
4.2 Local orientation analysis . . . . . . . . . . . . .
4.2.1 Filter-bank based methods . . . . . . . .
4.2.2 Gradient-based methods . . . . . . . . . .
4.3 Edge-aware structure tensor . . . . . . . . . . . .
4.3.1 Design idea . . . . . . . . . . . . . . . . .
4.3.2 Bilateral lter . . . . . . . . . . . . . . . .
4.3.3 Adaptive lter kernels . . . . . . . . . . .
4.3.4 Tensor ltering . . . . . . . . . . . . . . .
4.3.5 Parameter selection . . . . . . . . . . . . .
4.4 Detection scheme . . . . . . . . . . . . . . . . . .
4.4.1 Feature extraction . . . . . . . . . . . . . .
4.4.2 Segmentation . . . . . . . . . . . . . . . .
4.5 Experimental results . . . . . . . . . . . . . . . .
4.5.1 Range lters . . . . . . . . . . . . . . . . .
4.5.2 Amplitude and orientation elds . . . . .
4.5.3 Applications . . . . . . . . . . . . . . . . .

19
20
20
22
24
24
27
33
33
34
35
37
37
43
43
45
45
45
48
57

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.

Contents

5 Graphite detection
5.1 Related work . . . . . . . . . . . . . . . .
5.2 Morphological scale spaces . . . . . . .
5.2.1 Notation . . . . . . . . . . . . . .
5.2.2 Scale-space behavior . . . . . . .
5.2.3 Morphological stability . . . . .
5.2.4 Application to shape description
5.3 Detection scheme . . . . . . . . . . . . .
5.3.1 Feature extraction . . . . . . . . .
5.3.2 Segmentation . . . . . . . . . . .
5.4 Experimental results . . . . . . . . . . .

.
.
.
.
.
.
.
.
.
.

65
65
66
67
68
70
72
73
74
78
78

6 Surface evaluation
6.1 Defect severity . . . . . . . . . . . . . . . . . . . . . . . . . .
6.2 Graphite distribution . . . . . . . . . . . . . . . . . . . . . .

81
82
84

7 Conclusions

87

A Results
A.1 Defect inspection . . . . . . . . . . . . . . . . . . . . . . . .
A.2 Graphite detection . . . . . . . . . . . . . . . . . . . . . . .

91
91
95

B Path openings and closings

97

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

.
.
.
.
.
.
.
.
.
.

C Miscellaneous
99
C.1 Roundness . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99
C.2 Local adaptive thresholds . . . . . . . . . . . . . . . . . . . 100
D Nomenclature

101

Bibliography

109

ii

1 Introduction
Internal combustion engines are the heart of modern automotive vehicles. The cylinder crankcase, as the core component of the engine, provides an enclosed space for generating the vehicle power. Cylinder liners
are like the ventricles, in which pistons send rhythmic heartbeats to drive
the engine. For a traveling car the rotation speed of a motor is normally
more than 1000 r/min . This means that the friction occurs quite intensively between the piston and the cylinder wall. Surface structures of the
cylinder wall have important inuence on friction losses, oil consumption
and piston wear. These factors further inuence engine performances like
energy efciency, noxious emission and service longevity. For these reasons the quality of cylinder bore surfaces is of great interest in the eld of
engine production. Inspection techniques based on image analysis have
been commonly used for the quality control of engine cylinders. Inspired
by the increasing demand on inspection robustness and precision, this
thesis focuses on novel inspection solutions for cylinder bore surfaces that
possess a wide range of qualities.

1.1 Background
Surface textures on cylinder bores play an essential role for reducing friction and oil consumption. Previous motor tests manifest that tribological
processes in the cylinder-piston system, as shown in Fig. 1.1, are very
complex. Several different theories [13, 64, 66] have been presented to
characterize the relations between the surface topology and engine performances. Based on these theories, a variety of manufacturing methods
were developed to nish function-relevant cylinder liner surfaces. The
commonly used production approach is the honing technique, because
it can signicantly improve the geometrical accuracy of cylinder bores
and create specially designed surface structures. Engines nished by honing are desired to fulll the increasing legislative requirement for harmful emission, as well as to satisfy customers demands on environmentfriendly and energy-efcient automobiles. For these purposes, image-

1 Introduction

Figure 1.1 Cylinderpiston system in automotive engines.

based surface inspection is widely used in the research of manufacturing

techniques as well as in the quality assurance for mass production.
The evaluation of new honing techniques is normally performed ofine
in laboratories. It has been known that triboogical properties of honed
surfaces can be improved by optimazing the cylinder material, the honing tool or the honing process. Honed cylinder liners are carried into the
laboratory. Then, quality features describing textural and topological surface structures are measured in a manual or an automated fashion. In
this process, it is veried whether honed surfaces conform to the design
specications. The quality control for the mass production requires the
same measurement of surface features as that conducted in ofine applications. Surface qualities are evaluated according to industrial standards
that are formulated by car manufacturers. However, not all inspection
tasks are online realisable. A part of inspection tasks concern material
details, such as metal folds and graphite grains. This requires highly resolved surface data. Some 2D and 3D imaging systems are commonly
used by car manufacturers. For example, light optical microscopy and
scanning electron microscopy are adopted to investigate 2D texture features and material structures. Due to the complex internal structure of
microscopic systems, it is still difcult to integrate them in machines. Be-

1.2 Motivation

sides, white light interferometers and the confocal microscopy are useful measuring instruments for roughness measurement. With 3D imaging
systems, honed surfaces are described by so-called 2.5D images which encode surface heights. Image analysis methods can thus be used to analyze
surface textures as well as surface roughness. Since 3D imaging systems
are very sensitive to the mechanical vibration, currently they are not suitable for online applicaitons. Nevertheless, several researches [36, 52] were
conducted to design non-distructive 3D inspection systems. These systems still cannot be integrated in production lines. Besides, 2.5D images
are acquired by scanning the surface topology. Thus, the inspection process is time-consuming. To facilitate the acquisition of surface data, small
surface specimens are extracted at different places of cylinder walls by
splitting cylinder liners. With the aforementioned imaging systems, surface details with a size of micrometers can be clearly revealed in images.
As explained before, such a high image resolution is still diffclut to realise
in online inspection systems. Other inspection tasks concern the overview
evaluation of machining patterns (also known as surface nishes), such
as the measurement of honing angles, the balance of honing grooves as
well as the recognition of blowholes, bubbles and scratches with a size of
millimeters. These quality problems can be investigated in macro-scales.
Macroscopic imaging systems are already good enough for these tasks.
This enables fully automated 100%-inspection of cylinder bore surfaces.
Some compact 2D imaging systems [28, 45, 48] insert images sensors into
cylinder bores and scan cylinder walls around 360 . With these systems,
images showing complete cylinder walls can be directly acquired in cylinder bores. The inspection process is fast and non-distructive. These systems are suitable for both on- and ofine appliactions. Compared with
the conventional visual inspection, results generated by image analysis
are reliable and reproducible. Hence, automated vision systems are favored in the automobile industry. In the meantime higher requirements
are put forward to the performance of inspection systems.

1.2 Motivation
This thesis develops image analysis algorithms for standard 2D microscopic systems. The highlights of the work lie in the algorithm design
for two challenging inspection tasks the detection of metal folds and
graphite grains. By assuming that the micrographs of honed surfaces

1 Introduction

show a good image quality, the research aims to overcome the difculties for analyzing complex surface details.
In the eld of automated inspection of honed surfaces, the research interest lies in three apsects image acquisition, image processing algorithm, and quality parameters. This thesis contributes to the last two aspects. The work does not actually improve imaging systems. However,
the choice of proper methods for image acquisition builds the fundation
of the work. The tendency in image acquisition techniques is to nondestructively capture the surface data directly in the cylinder bores. As
long as surface images can faithfully reect the surface topography, the
proposed image processing algorithms can be commonly applicable to
both destructive and non-destructive imaging systems. In this thesis, a
destructive way to capture surface images is adopted. Surface samples are
prepared by splitting cylinder bores. Then, two standard approaches for
image acquisition are exploited. Pictures captured by a light optical microscope (LOM) and a scanning electron microscope (SEM) are adopted
for investigating the surface appearance at different scales. The choice of
imaging systems hat a reason that surface components inuence engine
properties in their own roughness scales. An inspection method that handles honed surfaces at a certain observation scale may lower the accuracy
of describing relevant objects at other scales. Hence, a more rational strategy for surface description is to match the image acquisition method with
the scales of surface components.
This thesis focus on the further development of inspection methods for
2D surface nishes. Two novel algorithms for defect inspection in SEM
images and graphite detection in LOM images are proposed. In the rst
algorithm a local approach for recognizing surface defects overcomes the
drawbacks of conventional model-based methods. It can be noted that
tool marks imaged in micrographs may be far from the ideal model of
honing textures. The image analysis for these surface objects is more difcult than that for regular honing patterns. In previous works [2, 14, 80],
honing grooves were assumed to dominate the surface. Thereby, texture
features could be extracted from the entire image. Defects were detected
as the abnormal of honing textures. However, in the worst case, expected
honing textures may be seriously degraded in the surface nish due to
an improper manufacturing process. The model-based analysis of honed
surfaces may lead to instable evaluation because of poor surface qualities.
Therefore, more effective and robust surface features are desired to deal
with a variety of honing structures. The presented approach for defect in-

1.3 Thesis organization

spection is edge-based. Edge features like the orientation and the strength
are used for constructing feature maps. To ensure the robustness of feature extraction, a novel orientation estimator is designed. The presented
inspection strategy shows some signicant advantages:

p Independent of the formation of honing textures,

p Applicable to a wide range of surface qualities and
p Potential to be extended to 2.5D applications.
The second algorithm concerns the investigation of laser-exposed cylinder bore surfaces. The produced surfaces consist of randomly scattered
pores which provide the surface functionality similar to conventional
honing grooves. These pores are formed by uncovered graphite grains.
Currently, the automated quality evaluation based on 2D images is still
lacked for such surfaces. In this thesis, a morphological feature for describing a set of surface components is proposed. On this basis, a scheme
for segmenting graphite grains is designed. This approach enables robust
detection of graphite grains even when inhomogeneous illumination and
textual disturbances exist.
Moreover, the proposed methods can assist the assessment of surface
qualities. The detection results are utilized for constructing quality parameters. Nowadays, several quality parameters [2, 14, 80] for honed surfaces have been implemented for the quality control. Another two parameters are proposed for characterizing the defect severity and the uniformity of graphite distribution, in which engine producers are increasingly
interested. Defects and graphite gains can be automatically evaluated in
use of the novel algorithms and quality parameters.

1.3 Thesis organization

This work is to gain insight into detection-based surface evaluation,
which concerns exactly localizing honing defects as well as segmenting
uncovered graphite grains. These topics originally stem from quantitative
metallography [22, 61], which studies the physical structure and components of metals. Physical features of the material, the mechanical process
and the imaging mechanism play essential roles for the surface structures
shown in images. In order to understand the inspection tasks correctly, it

1 Introduction

is necessary to study in depth the surface functions and manufacturing

processes. Chapter 2 will explicate these issues in detail.
In Chapter 3 the acquisition methods of micrographs are introduced.
Image qualities are tuned to the satisfactory status for displaying relevant surface components in surface nish. Surface structures can be visually recognized based on their shading. The illustrated pictures were
captured in motor factories. Some exemplars of inspection objects have
been identied by experts, which helped the understanding of inspection tasks. The algorithms proposed in this thesis will detect and evaluate
these objects automatically.
Chapter 4 presents a modication of the structure tensor for analyzing
local orientations. Furthermore, this method for the orientation analysis
is applied to defect detection on plateau-honed surfaces. Surface defects
are to be localized in SEM images that are well congured at a high magnication. Since surface components are spatially occluded, it is enabled
to regionally segment defects and honing grooves. Technically, featurebased detection scheme is developed to reveal defective locations in SEM
images. The weighted orientation dispersion is taken as the signature of
defective edges. The algorithm for orientation estimation combines the
classic gradient-based method with an edge-preserving lter. The experiments will show that the detection scheme features both an improved
robustness and accuracy.
In Chapter 5 a multiscale morphological feature for segmenting uncovered graphite grains is developed for inspecting laser-exposed cylinder
liners. LOM images with a low magnication are appropriate for studying the distribution of graphite grains. On machined surfaces, a nonuniform illumination and varying background textures may have a negative impact on the stability of graphite detection. Conventional methods
relying on image intensities are unable to yield correct segmentations.
The work begins with the study of morphological shapes of the image
topography. It can be observed that the interpretation of intensity peaks
and valleys is related to observation scales. Inspired by these ndings,
surface components at different roughness levels can be assigned to foreand background images. Graphite grains are accurately detected by postprocessing of the binarized foreground map.
Chapter 6 proposes quality parameters describing the defect severity
and the distribution of graphite grains. Detected surface components are
statistically analyzed in this chapter. By testing a series of surface samples,
quantitative evaluations are compared with the visual impression.

2 Honed surfaces
2.1 Cylinder manufacturing
The material and the manufacturing method are two critical factors affecting cylinder qualities. Since cylinder bodies must be able to withstand
the internal high-temperature and high-pressure environment, the cylinder material is required to possess enough hardness and toughness. Additionally, uncovered material ingredients are incorporated into surface
textures. The commonly used cylinder materials [65] include cast iron
with lamellar graphite (GJL), cast iron with vermicular graphite (GJV)
and aluminum-silicon alloy. GJL is a traditional material for engine cylinders. Recently, GJV is considered to be an alternative material due to its
excellent mechanical and physical properties. However, the production
cost of GJV is higher than that of GJL. In contrast to cast iron, aluminum
alloy gradually becomes popular, since it can considerably reduce the engine weight. For the sack of the inadequate cylinder strength, the application of the aluminum material is still conned to low power engines. Due
to the respective merits and drawbacks of these materials, they co-exist
now with each other in the engine world.
Honing [35, 6769] is an abrasive machining method, typically applied
to cylinder nishing. Honing tools are equipped with honing stones that
feature an abrasive material, such as silicon carbide or diamond. As illustrated in Fig. 2.1, the honing process consists of rotating and sliding the
honing tool. The velocity of honing stones can be decomposed into the
tangential and axial components. By adjusting these two speeds, it is convenient to designate the honing angle according to the following formula
[21]:
= 2 tan1

v2
.
v1

(2.1)

In this way, honed surfaces are structured with two sets of parallel
grooves. Such a texture is technically called cross hatch. Modern honing processes can achieve highly precise bore dimensions through multiple stages. The grain size of honing stones should be carefully chosen

2 Honed surfaces

Figure 2.1 Honing process.

in each honing phase. The smaller crystal grains honing stones have, the
ner grooves can be machined, that is, the smoother bore surfaces can be
produced. Therefore, the shape and the roughness of cylinder bores have
a close relation with honing stones.
For cylinders made of cast iron, honing with coarse abrasives has two
effects: honing stones expose graphite components by cutting off the material close to the surface; in the meantime they smear burrs into metal
akes along cutting edges. In consequence, opened graphite particles are
enclosed again by the folded metal. To improve surface roughness the
metal folds appearing as sharp peaks are required to be removed by ne
honing. By sophisticated control of machine parameters coarse tool marks
cannot be worn off thoroughly. In this case, grooves of different scales are
overlapped in the surface nish. Since deep grooves partition the surface
into diamond-shaped at areas, this method is technically referred to as
plateau-honing [18]. Alternatively, if merely the nest honing grooves
are preserved, the method corresponds to peak-honing [18]. The recent
advances in honing technology are made by integrating laser treatments,

2.2 Surface function

(a)

(b)

(c)

(d)

Figure 2.2 Honing textures (courtesy of [68]). (a) Plateau-honed. (b) Laserexposed. (c) Laser-honed. (d) MMC-casted.

for instance, deliberately engraving regular pocket structures on plateauhoned surfaces [35] or by laser-exposure of peak-honed surfaces aiming
to melt a layer of metal folds and consequently open graphite grains [68].
For aluminum cylinders, honing is used to expose metal matrix composites (MMC) [13]. Textures obtained in aluminum cylinder liners are completely different from those produced with cast iron. In terms of aforementioned honing methods four types of honing textures are available
in practice: plateau-honed, laser-exposed, laser-honed and MMC-casted.
See also Fig. 2.2. This thesis will focus on the rst two classes of honing
textures. The tribological sense behind these textures will be explained in
the rest of this chapter.

2.2 Surface function

During engine operation, the fuel energy is converted into the kinetic energy of pistons. Due to the friction between the cylinder wall and the
piston ring, energy losses are inevitable. An effective means to reduce
friction is to make use of a lubrication mechanism to isolate contact surfaces. The formation of the lubricating layer, that is, an oil lm, is a hydrodynamic result. Motor oil arrives at bore surfaces via the channel inside the piston. With the movement of the piston, the whole cylinder wall
can be lubricated by the oil. The thickness and the distribution of the oil
lm highly depend on the structure of the cylinder bore surfaces. At the
roughness scale, valleys in the surface prole serve as oil-reservoirs, from
which the piston ring obtains sufcient lubricant for pulling out a uid
lm. In addition to the inuence on lubrication, bore surfaces are also
tightly related to oil consumption and noxious emission. When the pis-

2 Honed surfaces

Figure 2.3 Tribological systems (courtesy of [35]).

ton ring runs back to the bottom of the cylinder liner, motor oil should
be scraped into the crankcase oil tray for the repeated use. Otherwise the
remaining oil will be combusted together with the fuel. In this case, CO2
emission will increase dramatically. The experts experience tells that oil
consumption is mainly caused by oil leakage [18]. Peaks on rough bore
surfaces do not only bring about strong piston wear, but also damage
cylinder walls by leaving axial grooved traces. Thereby, lubricant may
enter the combustion chamber along large gaps. For these reasons the desired surface structures of cylinder bores should be featured with small
roughness on the part contacting the piston while also guarantee the adequate oil reserve in valleys.
Modern honing technologies allow producing functionally designated
textures. For example, on plateau-honed surfaces lubricant is dispersed in
honing grooves, which constitute a channel system [35] communicating at
intersection points (Fig. 2.3 left panel). The design of the honing angle, ,
depends on the consideration that motor oil should be evenly distributed
in the tangential and axial directions. The latest techniques illustrate that

10

2.2 Surface function

a at honing angle ( < 30 ) leads to less oil distribution in the axial direction, and thus the piston tends to be hard-going on the running surface.
With an increased honing angle ( > 90 ), lubricant dominates in the axial direction. In this case too much oil goes into the combustion chamber.
To obtain the optimum lubricating effect the honing angle should be chosen in the interval between 30 and 90 . The honing technique could be
combined with laser exposure. The right panel of Fig. 2.3 shows that the
micro-pressure chamber system [35] substitutes the channel system to retain motor oil. Vermicular graphite is opened by melting metal folds for
the purpose of producing porous structures. As motor oil resides in pores,
the piston seems to be swimming on the surface. Such a structure overcomes the major drawback of the communicating channel system, that is,
the friction is increased again after the piston expels oil to the surrounding. At this point it becomes clear why the distribution of opened graphite
particles is crucial for surface lubrication. Keeping these surface functions
in mind, novel solutions for the inspection of some challenging surfaces
are explored in the following parts of this thesis.

11

3 Image acquisition
In the industrial environment, image acquisition of honed surfaces follows internationally standardized metallographic approaches [61, 75].
Light optical microscope (LOM) and scanning electron microscope (SEM)
are the major tools used for qualitative and quantitative analysis of honed
surfaces. In this thesis surface data are collected from specimens that are
prepared by splitting a cylinder liner. The cylinder wall is cut into stripes
or small pieces. To obtain a comprehensive understanding of multiscale
surface structures, examinations using LOM and SEM images complement each other in the aspects of overall surface appearances and material details.

3.1 Light optical microscopy

The LOM [11, 63] is the most efcient and economic instrument compared
to any other examination technique in metallography. It takes advantage
of a compound lens to focus light into the eyes or a camera. On metallic surfaces the interaction between the specimen and the incident light
is recognizable, when light reected from a surface region (the eld of
view, FOV) hits the optical system. At the end of the light path an CCD or
CMOS sensor converts the light signals into digital images. In Fig. 3.1, the
structure of LOM systems is schematically illustrated. Figure 3.2 shows a
LOM acquisition for a piece of laser-exposed cylinder bore surface.
Current optical microscopes possess sufcient resolving power for
imaging graphite grains. Based on optical geometry, the optical magnication of LOMs is the product of the powers of the ocular and objective
lenses. Nowadays, the maximum magnication of LOMs can reach values higher than 1000. Moreover, as the light from a point on the specimen
passes through the back aperture of objectives, a disk-like fringe pattern
appears in the image plane. Optical resolution is the smallest distance between two points on a specimen that can be distinguished as separate entities. Currently, the maxmal spatial resolution of LOMs is equal to 0.2 m.
These physical properties of LOM systems can fully meet the requirement
for the detection of graphite grains. Normally, uncovered graphite grains

13

3 Image acquisition

Figure 3.1 Light optical microscope system.

have a mean area of ca. 0.2 0.2 mm2 . In Fig. 3.2 every square milimeter
contains approximately 10 grains. Total 200300 grains can be observed
in the FOV. Such a magnication is very suitable for the study of grain
distribution. Additionally, Fig. 3.2 shows an image block captured by a
high-resolution CCD camera. The size of each graphite grain is equal to
an image area of 50100 pixels. This makes graphite grains distinct to
other surface components with different sizes. Another property of LOMs
that should be mentioned is the depth of focus (DOF). Only those surface points distancing from the focused plane within a tolerence can be
sharply imaged by a LOM. Specimens of cylinder bores have curved surface proles. The DOF should be improved with a low optical magnication or a small numerical aperture [6] in order to obtain a good image
quality. By sophiscatedly conguring the LOM, most part of the cylinder bore surface shown in Fig. 3.2 is sharply imaged. The distoration is
not recognizable, since the imaged surface patch can be approximated
as a plane, when the radius of the cylinder bore (100 mm) is far greater
than the extension of the surface patch. Besides, image contrast may be
inhomogeneous due to the non-uniform illumination. Such disturbance
degrades the image quality. In this inspection task, the qualied surface
nish should show ne plateau grooves and evenly uncovered graphite
particles. The dark graphite grains shown in this image are to be detected
with the method described in Chapter 5.

14

3.2 Scanning electron microscopy

Figure 3.2 Unqualied laser-exposed cylinder bore surface (evenly uncovered

graphite grains, too many deep grooves). FOV 5 5 mm2 . The
image size is 256 256.

3.2 Scanning electron microscopy

In metallography, material details are preferably examined with a scanning electron microscope. A SEM imaging system [62] is schematically
illustrated in Fig. 3.3. It utilizes a high-energy electron beam, namely the
primary electrons (PEs), for scanning the specimen. By means of magnetic and objective lenses, the electron beam emitted from the electron
gun is focused to a spot on the specimen surface. By altering the electrical current in deection coils, the electron beam can be deected with
a certain angle. In this way, the scanning spot translates to the next scanning point in a predened raster pattern. Whenever the electron beam hits
the surface, the energy exchange between incident electrons and specimen atoms creates a variety of signals, among which secondary electrons
(SEs) are most frequently used for imaging. SEs are released from the surface by excitation. A special detector collects SEs scattered in the sample
chamber, and then converts them into successive electrical signals. Furthermore, a signal processing system connected to the detector serves for
signal amplication and digitalization. The output signal can be assigned
to an image pixel in form of a gray value. When the scanning raster is
associated with the image grid, the surface region within the scanning
raster is ultimately able to be viewed as a digital image.
Unlike the LOM, the magnication of a SEM is not derived from the
power of lenses but from the following relation:

15

3 Image acquisition

Figure 3.3 Scanning electron microscopy.

SEM magnication =

displayed image size

.
raster size on the specimen

The lenses of the SEM are only to focus the electron beam into a spot.
Assuming that the image size is xed for the display, the specimen will
be enlarged as the raster size is reduced. Accordingly, the FOV becomes
smaller. As long as the spot is small enough to resolve the scanning raster,
the specimen can be sharply imaged. The brightness of SEM images relies
on the amount of collected secondary electrons. Regardless of the primary
electron beam and the specimen material, the surface pose is the main factor affecting the emission of secondary electrons. Figure 3.4 demonstrates
that the escape path turns shorter as the incident angle increases. It

16

3.2 Scanning electron microscopy

Figure 3.4 Secondary electron emission. As the primary electron beam enters
the specimen, secondary electrons escape from a teardrop-shaped excitation volume.

Figure 3.5 Topography contrast.

17

3 Image acquisition

Figure 3.6 Unqualied plateau-honed surface (too many metal folds, less evident plateaus). FOV 0.12 0.12 mm2 . The image size is 256 256.

means that secondary electrons are easier to emit from steep surfaces.
Another phenomenon is that more electrons can be captured from a surface toward the detector than that opposite to the detector. Considering a
rough surface prole composed of a number of small sections, the intensities could vary like the schematic diagram shown in Fig. 3.5.
Figure 3.6 displays a piece of a unqualied plateau-honed surface. Owing to the high resolution and the large DOF, the SEM image shows
a stereo effect that is not available in LOM images. These properties
are quite benecial for the structure-oriented analysis. This SEM image
is later employed for the development of image analysis algorithms.
This surface sample shows that the cross-hatch patterns are not successfully machined. Honing grooves are seriously interrupted into fragments.
Moreover, it can be observed that the defective locations marked with arrows are folded metal akes. Such a surface structure will induce a strong
surface friction. The piston ring is also likely to be damaged by sharp
metal folds. In addition, plateau grooves do not abound in this image.
As a result, the piston is hard-going on such a surface. The algorithm for
defect detection will be presented and experimentally validated with this
image in Chapter 4.

18

4 Defect inspection
For plateau-honed cylinders, the presence of two bands of honing
grooves is desired in the manufacturing process. Due to the imperfectness of metalworking, honing grooves are smeared and interrupted by
folded metal. These material defects result in frictional losses and accelerate the wear of the piston. Moreover, exfoliated metal akes increase the
particle emission in the running process of engine operation. Currently,
the grading of defect severity is still a demanding work. In most cases,
metal folds are visually analyzed in SEM images. The obtained inspection results are not reliable. For these reasons, quantitative evaluation of
surface qualities is in demand.
Plateau-honed surfaces possess a physically inhomogeneous topography. Two roughness levels exist in s honing textures. The surface components with low roughness are considered as smooth plateaus, on which
ne grooves show a very low image contrast. These smooth surface
regions are partitioned by deep honing grooves, which feature a high
roughness level. SEM images can faithfully reect the intrinsic surface
topography in a 2D fashion. Deep grooves are well contrasted in graylevel SEM images. Surface damages like metal folds exist on both roughness levels. These manufacturing failures are distinct, when they extend
beyond the surface. Weak scratches on plateaus have little inuence on
engine properties. Hence, the presented detection will concentrate on
the high roughness level, which shows higher image contrast than that
at plateaus. In this chapter the state-of-the-art approaches for detecting
metal folds are reviewed rstly. Afterwards, a new inspection strategy
based on the edge-aware structure tensor is proposed. Compared with
previous works, the method is edge-based and independent of honing
grooves. It will be shown that the presented method is applicable to inspect honed surfaces owning a wide range of qualities.

19

4 Defect inspection

4.1 Previous work

4.1.1 Approach using signal processing
The early work for the detection of metal folds (Blechmantel in German) and other defects (holes, groove interrupts, stains, etc.) was presented by Beyerer and Puente [3, 58]. Their image analysis approach is
based on a signal model. Ideal honing texture is assumed to consist of
compounded groove sets according to the designated spatial geometry.
Surface defects are treated as abnormal places in groove textures. In the
last few years this model has been the foundation of analyzing honed
surfaces in digital images. The classic application of Beyerers model was
separating straight lines from an isotopic background. The framework
was originally developed in the Fourier domain [5]. Recently, an improvement was made in [80], where the wavelet transform was utilized to optimize the textural decomposition in WLI images.
The family of algorithms following Beyerers model shows a consensus that defect inspection should become trivial in the separated background image. Here Beyerers scheme is briey recalled in Fig. 4.2. Figure 4.1 shows the detection result obtained by using the test image shown
in Fig. 3.6. The 2D Fourier coefcients representing textures consisting
of straight grooves are distributed along compact radial lines across the
origin. The background is assumed to be isotropic and, according to the
Riemann-Lebesgue lemma, its spectrum concentrates around the origin
of the spatial frequency domain. After splitting the image spectrum into
an anisotropic and an isotropic part, the groove texture and the back-

(a)

(b)

(c)

Figure 4.1 Signal decomposition of Fig. 3.6. (a) Groove texture. (b) Background.
(c) Defective edges.

20

4.1 Previous work

Figure 4.2 Beyerers scheme for defect detection.

21

4 Defect inspection

ground can be reconstructed by using the inverse Fourier transform. Defects can be localized in the background image by edge detection. To obtain closed contours of defective edges, Laplacian-of-Gaussian (LoG) lters are adopted in their work. Their results can be further optimized by
limiting the detection within grooved regions.
The model-based textural decomposition turns out to be problematic
for analyzing seriously defective surfaces. The spectral analysis of honing textures handles surface images globally. The algorithm works well
for honed surfaces in which grooves are present throughout the entire image. Practical manufacturing processes may be disturbed by many complex factors. Machine parameters, the material of honing stones and the
coolant could lead to signicantly varying surface qualities. The test image shown in Fig. 4.1 illustrates that the global algorithm is not capable of
thoroughly eliminating groove fragments from the background. Artifacts
may appear in separated images, because grooves reconstructed from the
2D Fourier domain are intact lines without gaps. The presence of groove
interrupts limits the anisotropic character of grooves, which may induce
an improper assignment of Fourier coefcients during the spectrum decomposition. Moreover, it should be argued that the inuence of image
noise is not ignorable in the stage of detection. During the LoG-ltering,
unexpected zero-crossings could be created by noise. As a result, defect
contours may be linked with noise contours. Such a problem is troublesome, since defects could be incorrectly localized. So far, the exact nding
of defects is still challenging for 2D image analysis of honed surfaces. In
view of the drawbacks of global signal decomposition, the algorithm developed in this thesis should be independent of the presence of honing
grooves.
4.1.2 Approach based on surface measurement
The height of surface proles provides additional information for surface inspection. In general, the surface topography can be separated into
material spurs, plateaus and valleys [78, 80]. The assessment for material defects not only depends on surface heights but also on textural features. Xin [80] and Dimkovsiki [14] brought forward similar interpretations of honing defects that are observed with a white light interferometer (WLI). They believed that tool marks on honed surfaces should be
non-interrupted grooves so that motor oil can ow smoothly. This family of methods segmented a 2.5D image into deep grooves and plateaus.

22

4.1 Previous work

(a)

(b)

Figure 4.3 Detection results published in [14]. (a) WLI image of a plateau-honed
surface. (b) Coarse grooves interrupted by metal folds that are colored in gray.

Boundaries of coarse grooves were determined by the Radon transform.

Plateaus were regions among coarse grooves. Based on surface heights,
they found blocked regions inside globally reconstructed grooves. These
groove interruptions were detected as metal folds. Furthermore, peaks
rising on plateaus were regarded as material burrs. This idea is illustrated
in Fig. 4.3. However, the approaches mentioned above suffer from some
drawbacks:

p The defect detection depends on the estimation of groove parameters, such as the position, the width and the angle of individual
grooves. Unfortunately, it is difcult to estimate these parameters at
smeared grooves. The long-distance interaction of image contents is
often needed to link groove fragments.
p Nowadays, some motor producers do not consider interrupted
grooves as manufacturing failures, but treat them as a part of the
micro-press chamber system. Such an understanding conicts with
the denition of defects advocated by Xin and Dimkovsiki.
To avoid confusion, only the defects that extend beyond the surface are
taken into account. Metal folds that have been pressed into surfaces are
not in the scope of the study, because they do not really increase surface
friction. The approach identies honing failures at defective edges which
have irregular shapes due to the smearing of honing stones.

23

4 Defect inspection

4.2 Local orientation analysis

In 2D gray-level images, strong intensity variations are referred to as image edges. The locations where the intensity variation happens form spatial edge shapes. Physical groove boundaries are imaged as straight edges
of high strength, while metal folds are distinct due to their sharp and
rough edges. In nature, straight and rough edges can be distinguished
with local orientations. Thus, local orientation analysis is the main techinique used in the presented detection scheme.
In the literature the orientation information has been known as a signicant feature of oriented textures. A varity of orientation operators was
presented. The choice of orientation operators is quite dependent of applications. Some special properties of textures were emphasized in the
process of feature extraction. In this context these methods are sorted into
two classes, that is, gradient-based and lter-based methods. In the following, the characteristics of these methods are reviewed. Then, a novel
approach is introduced in Section 4.3.
4.2.1 Filter-bank based methods
4.2.1.1 A short survey
Filter-bank based algorithms look for directional structures with scaled
and rotated lter kernels. The selected lter bank can be composed of
Gaussian derivative lters [37], Gabor lters [43] or steerable lters [20].
Filtered images construct an orientation space [9, 23] or orientation-scale
space [84], in which overlapped objects with multiple orientations can
be segmented. Since each kernel of the lter bank only covers limited
spatial frequencies, high-frequency noise could be discarded from lter
channels. Filter-bank based methods are thus more resistant to noise than
gradient-based ones. The prominent weakness of these methods is the
limited orientation accuracy due to the xed number of lters.
4.2.1.2 Gabor-lter bank
As reported in [1], the Gabor-lter bank is an oriented feature detector with good performance. Through creating an orientation-scale space,
many useful edge features can be extracted. Inspired by this, in Section
4.3.3 a set of feature-adaptive lter kernels is developed to improve the
estimation of local orientations. Several techniques [25, 38, 73] have been

24

4.2 Local orientation analysis

reported in the literature for the design of Gabor lters. The advice of
Manjunath and Ma [38] is adopted here, becase their method has been
validated in many applications on texture segmentation and object recognition. A 2D complex Gabor function and its Fourier transform can be
formulated as




1 x2
y2
1
f ( x, y) =
exp
+ 2 + 2ju0 x ,
2x y
2 x2
y


1
F (u, v) = exp
2

( u u0 )2
u2

v2

(4.1)



v2

(4.2)

where u = 1/2x and v = 1/2y . The band limit of a Gabor lter

is dened with its half-peak magnitude support in the Fourier domain.
The Gabor-lter bank is a class of self-similar Gabor functions that are
generated by scaling and rotating the basic lter f ( x, y). The members of
Gabor-lter bank are non-orthogonal. In order to reduce the redundancy
in ltered images, the lter supports in the Fourier domain are aligned to
touch each other. Given the number of orientations, N , and the number


of scales, N , a Gabor lter bank can be notated as f , (x) with =
1, 2, . . . , N and = 1, 2, . . . , N . An example is shown in Fig. 4.4.
Formally, the member of a Gabor-lter bank can be written as


f , ( x, y) = F x  , y ,

(4.3)

with
x  = ( 1) ( x cos + y sin ) ,
y = ( 1) ( x sin + y cos ) ,
The parameters of Equ. 4.3 are calculated with following formulas:

25

4 Defect inspection

=
=

( 1)
,
N
 u N 11
N

u1

u0 = u N ,

( 1) u N

,
( + 1) 2 ln 2
1




2
2 2
2

u
(2 ln 2) u

.
u N 2 ln
2 ln 2
v = tan
2N
u N
u2N

u =

The ltered images are complex, and can be written as

g , (x) = g, (x) f , (x) .

(4.4)

It is worth to note that in the spatial domain the real (odd) part of complex Gabor lters is capable of detecting step edges, while the imaginary
(even) part is useful for nding roof edges. Since both step and roof edges
should be detectable for the inspection of honed surfaces, the Gabor energy function is utilized for analyzing the structural anisotropy. Hence,
a matrix of size N N is constructed at a pixel location, x. The matrix

2
2





element is  g , (x) . As explained in [34],  g , (x) will be high when
f , (x) coincides with the edge width and the edge angle. To gain scale
2


invariant edge features, the maximum of  g , (x) is chosen from the
dimension that repesents the scales [33]. The resulting feature is merely
related to orientations:

2


g (x) = max  g , (x) .

(4.5)

The values of g (x) at all orientations indicate an energy distribution,

which is able to describe local multi-oriented structures.

26

4.2 Local orientation analysis

Figure 4.4 Gabor-lter bank in 2D Fourier domain. N = 6 and N = 4. u1 and

u N are the lowest and highest center frequencies, respectively. is
the angle interval.

4.2.2 Gradient-based methods

4.2.2.1 A short survey
Gradient-based approaches utilize the structure tensor [29] to estimate
local dominant orientations. The resulting orientations deliver higher accuracy at ne scales than lter-bank based methods. However, conventional derivative lters used to compute gradient vectors are sensitive to
image noise. For this reason, the estimation accuracy of local orientations
is often degraded due to image noise. To deal with this problem, some researchers improved the robustness to noise by using alternative computational methods. The orientation operator proposed in [32] implements
a band-pass lter by integration. Another author [40] approximated the
structure tensor by auto-correlation. Some efforts were also made to improve the structure tensor by adaptively varying the smoothing mechanism [7, 8, 15, 46, 81]. Besides, many applications [31, 41, 55] require to
smooth the orientation eld without destroying relevant singular points.
Diffusion-based regularization [10], model-based optimization [83] and
block-wise voting schemes [77] are approaches that were specially designed for this purpose. The resulting orientation elds were coarsened
in order to describe objects at desired scales.
27

4 Defect inspection

In this thesis, the detection of metal folds is associated with edge orientations. The idea is to distinguish rough and straight edges via a descriptor of edge shapes, which requires high accuracy not only for edge
angles, but also for edge locations. The conventional gradient-based approach is considered to be suitable for this applcation, and thus chosen
as the foundation of this work. Firstly, this approach is recalled. Its problems are illustrated through some tests. To improve the robustness, the
gradient-based approach is then optimized in Section 4.3.
4.2.2.2 Classic structure tensor
The optimal orientation estimation is assumed to fulll the least squares
principle. The question can be formulated as minimizing the following
objective function in terms of the unity vector n representing the optimal
orientation:
J=

1
N2


 
T 2
( g (x)) n .
x W

x = ( x, y)T denotes a pixel location. g (x) =

(4.6)
T

(x) is the graylevel gradient of the image g (x). W is the support of a local window,
W, which is sized to N N. The minimization problem can be solved by
eigendecomposition of the structure tensor,

T11 T12
(x)
TC ( x ) =
(4.7)
T21 T11
g x , gy

with


T11 (x) = g2x GW (x) ,


T22 (x) = gy2 GW (x) ,


T12 (x) = T21 (x) = gx gy GW (x) ,
where the structure tensor TC (x) is dened as the smoothed dyadic product of the gradient vector, GW (x) denotes a Gaussian or averaging lter

28

4.2 Local orientation analysis

kernel, and is the convolution operator. Computationally, the eigenvalues can be calculated directly with



1
2
det (x) + det (x) 4tr (x) ,
1 (x) =
2



1
det (x) det2 (x) 4tr (x) .
2 (x) =
2

(4.8)
(4.9)

Here, det (x) is the determinant of TC (x), and tr (x) is the trace of TC (x):
2
det (x) = T11 (x) T22 (x) T12
(x) .
tr (x) = T11 (x) + T22 (x) ,

Moreover, the angle of the eigenvector corresponding to the largest eigenvalue is expressed as
O (x) = tan1

2T12 (x)
.
( T22 T11 ) (x)

(4.10)

The local dominant orientation is perpendicular to O (x) /2.

4.2.2.3 Property
The mathematical expressions in Section 4.2.2.2 are usually used for the
calculation. By smoothing the whole image domain, the mean squaredgradient eld can be obtain as follows:
u (x) = 1 (x) ejO(x) .
The pros and cons of the classic structure tensor are illustrated with
the test image shown in Fig. 4.5. The window size critically affects the
orientation estimation. The large window signicantly reduces noise but
introduces a strong blurring in both the amplitude and orientation eld.
In contrary, the rough edge is preserved by using a small window. However, noise reduction may be insufcient at straight edges. Besides, the
classic structure tensor can only deal with the intrinsic 1D structure [17]
that shows only one dominant orientation. At the structure mixture (e.g.,
junctions) the estimated orientation will be far from each oriented structure contained in the local window.

29

4 Defect inspection

To detail the reasons of aforementioned issues, the eigenvalues and

eigenvectors of the classic structure tensor are formulated in an alternative form. According to [31], the eigenvalue analysis of the structure tensor is equivalent to averaging squared-gradients in a local window. Let
me represent the gradient vector with a complex variable:

g (x) = gx (x) + jgy (x) .

(4.11)

The squared-gradient can be expressed as

r (x) ej(x) = g2x (x) gy2 (x) + 2jgx (x) gy (x) .

(4.12)

Then, squared-gradients in a local window can be visualized on the complex plane with a compass plot. The obtained vector map demonstrates
a gradient distribution which can reect edge shapes. Articial patterns
shown in Fig. 4.6 are used to illustrate four typical structures straight
edges, rough edges, junctions and smooth patches. Gaussian noise is deliberately added to these pictures. The corresponding gradient distributions are depicted in Fig. 4.7. It can be seen that the dispersion of squaredgradients changes with the edge strength and the noise level. Therefore,
the mean squared-gradients are also contrast-variant and noise-sensitive.
The performance of the structure tensor can be further discussed in real
images. A surface image is represented with the following model:
g (x) = i (x) t (x) + n (x) ,

(a)

(4.13)

(b)

Figure 4.5 Vector elds, 1 (x) ej(O(x)/2+/2) , estimated by the classic structure
tensor. (a) N = 5, (b) N = 21.

30

4.2 Local orientation analysis

(a)

(b)

(c)

(d)

Figure 4.6 Noisy articial patterns. (a) Straight edge. (b) Junction. (c) Rough
edge. (d) Smooth patch.

where t (x) is the undisturbed image signal of a relief texture, i (x) denotes
the inhomogeneity, and n (x) is additive white Gaussian noise (AWGN).
The gradient eld derived from Equ. 4.13 is

g (x) i (x) t (x) + n (x) ,

(4.14)

where the derivative of i (x) is neglected, because i (x) normally varies

very slowly in local regions. Equ. 4.14 means that i (x) scales the gradient
amplitude, |t (x)|, but does not tune the gradient angle of t (x). In comparison, image noise distorts both the gradient amplitude and the angle.
Let me notate the squared-gradient of t (x) in the polar coordinate system
as rt (x) ejt (x) . Then, the squared-gradient of g (x) can be described with


r (x) ej(x) = i2 (x) rt (x) + rn (x) ej( t (x)+ n (x)) ,

(4.15)

where rn (x) and n (x) are the biases in the radial amplitude and the
rotated angle, respectively. This representation inherits the properties of
Equ. 4.14 and combines the signal and the noise in a single term. Furthermore, it should be noted that strong gradients have higher votes to the
vector addition than weak ones. This enables to describe local dominant
structures by simplifying Equ. 4.15 as follows:

p Squared-gradients at strong edges can be approximately characterized with i2 (x) rt (x) ej( t (x)+ n (x)) ;
p Weak edges and smooth regions can be expressed as rn (x) ejn (x) ,
since these places are dominated by noise.

31

4 Defect inspection

(a)

(b)

(c)

(d)

Figure 4.7 Gradient distributions of noisy patterns. (a) Straight edge. (b) Junction. (c) Rough edge. (d) Smooth patch.

Based on the analysis above, the mean squared-gradient in a local window can be formulated as

RW ejW

32

1
N

x W


i2 (x) rt (x) + rn (x) ej( t (x)+ n (x)) .

(4.16)

4.3 Edge-aware structure tensor

The decomposition is then accomplished in the following equations:

RW ejW = Rt ejt + Rn ejn

(4.17)

with
Rt ejt
Rn ejn

1
N2
1
2

N

1
(x) =
0

(x) i2 (x) rt (x) ej( t (x)+ n (x)) ,

x W

(1 (x)) rn (x) ejn (x) ,

x W

if x at strong edges,
otherweise.

In the case of weak image noise, the gradient distribution of strong edges
still keeps a signicant anisotropy. Hence, the rst term of Equ. 4.17,
Rt ejt , represents a smoothed version of strong edges. The second term,
Rn ejn , can reduce noise by vector addition due to the large dispersion
of noisy vectors. From this model it can be known that edges and noise
are smoothed in the whole window region W . As a result, edges cannot
be preserved by such isotropic ltering. Moreover, this model also indicates that edges and noise can be separated in local windows under peroper assumptions. This property enables the design of anisotropic lters
which should limit the smoothing within specied regions. The study of
anisotropic lters is presented in the next section.

4.3 Edge-aware structure tensor

4.3.1 Design idea
In this section, the classic structure tensor is improved by means of tailoring local squared-gradients. Fig. 4.8(a) sketches a group of squaredgradients in a local window whose center is located at an edge. In this
case, vectors similar to the one at the window center should take more
votes for orientation estimation. In another situation, the window center may be a noisy pixel, as shown in Fig. 4.8(b). Noisy vectors are expected to be selected for the averaging. This concept is realized with an

33

4 Defect inspection

(a)

(b)
Figure 4.8 Improved orientation estimation (a) at an edge pixel and (b) at a
noise pixel. Black solid lines are squared-gradients at window centers, while gray solid lines are squared-gradients in the neighborhood. In the rst stage vectors with a norm signicantly different
from the window center are removed. In the second stage the mean
squared-gradients are indicated with black dash lines.

algorithm that combines the classic structure tensor with a bilateral lter and an anisotropic lter. The achieved structure tensor is named as
edge-aware structure tensor (EAST), which demonstrates superior characteristics in noise reduction and edge preservation. Large local windows
are preferred in the structure tensor for signicantly reducing noise. The
proposed algorithm is able to reach the requirement on the structure selectivity depicted in Fig. 4.8.

4.3.2 Bilateral lter

The bilateral lter [51, 72] and its variants are important techniques in the
world of edge-preserving ltering. This attractive image processing tool
has been widely applied to image editing, image denoising and contrast
enhancement. It takes the following form:

34

4.3 Edge-aware structure tensor

g (x) =

1 
g (p) GW (
p x
) GR (| g (p) g (x)|) ,
C

(4.18)

p W

where C is a factor for normalizing the sum. GW denotes a 2D Gaussian,

which is related to the Euclidean distance between x and its neighbor p.

means the L2 -norm. GR is a range function with a Gaussian form,

which is attributed with | g(p) g(x)|, that is, the L1 -norm of image intensities. Since only similar intensities take part in the smoothing, strong
edges can be preserved in ltered images. Furthermore, some authors [26,
54] improved the original bilateral lter with adaptive range functions relying on another image. This class of lters is known as the cross/joint
bilateral lter, and takes a common form:
g (x) =

1 
g (p) GW (
p x
) GR (| g0 (p) g0 (x)|) .
C

(4.19)

p W

where g0 is a reference image different from g.

With some extensions, the bilateral ltering can be integrated into the
classic structure tensor. The new structure tensor is highlighted with the
properties designed in Fig. 4.8. It is straightforward to extend the bilateral
lter to vector elds or tensor elds [15, 81]. In these applications, the
range lter, GR , was linked with the distance of vectors or tensors. The
ltering was carried out for each element of vectors or tensors. In Equ.
4.17 the mean squared-gradient was modeled by using a constant box
kernel as the spatial lter. The local window was separated into edges
and noise regions. If (x) is substituted with a bilateral kernel, the mean
squared-gradient eld can be expressed as
u (x) =

1 
r (p) ej(p) GR (
g (p)

g (x)
) .
C

(4.20)

p W

This mathematical expression can theoretically explain the principle of

the bilateral structure tensor proposed in [81].
4.3.3 Adaptive lter kernels
The approach differs from the previous work in the improved structure
selectivity, which leads to an anisotropic structure tensor. The lter ker-

35

4 Defect inspection

nel should be adaptively tuned in order to alleviate the structure mixture in local windows. In this work the Gabor-lter
 bank is used to detect

multi-oriented structures. The Gabor-features, g (x) , = 1, 2, . . . , N ,
extracted in Section 4.2.1.2 are capsulated into a vector,


g (x) = g1 , g2 , . . . , g N (x) ,

(4.21)

because they belong to a same type of features. Additionally, the gradient

magnitude,
g (x)
, is also involved in the lter kernel. The ltering is
expected to occur at strong edges. As a result, two range functions are
used for the bilateral ltering. They are then combined in a single lter
kernel:
G1 2 (p, x) = G1 (p, x) G2 (p, x)

(4.22)

with

G1 (p, x) = exp

(
g(p)

g(x)
)2
12


,


2


2
2

g
g

g
(
p
)

p
g
(
x
)

(
)
(
)



G2 (p, x) = exp
.
22
Formally, the mean squared-gradient eld is expressed as
u (x) =

1 
r (p) ej(p) G1 2 (p, x) .
C

(4.23)

p W

As explained in Section 4.2.2.3, the eigendecomposition of the structure

tensor is equivalent to ltering a vector-valued image. In this sense, G1
and G2 are range lters for vector amplitudes and vector angles, respectively. G1 contributes to localizing strong edges, whereas G2 is an
anisotropic lter responsible for searching oriented structures similar to
the window center.

36

4.3 Edge-aware structure tensor

4.3.4 Tensor ltering

Let me include the structure tensor in the framework of bilateral ltering.
The classic structure tensor (CST) dened in Equ. 4.7 can be rewritten as
a weighted sum of local tensors, i.e.,
TC ( x ) =

T (p) GW (
p x
)

(4.24)

p W

with

T (p) =

g2x

g x gy

g x gy

g2x

(p) .

GW plays a role of the spatial lter. Furthermore, it is easy to derive Equ.

4.16 from Equ. 4.24 in that the eigendecomposition is a linear operation.
In the same way, the edge-aware structure tensor (EAST) can be dened
as
TE (x) =

1 
T (p) G1 2 (p, x) .
C

(4.25)

p W

Similarly, the bilateral structure tensor (BST) can be expressed as

TB (x) =

1 
T (p) G1 (p, x) .
C

(4.26)

p W

4.3.5 Parameter selection

4.3.5.1 Parameters in Gabor lters
The performance of the EAST can be adjusted by a set of parameters. In
the step of feature extraction, the number of orientations and scales, N
and N , as well as the lowest and highest center frequencies, u1 and u2 ,
can fully determine the Gabor-lter bank. When the number of scales is
larger than two, the remaining center frequencies at other scales can be
automatically xed according to the lter layout. Gabor lters are normally used to extract frequency components that correspond to meaningful image contents. However, the spatial frequency distributions of real
images are often unknown. To deal with this problem, the Gabor-lter

37

4 Defect inspection

bank is constructed with following considerations. The number of scales,

N , is set to 3. Furthermore, the lowest and highest normalized center
frequencies are set to 0.125 pixel/cycle and 0.45 pixel/cycle, respectively.
In this way, Gabor lters are congured to be oriented low-, band- and
high-pass lters, which can cover the entire spectrum. The number of lter orientations decides the angular resolution for discriminating oriented
structures. By setting N to 18, the angle interval of Gabor lters, , is
equal to 10 .
4.3.5.2 Adaptive smoothing parameters
Both range lters used have a common Gaussian form with a zero mean.
Thus, they can be fully determined by their standard deviations. Each
range lter is observed as a sigmoidal threshold function which transforms feature distances into [0,1]. G1 serves as a detector of edge/nonedge structures. Its standard deviation plays a role of a soft threshold
that classies image structures in local windows according to the similarity of edge strength. In the EAST another range lter is used to deal
with the problem induced by structure mixture. G2 classies local orientations into two groups that are similar and dissimilar to the orientation
of the window center. Since Gaussian noise does not indicate obvious orientations, noisy pixels in smooth regions are taken as similar in terms of
the structure anisotropy. 2 plays a role of a soft threshold to discriminate
the orientation similarity. The larger these two parameters are, the more
strongly the squared-gradient vectors can be smoothed.
In most applications the smoothing parameter for the range lter is set
to a globally unied value. The empirical studies reported in [82] manifest that the global parameter is not suited for detail preservation in all
regions. The authors of [79] derived adaptive parameters from local phase
characteristics. Their bilateral lter improved the perceptual effect of image denoising. [15] brought forward a statistical method for estimating
the optimal smoothing parameters. However, this method relied on an
iterative optimizing process which was not convenient to handle. In this
work, an adaptive method for parameter estimation is developed. In Equ.
4.22 the range lters are attributed with
d1 (p, x) = |
g(p)

g(x)
| ,




d2 (p, x) = 
g(p)
2 g (p)
g(x)
2 g (x) .

38

(4.27)
(4.28)

4.3 Edge-aware structure tensor

The smoothing parameters are associated with the distribution of feature

distances in local windows. The effectiveness of the method is ascertained
by the fact that the predented feature vectors are powerful for discriminating oriented structures.
Histogram-based approaches are normally used to study the distribution of statistical variables. To build the histogram of dk with k = 1 or 2,
feature distances should be divided into bins. It is nontrivial to do this,
because the dynamic range of dk varies in each local window. The size
of bins will critically affect the histogram of dk . Instead, the
of

 prole

sorted feature distances is studied in local windows. Let d(l ) be a

set of increasingly sorted values of dk in a local window, where l
{1, 2, 3, . . . , lmax } denotes the index. For local windows of size N N,
lmax = N 2 1 holds. The feature distance located at the window
center

is ignored, because it is always equal to zero. Moreover, d(l ) implies
the histogram of dk , which is denoted as H (d) in this context. Note that
each of range lters in the EAST is adopted to discriminate two sets of local structures. Edge and non-edge structures are separated in G1 , while
similarly and dissimilarly oriented structures are distinguished in G2 .
If image features for local structures have a high descriminative power,
the histogram of dk will be unimodal or bimodal. Accordingly, the prole of d(l ) will have large convex or concave curvatures. Based on this
relation, the distribution of dk can be separated into several partitions
which represent obviously small, transitive and obviously large feature
distances. Correspondingly, the shape of d(l ) is simplifed with connected
line segments showing one or two inection points. Fig. 4.9 schematically demonstrates four possible simplied proles of d(l ), as well as the
shapes of H (d) regardless of histogram bins. Furthermore, the indices underlying these line segments construct several sets containing classied
pixels, which are labeled with S, A and D, respectively representing obviously small, transitive and obviously large feature distances mentioned
above. S = for type I, while D = for type IV. By simplifying and
splitting the prole of d(l ), a data-driven approach can be developed for
parameter estimation.
For types I, III and IV, transitive structures lie in the tail of H (d). Thus,
the structure classication is trivial in these cases. In comparison, the distribution of type II is dominated with transitive structures. To estimate
1 , the data in S and A zones are taken into account. This ensures that the

39

4 Defect inspection

Type I

Type II

Type III

Type IV
Figure 4.9 Left column: simplied proles of feature distances. Right column:
distribution models.

40

4.3 Edge-aware structure tensor

transitive regions from edges to smooth patches can be revealed in


2
1 

1 = d(1) +
d(l ) d(1) ,
(4.29)
lA
l S A

where d is derived from d1 , lA is the upper limit of A zone. Considering

the self-distance at the window center, the soft threshold is biased with
d(1). The second term of Equ. 4.29 is a rooted secondary center moment.
Moreover, 2 implies an angular tolerance for detecting consistent orientations with respect to the window center. In the proles of types I and
II, the data in S and A zones indicate a wide range of angle differences.
Therefore, these situations should be treated carefully. The estimate of 2
is d(1) plus an empirical value that is smaller than the mean of sorted feature distances in S and A zones. Furthermore, the distributions of types
III and IV have distinct peaks in the low value area of d. In these cases,
only S zone is considered for estimating 2 . A zone is not considered in
order to ensure the accuracy of orientation selection. In summary, the estimation functions are formulated in terms of prole shapes:
for types I and II,
2 = d(1) + ( ) ,

(4.30)

for types III and IV,


2 = d(1) +

2
1 

d(l ) d(1) ,
lS

(4.31)

l S

where the values of d are derived from d2 , lS is the upper limit of S zone.
Furthermore,

( ) =
d(l )
lA
l S A

with [0, 1]. The choice of is discussed later in Section 4.5.1. In a

special case, the local window may contain constant intensities. 1 and 2
will be equal to zero. As this situation appears, G1 2 is dened as a box
kernel.
The last question is how to determine the inection points in the simplied prole of d(l ). In the scheme, these points are calculated with the
help of following indices:

41

4 Defect inspection

Figure 4.10 Estimating inection points in d.



l1 = arg min d(l ) d(l ) ,

(4.32)



l2 = arg max d(l ) d(l ) ,

(4.33)

with
d(l ) = d(l ) +

d N 2 1 d (1)
N2 1

l.

d(l ) denotes a straight line connecting the begin and end points of d(l ).
The graphical illustration of l1 and l2 is shown in Fig. 4.10. lS and lA can
be found according to Table 4.1.
Table 4.1 Upper boundaries of S and A zones.

42

distribution
types

lS

lA

l2

II

l2

l1

III

l1

l2

IV

l1

lmax

4.4 Detection scheme

4.4 Detection scheme

4.4.1 Feature extraction
It has been known from Fig. 4.7 that orientations along straight edges
show a unimodal distribution, whereas orientations along rough edges
are randomly scattered. Moreover, multi-oriented structures like junctions possess two or more dominant orientations. Therefore, the orientation uniformity is high in individual modes, but low in the whole distribution. Smooth patches do not indicate an obvious anisotropy. It can be
observed that plateau-honed surfaces consist of these four typical structures. Rough edges are regarded as defects. Since the orientation dispersion of noisy vectors makes no sense for the application, only the dispersion of edge orientations is investigated in the vector eld.
u (x) = 1 (x) (x)ejO(x) ,

(4.34)

where (x) is the edge map created by segmenting the amplitude eld,
1 (x), with Otsus threshold [50]. Then, the orientation dispersion is measured with the small eigenvalue of the classic structure tensor. The window size is xed to 5 5 in the assumption that such a small window can
only contain a single edge. The derived small eigenvalues are denoted as
2 (x).
Since the edge strength is the main perceptual evidence in 2D gray
value images to identify the severity of surface damage, image contrast
is also an important cue for defect detection. Combining the contrast and
orientation features of defective edges, the defect signature is ultimately
dened as

S (x) = N (1 ) N 2 .

(4.35)

N () is a min-max normalization function, which linearly transforms the

maximum to one and the minimum to zero. Since eigenvalues have large
dynamic ranges, 1 and 2 are logarithmically stretched before normalization. Because metal working is imperfect, it can be considered that defects always exist in surface samples. This means that slight orientation
dispersion cannot be enhanced by normalization.

43

4 Defect inspection

Figure 4.11 EAST-based detection scheme.

44

4.5 Experimental results

(a)

(b)

(c)

(d)

Figure 4.12 Detection at junctions. (a) Intersection of straight lines. (b) Detection
result of (a). (c) Intersection of a rough line and a straight line. (d)
Detection result of (c).

4.4.2 Segmentation
The most salient defects can be segmented by thresholding S (x) with
Otsus method. The EAST-based detection scheme is summerized in Fig.
4.11. Moreover, inhomogeneous intensities are irrelevant signals for surface characterization. These signals can be eliminated from the original
image by preprocessing with homogenization techniques [4, 76]. Furthermore, it should be pointed out that any intersections at rough edges are
a part of defects. In other words, only junctions of straight edges should
be excluded from detection results. This consideration is explained with
Fig. 4.12. In the postprocessing, isolated small and round regions are removed, since defective edges are large and elongated objects. The roundness measure is dened in Appendix C.1.

4.5 Experimental results

4.5.1 Range lters
In this section, lter kernels as well as parameter settings for the EAST are
validated. The improvement of the lter kernel is illustrated by comparison with the BST. In the experiments, gradients are computed with horizontal and vertical central difference operators. Filter kernels are sized to
21 21 for both structure tensors. The smoothing parameter for the BST
is automatically computed with Equ. 4.29. Moreover, the inuence of is
investigated with an image pattern shown in Fig. 4.14(a). In this example,
both the proles of sorted d1 and sorted d2 are shaped as type I, as depicted in Fig. 4.13. Since edges are normally sparse in local windows, d(l )

45

4 Defect inspection

(a)

(b)
Figure 4.13 Proles of d(l ) obtained from (a) d1 and (b) d2 . The local window is
shown in Fig. 4.14.

46

4.5 Experimental results

(a)

(c)

(b)

(d)

(e)

Figure 4.14 Range lter adjusted by . (a) Local window containing a rough
line. (b) G1 , (c)-(e) G2 with = 0.2, 0.6 and 0.9.

is likely to be types I or II at edges. Thus, the parameter, , is useful for adjusting the smoothing degree at edges. The window center marked with a
white circle is located at a rough edge. With the estimated 1 , the weights
for edges are signicantly higher than the surrounding in G1 . This indicates that edges and non-edge regions can be reasonably classied by
the automatic choice of 1 . By observing Figs. 4.14(c)-(e), it can be noted
that the larger the value of is, the stronger the edge will be smoothed. In
the application it is intended to preserve orientation uctuations at rough
edges. Thus, in the following is set to 0.2 by default.
Fig. 4.15 demonstrates three synthetic patterns and corresponding
adaptive lter kernels. In the rst row the window center is located in
a smooth region. In all three lter kernels, the weights for edges are obviously suppressed. This can effectively avoid the interaction between
edges and smooth regions during the ltering. In this test, 2 shows its
ability of discriminating oriented and non-oriented structures. G1 contributes to the localization of edges and smooth regions. Moreover, the
superiority of the EAST can also be observed as edges are smoothed.
As shown in the second row, G1 2 is oriented to the edge direction.
Surrounding edges having similar directions also gain large weights in
G1 2 . In this case, dispersed orientations will not be processed together.

47

4 Defect inspection

Figure 4.15 Range lters for articial patterns. (a) Synthetic patterns, (b) G1 , (c)
G2 , (d) G1 2 .

These two experiments verify that the EAST exactly obeys the design idea
demonstrated in Fig. 4.8. Besides, if a local window contains only noise,
the BST and the EAST will attempt to nd similar structures in noise. The
example for this case is shown in the last row.
Furthermore, the BST and the EAST are tested by using real SEM images of plateau-honed surfaces. Practical examples for the typical structures mentioned in Section 4.2.2.3 are shown in Fig. 4.16. In complex technical surfaces, edges are likely to be intersected in a small area. Hence,
G1 is not sufcient for generating accurate estimates of edge orientations.
This problem can be effectively addressed with the anisotropic lter, G2 .
These examples also verify that the strategy for estimating smoothing parameters is applicable to both synthetic and real images.
4.5.2 Amplitude and orientation elds
In this section, the performance of the CST, the BST and the EAST is evaluated with an articial picture. The articial image, shown in Fig. 4.17,
consists of random lines. Their positions and angles obey the uniform
distribution. Since this picture is rich of junctions, it is especially suited

48

4.5 Experimental results

Figure 4.16 Range lters for real image patches. The rows from top to bottom
are surface samples centered at a straight edge, a rough edge, a
junction and a smooth patch. The columns indicate (a) Real image
patches, (b) G1 , (c) G2 and (d) G1 2 .

for illustrating the advantage of using an anisotropic lter in the structure tensor. Moreover, the inuence of parameters is also discussed. The
CST and the BST are only affected by the window size, N, while the
EAST has a set of parameters. As mentioned in Section 4.3.5, the performance of the EAST is mainly inuenced by three parameters, N , N
and . During
the visual evaluation, these parameters are rstly set to

N , N, = (18, 21, 0.2). Afterwards, one of these parameters is altered

for the quantitative evaluation. To make objective assessments, a ground
truth is needed in the experiments. Here the amplitudes of squaredgradients, r (x), are taken as the ground truth of amplitudes. Besides,
since the digital lines are not perfectly straight in a low resolution image,

49

4 Defect inspection

Figure 4.17 Synthetic random lines.

the true edge orientations are assumed to be a slightly smoothed version

of (x). The CST with a window size of 3 3 is used for the smoothing.
The ground
truth of angles is denoted as (x), which is dened in
"
!
180 , 180 .

(a)

(b)

(c)

(d)

Figure 4.18 Visual evaluation. (a) Ground truth. (b)-(d) Estimated vector elds
associated with the CST, the BST and the EAST, respectively.

Figure 4.18 shows r (x) ej( (x)/2+/2) derived from noise-free gradients
as well as 1 (x) ej(O(x)/2+/2) obtained by structure tensors. These vectors are chosen from a local window depicted in Fig. 4.17. In Fig. 4.18(b)
the vectors obtained by the CST are strongly smoothed so that structure
details become invisible. In Fig. 4.18(c), the vector amplitudes are almost
zero at the junction. The drawback of the BST becomes clear in this example. In comparison, the EAST generates vectors that are most similar to
the ground truth.
Furthermore, a quantitative assessment shall be conducted. Vector amplitudes and vector angles are evaluated separately. Vector amplitudes
indicate the goodness of the edge localization, which can be described by
the peak signal to noise ratio (PSNR):

50

4.5 Experimental results

PSNR = 10 log10

1
A

max r2 (x)
x g

(1 (x) r (x))2

(4.36)

where A is the total number of pixels in the image. Vector angles reect the orientation accuracy. The mean angular error (MAE) is computed
with
MAE =

1 
arc (O (x) , (x)) ,
2Ae

(4.37)

x e

arc (O (x) , (x)) = 180 |180 |O (x) (x) || ,

where arc (O (x) , (x)) [47] computes the
distance from
! smallest angular
"
O (x) to (x) in the unit circle. O (x) 180 , 180 is estimated by one
of the structure tensors introduced in Section 4.3.4. The angles are selected
from
e = {p, r (p) > 0} .
Ae is the size of e . The mean orientation error is the half of the mean
error of O (x).
In the following, it is illustrated how the number of orientations of the
Gabor lter bank, N , inuences the performance of the EAST. In this
test the angle resolution of the Gabor lter bank is altered from 5 to
30 . Fig. 4.19 shows that the PSNR is approximately invariant to N . This
illustrates that the random lines shown in Fig. 4.17 are distinguishable
when the angle resolution is less than 30 . Thus, edge strength is not suppressed in the ltered amplitude eld. However, the orientation accuracy
is closely related to N . This problem is illustrated in Fig. 4.20. If N is
too small, multi-oriented edges cannot be exactly described with Gabor
features. If N is too large, the anisotropic lter kernel of the EAST will
be oriented to zigzag segments of discrete lines. In this experiment, the
optimal orientation number for the Gabor-lter bank is found at N = 18.
As shown in Appendix A.1, this setting is also suitable for the ltering of
other images.
Next, the parameter is taken into account. In Fig. 4.21 the amplitude
eld is slightly degraded by an increasing . As known from Fig. 4.14,

51

4 Defect inspection

Figure 4.19 Amplitude evaluation of the EAST depending on N .

does not affect the edge localization. Therefore, the degradation is not induced by the blurring across edges, like in Fig. 4.18(b), but by the smoothing of junctions, like in Fig. 4.18(c). Moreover, the MAE curve is shown in
Fig. 4.22. Choosing a value of that is too small will result in an undersmoothing of squared-gradients. Angle errors will be large due to noise.
In addition, if the value of is too large, the anisotropy of G2 will be
weakened. In this case, the orientation accuracy is lowered by the interaction of multi-oriented structures. Over-smoothed orientations cannot
exactly reect the shape of edges.
The third test is to investigate the inuence of the window size, which
is a common parameter for the CST, the BST and the EAST. Therefore,
a comparative study is performed. Test results are exhibited in Fig. 4.23
and Fig. 4.24. The CST is most sensitive to the window size. Poor edge
localization causes strong amplitude biases at both edge and non-edge
locations. The BST is better than the CST, since for the BST the ltering
does not take place across edges. In comparison, the EAST shows the
best edge-preserving ability. The estimated amplitude eld is nearly the
same whatever the window size is. The angle accuracy of the CST and
the BST depends on the degree of structure mixture in local windows.

52

4.5 Experimental results

Figure 4.20

Figure 4.21

Angle evaluation of the EAST depending on N .

Angle evaluation of the EAST depending on .

53

4 Defect inspection

Figure 4.22 Angle evaluation of the EAST depending on .

Figure 4.23 Amplitude evaluation of structure tensors depending on N.

54

4.5 Experimental results

Figure 4.24 Angle evaluation of structure tensors depending on N.

Hence, large windows are not suitable for estimating edge orientations.
The EAST breaks through this limitation and achieves good estimates for
angles. Local windows used by the EAST specify the area for searching
similar structures. The larger the window becomes, the more structures
that resemble the window center are likely to be found. Consequently,
the squared-gradient eld will be strongly ltered by using large windows. Hence, the under- and over-smoothing problems persist as well in
the selection of the window size. In this test the optimal window size is
N = 21.
Lastly, the noise resistance of all three structure tensors is illustrated.
Gaussian noise with different standard deviations is added to Fig. 4.17.
The noise level, n , is denoted as the ratio of the noise standard deviation
to the maximum gray value of the image. Once multi-oriented structures
cannot be separated in the lter kernel, amplitudes at junctions will be
blurred or weakened. PSNR values will be thus decreased. Fig. 4.25 shows
the PSNR curves computed for the CST, the BST and the EAST. As the
noise level is increased, PSNR values for the EAST rapidly declines from
a large value. This result can be interpreted by recalling the vector model
presented in Equ. 4.17, i.e.,

55

4 Defect inspection

Figure 4.25

Amplitude evaluation of structure tensors depending on n .

Figure 4.26 Angle evaluation of structure tensors depending on n .

56

4.5 Experimental results

RW ejW = Rt ejt + Rn ejn .

In this model, additive noise is split into two parts. The one is explicitly
represented by Rn ejn , the other is included in Rt ejt . The range lter, G1 ,

used in the BST and the EAST, intends to remove Rn ejn and preserve
Rt ejt . In the case where weak noise is superposed on strong edges, the
local orientation can be accurately estimated with Rt ejt . However, this
assumption will be violated when the image is contaminated by strong
noise. If edges are not perceptible in strong noise, the EAST and the BST
will attempt to preserve noise edges. In contrast, the CST blurs all edges
in the amplitude eld. This leads to strong noise reduction but poor edge
localization. Moreover, MAE curves shown in Fig. 4.26 also illustrate the
same issue. Angular errors generated by the EAST are small when image
noise is weak and are then quickly increased as the noise level becomes
high. For the application, current SEMs can create high quality pictures.
Image noise is not critical for inspecting cylinder bore surfaces.
4.5.3 Applications
In this section the usefulness of the EAST for surface inspection is evaluated. The SEM image shown in Fig. 3.6 is adopted for the test. Due to
the good image quality, image preprocessing
 is not needed in the experi

ments. Parameters are also set to N , N, = (18, 21, 0.2). The process of
EAST-based feature extraction is exhibited in Fig. 4.27. By observing these
feature maps, it can be seen that noise and straight edges are gradually
removed. Defects are highlighted in the defect signature map. Instead of
the EAST, the CST and the BST are also applied to the calculation of the
signature of the defect. These two variants of the detection method are
involved in the following tests.
Then, global and local detection approaches are to be quantitatively
evaluated. Since the real ground truth is not available for complex technical surfaces, defective edges and salient groove edges are manually
marked in the original image. The marker is shaped as a small window of size 2 2. Figure 4.28 demonstrates the achieved map of markers. Furthermore, an evaluation method for error analysis is visualized in
Fig. 4.29. False positive errors (FP) correspond to false alarms at groove
edges. Missed defects are regarded as false negative errors (FN). True

57

4 Defect inspection

(a)

(b)

(c)

(d)

Figure 4.27 Feature extraction. (a) g, (b) 1 , (c) 2 , (d) S .

positive detections (TP) are the correct segmentations of defects. Marked

groove edges that are not detected as defects represent true negative detections (TN). All these measures are computed taking the number of
marked pixels into account. Based on these notions, the true positive rate,
P (Defect|Defect), and the false positive rate P (Defect|Groove), are calculated as follows:

58

4.5 Experimental results

Figure 4.28 Ground truth for the SEM test image. Black markers are defective
edges, while gray markers are groove edges.

Figure 4.29 Measures for error analysis.

P (Defect|Defect) =

TP
,
TP + FN

(4.38)

P (Defect|Groove) =

FP
.
TP + FP

(4.39)

The descriptive power of the defect signature can be veried with receiver operating characteristic (ROC) curves [71]. A ROC curve is created
by relating the true positive rate with the false positive rate at different
threshold settings. Figure 4.30 shows ROC curves for the EAST-based
method, where different values of and different window sizes are used.
These curves do almost overlap. This indicates that the performance of
the EAST-based method is not signicantly inuenced by these parameters. In contrast, the CST- and the BST-based methods are instable when
the window size is changed. This issue becomes obvious in Fig. 4.31 and

59

4 Defect inspection

Figure 4.30 Evaluating the EAST-based approach in ROC curves.

Figure 4.31 Evaluating the CST-based approach in ROC curves.

Fig. 4.32. In addition, these three methods show a similar performance as

the window size is small. Since in small windows the risk for a structure
mixture can be greatly reduced, the benet of using the anisotropic lter
is not obvious. It should be pointed out that small windows are unable
to detect orientation variations at large scales. Small windows will also
lower the noise immunity of local approaches. Therefore, large windows
are preferred in the EAST-based method.
The accuracy of segmention is not emphasized in the evaluation that

60

4.5 Experimental results

Figure 4.32 Evaluating the BST-based approach in ROC curves.

uses a ground truth. The false positive errors are further evaluated
with a set of comparative studies. Figure 4.33 provides a comprehensive illustration of the detection accuracy. The condition probabilities
( P (Defect|Defect) , P (Defect|Groove)) are given under the images. In
this way, detection results at different parameter settings are associated
with numerical evaluation results. In the output images, defective locations are marked in white. It can be seen that both the true positive rate
and the false positive rate are improved as the value of and the window
size N are increased. Simultaneously, the localization of defects becomes
slightly worse. Nevertheless, the detection results are quite similar. From
this test it can be known that the overall performance of the EAST-based
detection scheme is not critically affected by the parameter settings. In
Fig. 4.34 detection results generated by the CST-based method are placed
together. As the window size is increased, both the detection rate and
the segmentation results are degraded. A false alarm rate of more than
10% is generally unacceptable for practical applications. The BST-based
method achieves a higher relative true positive rate; see Fig. 4.35. However, the false alarm rate is still too high. Moreover, both CST- and BSTbased methods yield high true positive rate at the expense of a poor localization. Hence, it can be argued that these two methods are unstable with
respect to parameter settings. In comparison, the global detection scheme
cannot achieve satisfactory result, as shown in Fig. 4.36, when the surface
quality is seriously degraded.

61

4 Defect inspection

(0.61, 0.04)

(0.66, 0.05)

(0.68, 0.05)

(0.65, 0.05)

(0.74, 0.08)

(0.77, 0.09)

(0.65, 0.05)

(0.77, 0.09)

(0.79, 0.12)

Figure 4.33 Evaluating EAST-based segmentation results. Columns: from left to

right = 0.2, 0.6, 0.9. Rows: from top to bottom N = 11, 21, 31.

62

4.5 Experimental results

(0.85, 0.18)

(0.77, 0.24)

(0.65, 0.23)

Figure 4.34 Evaluating CST-based segmentation results. From left to right N =

11, 21, 31.

(0.85, 0.16)

(0.81, 0.20)

(0.84, 0.19)

Figure 4.35 Evaluating BST-based segmentation results. From left to right N =

11, 21, 31.

(0.31, 0.01)
Figure 4.36 Evaluating the global method.

63

5 Graphite detection
The mechanical properties of engine cylinders can be optimized by incorporating vermicular graphite in cast iron. Exposed graphite particles
could fall off after test running. The empty cavities on cylinder bore surfaces result in porous structures which constitute a micro-pressure chamber system. The multi-stage honing process is not sufcient to uncover
graphite grains. A further treatment with laser irradiation is needed to ablate a very thin layer of metal from bore surfaces. After the laser exposure,
metal folds are removed. Worm-like graphite particles are expected to become visible throughout the surface. Moreover, deep honing grooves may
remain on the surface. They represent oil channels connecting pores, and
are undesired in the surface nish. Figure 3.2 shows a series-production
cylinder bore surface. Textural disturbances and non-uniform intensities
are two major problems for detecting graphite grains. In this chapter an
inspection algorithm is designed for segmenting graphite particles on
laser-exposed cylinder bore surfaces. The approach builds upon a new
understanding of how to describe the image topography in morphological scale spaces.

5.1 Related work

The metallographic characteristics of graphite cast irons have been intensively studied in material science. The mechanical properties of work
pieces are very dependent on the proportion of spheroidal, vermicular
and lamellar graphite in cast iron. Several image analysis algorithms for
graphite classication [30, 49, 57] were proposed. These algorithms quantitatively describe the shape and distribution of uncovered graphite particles. The method presented in this chapter is comparable with them in
the binarization of specimen images.
Local adaptive thresholds are popularly used to segment graphite
grains in LOM images. These techniques were specially developed to
cope with the intensity non-uniformity. The approaches for computing the threshold locally have been summarized in a survey paper
[70]. The representative methods are Niblacks, Sauvolas and Bernsens

65

5 Graphite detection

thresholds, which are parameterized with different inhomogeneity measures. The segmentation criteria proposed by Niblack and Sauvola tuned
threshold values with the local mean and the local variance in different forms. Bernsen chose the mean of the local maximum and minimum
as the threshold if the local contrast was large enough. The thresholding techniques mentioned above are effective for well-prepared specimens of graphite cast irons. In the case where surface textures are removed by polishing, a good image contrast can be easily obtained by using a bright eld illumination. The segmentation of dark graphite grains
and the bright background was tractable. However, graphite inspection
on laser-exposed surfaces may suffer from strong textural disturbances,
which result in poor segmentation results when adaptive thresholds are
used. This problem is addressed with a novel strategy based on the image
topography. The discrimination between foreground objects and background textures will be accomplished pixelwise.
In this work gray-level images are interpreted as topographic maps by
taking the image intensity as the third extent besides the spatial coordinates. In this sense, the concept behind the approach is similar to the
idea of analyzing the 2.5D surface topography. According to the literature, mathematical morphology turns out to be a powerful tool for surface characterization [60]. It can be found that the results achieved by
Decencire et al. [12] are quite close to presented approach, although their
approach was originally designed for the measurement of surface roughness. With an alternate sequential lter (ASF) [53], they decomposed the
surface topography into three elements reference surface, supercial
roughness and valleys. Analogously, the approach separates the topography of LOM images into different roughness levels. Graphite grains are
to be detected in the highest roughness level.

5.2 Morphological scale spaces

In this section a method is proposed for characterizing the image topography by multiscale morphological ltering. This section starts with the notation of the morphological operators used in this context. By analyzing
the scale-space behaviors of hat transforms, topographical shapes can be
reasonably interpreted with feature curves. Numerical shape signatures
are then derived by dual observations of morphological scale spaces. The
application to the graphite detection will be detailed in Section 5.2.1.

66

5.2 Morphological scale spaces

5.2.1 Notation
Mathematical morphology [39] is a nonlinear methodology for image
analysis. The basic morphological operations are dilation and erosion. Sequential calculation of dilation and erosion leads to the morphological
opening and closing. Considering a signal g(l ), l g , to be a function
mapping a linear coordinate into R (, + ), the dilation of g(l ) by
a at structuring element B(l ) can be denoted as:

( g B) (l ) =

{ g (lx ) + B (l lx )} ,

(5.1)

l g

where is a supremum operator. l x denotes the linear coordinate in the

neighborhood of l. Accordingly, the erosion is dened as:

( g  B) (l ) =

{ g (l ) + B (lx l )} ,

(5.2)

l g

where is an inmum operator. With the denition of dilation and erosion, the opening can be expressed as:
y o ( l ) = ( g B ) ( l ) = ( g  B B ) ( l ).

(5.3)

Then, the closing can be written as follow:

y c ( l ) = ( g B ) ( l ) = ( g B  B ) ( l ).

(5.4)

Furthermore, the top-hat transform is achieved by subtracting the opened

image from the original one,
h ( l ) = ( g g B ) ( l ),

(5.5)

and calculate the bottom-hat transform by subtracting the original image

from the closed one (in order to obtain a positive result),
h (l ) = ( g B g) (l ).

(5.6)

Note that only the peaks (valleys) smaller than the structuring element
can be detected by the top-hat (bottom-hat) transform. Therefore, the
multiscale top-hat and bottom-hat transforms can be formulated by incorporating the size of structuring elements as an additional attribute:

67

5 Graphite detection

h(l, s) = ( g g Bs ) (l ),

(5.7)

h (l, s) = ( g Bs g) (l ),

(5.8)

where s is a natural number representing the scale index of a structuring

element. Bs (l ) is computed by recursively dilating B(l ) with itself:

Bs (l ) = B B B (l ).
&
'(
)

(5.9)

s times

In particular, h(l, 0) and h (l, 0) are designated as zero, i.e.,

h(l, 0) = h (l, 0) 0.

(5.10)

In the topographic analysis, the top-hat transform was used to extract

bright features from peaks, while the bottom-hat transform was utilized
to locate dark features at valleys [44].
5.2.2 Scale-space behavior
To facilitate the visualization, an example is shown in Fig. 5.1(a). A discrete 1D signal is opened with a series of line-like structuring elements.
The length of the structuring element was sequentially selected from the
set {3, 5, 7, . . . , 21}, i.e., the scale index, s, ranged from 1 to 10. The original signal and opened signals were superposed in a single diagram. The
opening operation attened the signal by cutting down its peaks. Thus,
the opened signals can be viewed as cutting lines. Analogously, the closing operation smoothed the signal by lling up its valleys. Due to the
duality of both hat transforms similar properties are obtained from the
bottom-hat scale space. In the following, the discussions concentrate on
the top-hat scale space derived by opening.
By observing a pixel position, the traced path through the top-hat scale
space creates a bright feature curve. The amplitude increment of peaks is
equal to the falling distance of the cutting line. Thereby, the original signal
can be decomposed into several lattices. For a specied scale range larger
than the scale s 1 and smaller than the scale s, the extracted lattice represents the bright feature. Formally, the bright feature can be computed
with

68

5.2 Morphological scale spaces

(a)

(b)

Figure 5.1 Scale-space behaviors. (a) Opened signals. (b) Bright features at l0 .

Figure 5.2 Peak sharpness inuenced by the height and the width.

h(l, s) = h(l, s) h(l, s 1),

(5.11)

In a similar way, the dark feature is dened as

h (l, s) = h (l, s) h (l, s 1).

(5.12)

It should be noted that bright and dark features are not always available
at signal points because they are scale-limited. This issue can be illustrated by observing a signal point at l0 . From Fig. 5.1(b) it can be seen that
no bright features are obtained from the top-hat transform at low scales
(13), since the peak that covers l0 is broader than 7 points (equivalent to
the size of the structuring element at scale 3). Thus, the cutting line coincides with the original signal at l0 . By enlarging the size of the structuring

69

5 Graphite detection

element, the peak between scales (34) is detected until the cutting line
dropped off to a local minimum and then stops to change. The falling
process of the cutting line continues when the size of structuring element
reaches the width of a larger peak covering l0 . Thus, the magnitude of
the bright feature curve does not increase in the scale interval (45). The
same situation happens at scales (79). This scale-space behavior results
in a stair-like bright feature curve. Furthermore, It should be pointed
out that the slope of bright feature curve is related to the sharpness of
the peaks. Figure 5.2 shows this property intuitively. The high peak is
sharper than the low one as two peaks have the same bottom width (at
a xed scale). In other words, when extracting bright features from two
neighboring scales, the thickest lattice (the largest slope) of bright features will obtained from the highest peak. The analog properties can also
be obtained in the bottom-hat scale space. Combining both top-hat and
bottom-hat scale spaces enables a comprehensive description of signal
roughness.
5.2.3 Morphological stability
The characterization of image topography is normally related to observation scales. For instance, small valleys may belong to large peaks and thus
cannot be detected in large scales. In turn, small peaks located at large valleys will be neglected when large valleys are in the focus. Hence, pixels
may change their roles in the image topography with regard to the observation scale. This phenomenon often appears on complex technical surfaces, and can be explained with the fractal theory [16]. However, some
valleys or peaks are rarely confused with their counterpart, when their
size or intensity stands out in the neighborhood. The fractal characteristic
is destroyed at these places. In optical micrographs graphite grains are
always shown as isolated dark particles. They represent signicant image valleys, even if rough groove textures are distributed in their vicinity.
Utilizing this feature graphite grains can detected. For the intuition, topographical shapes are graphically illustrated in Fig. 5.3. By enlarging the
observation domain from (a) to (b), the morphological shape varies from
a peak to a valley. From this example it can be known that visual recognition is accomplished by dual observations of topographical maps, from
above and below, respectively. The visual classication of topographical
shapes can be formulated with the denitions given below. l0 represents
an observed signal point.

70

5.2 Morphological scale spaces

Figure 5.3 Scale-related morphological shapes in a rough curve.

Denition 5.1 (Observation domain) Given a domain that does not

contain any holes. is an observation domain of l0 , if and only if l0
and g . The size of represents the observation scale.
Denition 5.2 (Complete set of observation domains) A complete set
of observation domains at the point l0 is denoted as U = {k } with
k = 1, 2, 3, . . . , K, which is composed of and all its translations that
fulll Denition 5.1.
Denition 5.3 (Peaks and valleys) Let k be the boundary of k , which
is an observation domain of the point l0 . At an observed scale, the point
l0 is the member of a peak, if k U for which g (l0 ) is larger than all
values in { g (l0 ) , l0 k }. In contrary, the point l0 is a member of a
valley, if g (l0 ) is smaller than all values in { g (l0 ) , l0 k }.
Denition 5.4 (Flat and rough regions) Flat regions do not
mean absolutely constant areas, but the points that are neither recognized as peaks nor as valleys in a range of observed scales. Rough
regions refer to the points that are simultaneously recognized as peaks
and valleys.
Denition 5.5 (Morphological stability) Given a range of observation
scales, [s1 , s2 ], state a pixel is morphologically stable, if it is steadily

71

5 Graphite detection

recognized as a member of valleys, peaks or at regions; otherwise

it is located in one of the rough regions, which are morphologically
unstable.
Numerically, the visual process described above is equivalent to topand bottom-hat transforms in a scale space. In Table 5.1 the topographical
shapes are related to morphological feature curves. This idea etablishes
the foundation of the algorithm for detecting graphite grains. In the next
section a shape descriptor is developed for practical applications.
Table 5.1 Classication of topographical shapes.

Shape types

h(l, s), s [s1 , s2 ]

h (l, s), s [s1 , s2 ]

Stable peak

Inconstant

Constant

Stable valley

Constant

Inconstant

Rough region

Inconstant

Inconstant

Flat region

Constant

Constant

5.2.4 Application to shape description

It is intended to identify topographical shapes of image intensities by
pairwise observation of h(l, s) and h (l, s). However, for a numerical analysis it is not convenient to consider feature pairs. In order to merge bright
and dark features into a single measure, a signature for the morphological
stability is developed in the following form:
(l ) =
m

(l )
m(l ) m
,
(l )
m(l ) + m

(5.13)

(l ) are the steepness of feature curves, which are rewhere m(l ) and m
(l )  0. There are two options to calculate m(l )
quired to fulll m(l ) + m
(l ). The rst one is to employ the global steepness of peaks and
and m
valleys for this purpose:
m(l ) =

72

h(l, s2 ) h(l, s1 )
,
s2 s1

(5.14)

5.3 Detection scheme

(l ) =
m

h (l, s2 ) h (l, s1 )
.
s2 s1

(5.15)

Here, only the features at the lowest and the highest scales are considered
for representing the feature variations. Secondly, details of feature curves
can be taken into account:
m(l ) =

h(l, s2 ) h(l, s1 )
,
s2 s1 s0

(5.16)

(l ) =
m

h (l, s2 ) h (l, s1 )
,
s2 s1 s0

(5.17)

where s0 and s0 are the length of scale intervals in which no bright and
dark features are found. Now, the numerical criterion for classifying topographical shapes is presented:
(l ) is equal to +1 on stable peaks and 1 on stable valleys;
p m
(l ) (1, 1) indicates rough regions;
p m
(l ) is not dened on at zones where m(l ) and m
(l ) is less than
p m
a threshold Tm .
(l ) is computed, at zones should be investigated separately.
Before m
Morphological stability measures computed with the overall and detail 1 (l ) and m
2 (l ), respectively.
dependent steepness are denoted as m

5.3 Detection scheme

This section applies the developed shape descriptor to the graphite detection on laser-exposed surfaces. The usefulness of the method is explained
by modeling the surface image in a photometric point of view. In the last
step, the post-processing is designed to eliminate residual grooves from
the segmented foreground objects. Graphite grains are to be detected at
the remaining objects.

73

5 Graphite detection

5.3.1 Feature extraction

In optical micrographs, graphite grains and deep grooves are shown as
black regions. They are regarded as the foreground. The remaining image
domain including metal shining and ne grooves comprises the background. The image formation process introduced in Section 3.1 inspires
me to model LOM images of laser-exposed surfaces in a nonlinear fashion. Let g(l ) be a surface image, which can be formally written as

l ,
(l )
g(l ) =
(5.18)
i (l ) (l ) l ,
with g = , = . (l ) denotes the foreground. i (l ) decribes illumination non-uniformities. (l ) represents an oscillatory background texture. In this model, image signals are nonlinearly connected,
since physical surface structures occlude each other. Moreover, as an assumption the non-uniformity and oscillatory textures obey a multiplicative model. At low magnications, optical micrographs of laser-exposed
surfaces demonstrate the following photometric characteristics.

p Graphite grains have a non-metallic property, and residual grooves

are deep structures. Thus, little light can be reected from them into
the imaging system. Foreground objects in (l ) are approximately
invariant to the spatial illumination. Their intensities look like deep
basins in image topography.
p Oscillatory textures are composed of metallic microfacets with random orientations. These microstructures lead to clustered intensity
peaks and valleys in (l ), which are sensitive to the design of the
imaging system, e.g., the illumination mode and the specimen pose.
p In general, i (l ) varies much more slowly than other signal components. Hence, the texture signal multiplied with i (l ) still keeps the
peak-valley structure.
Figure 5.4 illustrates these characteristics based on an image row. Surface
components are manually marked in the image prole. In the following,
it can be seen that morphological stability provides a scale-space evidence
for graphite detection.
Then, top-hat and bottom-hat transforms are applied to the test image.
The structuring elements are disk-shaped with radii increasing from 1 to

74

5.3 Detection scheme

Figure 5.4 Prole of image intensities. Displayed intensities are located on the
line marked in the gray image patch. The arrows indicate the foreground objects.

10 pixels. The choice of the scale range will be explained later. Figure 5.5
shows bright and dark feature curves of some pixel samples. For the foreground objects, the bright feature remains near zero in the top-hat scale
space, whereas the dark feature increases in the bottom-hat scale space. In
the textural background, local minima and maxima are leveled randomly.
Therefore, stair-like feature curves appear in both scale spaces. By comparison, bright areas correspond to high peaks. Because of the duality of
top-hat and bottom-hat transforms, the scale-space behaviors of shining
areas are opposite to foreground objects.
In Fig. 5.5 bright and dark feature curves are shown in pairs. It can be
found that they match the characteristics described in Table 5.1. During
the foreground segmentation the emphasis is only put on intensity valleys. This allows me to rectify the signature of morphological stability by
setting its positive and undened positions to zero, that is,

(l )
m
 (l ) = 0
m

(l ) < 0,
m
(l ) 0,
m
(l ) < Tm .
m(l ) < Tm , m

(5.19)

75

5 Graphite detection

(a)

(b)

(c)
Figure 5.5 Dual observation of the image topography. (a) Valley. (b) Oscillatory
texture. (c) Peak.

76

5.3 Detection scheme

Figure 5.6 Scale selection. Columns from left to right: s1 = 1, 3, 5. Rows from
top to bottom: s2 = 10, 15, 20.

1 (l ) and m
2 (l ) are notated as m
1 (l )
Accordingly, the rectied versions of m

2 (l ), respectively. For LOM images digitized with 256 gray levels,
and m
at regions can be found by setting the threshold, Tm , to 5. This way,
signicant valleys cannot be lost by the rectication. Considering two op 1 (l ) and m
2 (l ) are combined with a mintions of the steepness measure, m
imum operator:


2 (l ) .
1 (l ), m
g (l ) = min m
(5.20)
m
l

Furthermore, the discussion comes to the inuence of scale selection.

Firstly, the size of B(l ) determines the step length for enlarging the structuring element. Given a disk-like structuring element B(l ) with a radius

77

5 Graphite detection

of r B pixels, the radius increment from Bs (l ) to Bs+1 (l ) amounts (2r B 1)

pixels. In order to reveal as much topographical information as possible, the scale resolution is set to the minimum, i.e., r B = 1. Secondly,
the scale range with a lower limit, s1 , and an upper limit, s2 , is specied
to sieve valleys of different scales. This property is shown by choosing
s1 {1, 3, 5} and s2 {10, 15, 20}. A matrix of feature maps is then obtained in Fig. 5.6. It can be seen that small particles are suppressed as s1
becomes large, whereas large valleys in non-uniform intensities are extract by increasing s2 . Therefore, the scale range should be adjusted according to the magnication of LOM images. For the optical micrographs
used in the experiments, s1 = 1 and s2 = 10 are considered to be moderate.

5.3.2 Segmentation
g (l ), is segmented with Otsus threshold. The foreThe feature image, m
ground objects are located in the segmented regions owning the lowest
g (l ). To identify graphite grains, the foreground should be furvalue of m
ther separated into grooves and graphite grains. Firstly, the morphological path opening introduced in Appendix B is used to nd elongated objects that are not perfectly straight. The image grid is taken as an oriented
graph. Paths dened in this graph are utilized as structuring elements.
This step accomplishes a preliminary separation of grains and grooves.
Since the path opening eliminates lines that contain paths larger than a
given path length, short and thin lines are retained in the grain map. To
deal with this problem, the binary maps separated by path opening are
corrected by investigating the roundness of connected objects. Elongated
objects are moved from the grain map to the groove map. Similarly, round
particles are moved from the groove map to the grain map.

5.4 Experimental results

The detection scheme for graphite grains is validated in two stages.
Firstly, the focus lies in the foreground segmentation. The approach is
compared with local thresholding approaches and the alternating sequential lter (ASF). Appendix C.2 summarizes local adaptive thresholding
methods. The ASF is dened as

78

5.4 Experimental results

(a)

(b)

(c)

(d)

(e)

(f)

Figure 5.7 Results of postprocessing. (a) and (d): Grains and grooves separated
by path opening. (b) and (e): Corrected binary maps. (c) and (f):
Grains and grooves marked in the original image.

a s ( l ) = g B B B B B ( l ).
&
'(
)

(5.21)

s times

In the experiment, s is set to 20, and B is a at disk-like structuring element of radius 1. The ultimate detection results are veried in the second
stage.
Figure 5.8 illustrates a series of foreground segmentations. Local adaptive thresholds and the ASF are resistant to non-uniform intensities, but
are unable to tackle the problem induced by background grooves. The
method is based on the fact that graphite particles, ne grooves and nonuniform intensities are present at different scales. This characteristic was
exploited to reject background disturbances. Thus, both the lighting and
textural disturbances have little impact on the segmentation result.

79

5 Graphite detection

(a)

(b)

(c)

(d)

(e)

(f)

Figure 5.8 Foreground segmentation. (a) ASF. (b) Bernsen. (c) Niblak. (d)
Sauvola. (e) Local mean. (f) Method in this work.

Moreover, the correctness of graphite detection is veried by clipping graphite grains from the original image. Figure 5.7 illustrates that
graphite grains adhered on coarse grooves are also eliminated, because
they are also a part of the oil channel system. Additionally, some short
groove fragments are still perceptible in the corrected grain map. Since
their shapes are close to porous structures, they are considered as a part
of the micro-chamber system. These experiments illustrate that the detection result can serve as a basis for an automated surface evaluation. More
results for the validation can be found in Appendix A.2.

80

6 Surface evaluation
This chapter gains an insight into the quality parameters for the evaluation of 2D surface nishes. Table 6.1 indicates some quality measures for
laser-exposed surfaces, which are mostly emphasized by engine producers. The quality measures for plateau-honed surfaces are incompletely
summarized in Table 6.2. Some applications issues for these quality measures should be noticed. Firstly, some of these measures are related with
the image contrast, the magnication or the eld of view. They describe
the visual impression of surface qualities. For this reason, an individual
quality measure is comparable among surface samples only under specied imaging conditions. Furthermore, surface images should be acquired
at different places on the cylinder wall during the non-100% inspection.
The statistical analysis of quality measures for a group of surface samples
is needed to evaluate the overall quality of cylinder bore surfaces. The
mean and standard deviation of each quality measure are computed for
the nal evaluation. Based on the detection schemes presented in this thesis, another two quality parameters are developed. The usage of the new
quality measures is as same as introduced before.
Table 6.1 Quality measures for laser-exposed cylinders.

Surface components
Grooves
Graphite grains

Quality parameters
Area of residual deep grooves.
Size of grains,
distribution uniformity,
density of grains.

81

6 Surface evaluation
Table 6.2 Quality measures for plateau-honed cylinders.

Surface components

Quality parameters [2, 14, 59, 80]

Area of metal folds,

Defects

overall degree of defectivity,

area of bubbles and blowholes.
Honing angle,
groove dispersion,
balance of groove sets,

Groove texture

amount of axial grooves,

groove width,
area of grooves and plateaus.
area of groove interrupts.

6.1 Defect severity

The proposed measure for defect severity describes the distribution of
metal folds, which can be associated with the density and strength of defective edges. Unlike the overall degree of defectivity presented in [59] the
damage of honing grooves is not considered in the defect severity. This is
advantageous for the study of individual surface components. The measure of defect severity is developed in the following formula:
#
x d 1 (x )
.
(6.1)
D=
A
The numerator is a sum of the strength of detective edges. d is the
segmented defect region. The denominator, A, is the area of a rectangular image. Without loss of generality, all results presented in this section
are measured in pixels. Figure 6.1 shows a series of surface samples visually sorted in the order from seriously defective to slightly defective.
The displayed images are of size 256 256. The detection results are also
demonstrated in the same gure. In Fig. 6.2, the plot shows the values of

82

6.1 Defect severity

Defects

Binary map

Figure 6.1 Detection results of surface samples.

83

6 Surface evaluation

Figure 6.2 Surface evaluation in defect severity.

defect severity against the sample number. It can be seen that the result
achieved by the numerical measure coincides with the visual impression.

6.2 Graphite distribution

In the segmented graphite map, isolated entities are taken as graphite
grains. These entities are labeled through the connectivity analysis according to [39]. The geometric centers of the labeled regions yield a point
pattern, which indicates grain locations. When graphite grains are clustered, lubricant oil will be unevenly distributed on the surface. Additionally, the size of graphite grains is also expected to be uniform. To
characterize the graphite distribution, spatial statistics [19] is employed.
Morans autocorrelation index [42] (generally denoted as I) is used to describe the spatial distribution of graphite grains. In this work, the corresponding formula is


# #
Nc p q w pq c p c cq c
I=

2 .
# #
p
q w pq
p c p c

84

(6.2)

6.2 Graphite distribution

(a)

(b)

(c)

Figure 6.3 Point patterns. (a) Random, I = 1. (b) Uniform, I = 0.06. (c)
Cluster, I = 0.78. Random values from [1, 100] are assigned to these
points.

(a)

(b)

(c)

Figure 6.4 Measures in Morans index. I = 0.83. (a) Voronoi diagram. (b)
Grain centers. (c) Delaunay triangles.

Here, Nc is the number of detected graphite grains. c p and cq are two

observations of a random variable c. In the present application, c denotes the size of graphite grains, and c the mean grain size. w pq is the
weight indicating the adjacency of observations at p and q. As suggested
in [24], w pq is chose to 1/d pq for close neighbors, where d pq denotes the
distance between neighboring grains. Otherwise, w pq is set to 0. Additionally, w pp is also set to 0 in the implementation. The adjacency relation
of graphite grains is dened with a Voronoi diagram. Connecting neighboring graphite centers results in a map of Delaunay triangles, where the
grain distance, d pq , is measured with the side length of these triangles.
In the theory of spatial statistics, the spatial distribution of point patterns can be classied into three types, as shown in Fig. 6.3. As long as the

85

6 Surface evaluation

variable values are inconstant at these points, the spatial relation works
in Morans index I. The expected value of Morans index I is equal to
1/( Nc 1). The uniform distribution has a high negative spatial autocorrelation, i.e., I is signicantly smaller than 1/( Nc 1) and near
to 1. The random distribution does not show any spatial autocorrelation. In this case, I is approximately equal to 0 if Nc is very large. The
cluster distribution indicates a high positive spatial autocorrelation. Thus,
its I value is close to 1. Figure 6.4 illustrates the process for implementing Morans index I. The result shows that graphite grains contained in
the test image are randomly distributed, which is desired for the laserexposed honing.

86

7 Conclusions
This thesis has presented two image-based inspection approaches for the
quality evaluation of cylinder bore surfaces. In the automotive industry,
cylinder blocks are nished by honing, which can signicantly improve
the tribological performance of engines. The surface nish has a critical
inuence on engine lubrication and the wear of pistons. Therefore, cylinder bore surfaces are manufactured into specied textures which provide the desired surface functions. The work contributes to detect surface components and characterize these components for surface quality
assessment.
Two types of cylinder bore surfaces have been investigated in this thesis. Firstly, defect detection was carried out on plateau-honed surfaces,
where honing grooves serve as oil channels in the lubrication system.
Secondly, the study concerned laser-exposed surfaces, where graphite
grains form pores for reserving lubricant in running surfaces. To evaluate grooved and porous structures, different inspection strategies are
proposed. On plateau-honed surfaces defective positions are located at
cutting edges of tool marks, which are only perceptible in highly magnied images. Thus, the scanning electron microscopy was adopted for
this purpose. The details of surface structures were explored in this part.
On laser-exposed surfaces, graphite grains are shaped as scattered particles. To assess the graphite distribution, a low image magnication is
needed to show an overview of the surface. Hence, image data used for
this inspection task were captured by a light optical microscope. The work
focused on image analysis algorithms which are suited for the aforementioned micrographs.
In the rst algorithm developed in this thesis, metal folds were considered to be manufacturing failures on plateau-honed surfaces. These
defects were actually metal burrs that could seriously wear pistons. Since
metal folds were created in form of rough edges, defects on honed surfaces can be identied according to edge shapes. This needed an accurate estimation of edge positions and orientations on complex technical surfaces. Unfortunately, current orientation detectors were not good
enough for the inspection task. To improve the performance of conven-

87

7 Conclusions

tional gradient-based methods, edges and image noise were modeled in

the squared-gradient eld. Smoothing this vector eld is fully identical
with the eigenvalue analysis of the structure tensor. The problem was
thus converted to constructing a structure-adaptive smoothing lter. The
lter kernel was designed in a bilateral form. A pair of range lters was
developed to achieve edge-preserving and orientation- selective properties. The one was attributed with gradient amplitudes, and the other relied on a set of features that were extracted by a Gabor-lter bank. Additionally, parameters for range lters were automatically selected with a
novel method, which was based on a reasonable inference of image structures. Smoothing parameters were formulated according to prole shapes
of sorted feature distances which were calculated in a local window. A
novel structure tensor was constructed by extending the developed bilateral lter to the tensor eld. It was then named as edge-aware structure
tensor. The superior characteristics of this new orientation detector were
validated with both synthetic and real images. Furthermore, a signature
of the defect to identify rough edges was designed by combining orientation dispersion and defect saliency. The nal segmentation of defective
edges was accomplished with an automatic threshold. The visual and
quantitative evaluation manifested that the proposed approach achieved
the best results in comparison with the state-of-the-art techniques.
The second algorithm was designed for detecting graphite grains in
optical micrographs. Laser-exposed surfaces are composed of spatially
occluded surface components, which show different photometrical characteristics. Surface components invariant to the illuminant were taken
as foreground objects, such as graphite grains and deep grooves, while
the remaining plateau grooves constitute a background whose appearance varies with imaging conditions. These surface components can be
described by features related to the image topography. Especially, foreground objects were always represented by deep intensity valleys in contrast to their neighborhood. This visual impression was numerically described with a metric for morphological stability. Top-hat and bottom-hat
lters were useful for characterizing the image topography from above
and below. This process tuned out to be the same as that in the visual
recognition of graphite grains. It can be also noted that topographical
shapes are scale-related. Hence, foreground objects were searched in a
specic scale range. To detect peaks and valleys in the image topography,
bright and dark features were extracted at each pixel location by hat transforms. Two morphological scale spaces obtained by relating these features

88

7 Conclusions

with the size of the structuring element. These scale spaces could signicantly illustrate topographical shapes of honed surfaces. Foreground
objects were shown as stable valleys. Morphological stability was characterized by the steepness of peaks and valleys with respect to a same
scale range. The boundaries of the detected peaks and valleys could be
effectively determined based on the concept of morphological stability.
Furthermore, the grain map was created by eliminating deep grooves
from the binarized foreground map. An advanced morphological operator, path opening, was employed to remove elongated objects in binary
images. The proposed method for detecting graphite grains was resistant to the disturbances induced by non-uniform illumination and background textures. The method was compared with local adaptive thresholding techniques and alternating sequential lters, which are commonly
used for segmenting technical surfaces. The method was able to perform
more stable segmentation of meaningful objects.
With the algorithm proposed above, a component-oriented evaluation
of surface qualities was performed instead of a global assessment of surface patches. The reason for the component-oriented evaluation can be
explained with an example. If plateau-honed surfaces are seriously damaged, expected cross-hatch textures are not successfully produced. In this
case, textural features for globally evaluating honing grooves are less
meaningful. To assess surfaces of a wide range of qualities, individual
surface components have to be taken into account. In this thesis defect
severity of plateau-honed surfaces was evaluated with the weighted spatial density of defects. The suitability of this measure was veried by
sorting a series of damaged surface samples. Ranking the surface qualities based on the proposed quantitative measures led to the same results
as the visual assessment. Moreover, the graphite distribution in laserexposed surfaces was studied. Spatial auto-correlation of grain areas was
taken as a quality measure to indicate the spatial uniformity of graphite
grains. These quality parameters also veried the application potential of
the image analysis algorithms.
In conclusion, the proposed algorithms can accurately and robustly
separate images of cylinder bore surfaces into meaningful surface components. Even very challenging surfaces are able to be correctly evaluated in the segmentation results. At present, the inspection methods introduced in this work are still limited in laboratory applications. The extension to the inspection for mass production requires imaging systems
which should be capable of generating high-quality images at large mag-

89

7 Conclusions

nications. Moreover, the EAST can be utilized for corner detection and
diffusion ltering, which are typical applications of structure tensors. The
measure of morphological stability provides a robust feature for objects
with constant intensities. It will be useful for the segmentation tasks that
rely on the size of objects rather than their intensity values.

90

A Results
A.1 Defect inspection
The section presents additional experimental results that achieved by testing a series of surface samples. Feature images as well as the nal defect
map are calculated

 with the EAST-based detection scheme. Parameters

are set to N , N, = (18, 21, 0.2). These results show that the proposed
algorithm can deal with a wide range of surface qualities.

Defects

Binary map

91

A Results

92

Defects

Binary map

Defects

Binary map

A.1 Defect inspection

Defects

Binary map

Defects

Binary map

93

A Results

94

Defects

Binary map

Defects

Binary map

A.2 Graphite detection

A.2 Graphite detection

Several surface patches are examined in LOM images showing greatly
varying illuminant conditions and background grooves. The detection
scheme turns out to be very robust to these disturbances. The proposed
method achieves good segmentation results in these experiments.

Grains

Binary map

95

A Results

g
96

Grains

Binary map

B Path openings and closings

Recently, advanced directional morphological operators, named as path
openings and closings, were proposed in [27]. The conventional morphological method for detecting narrow and elongated objects used straight
lines as structuring elements. Different from that, path openings and closings consider digital images as directed graphs. Structuring elements
were specied with graph paths of a limited length. The benets of doing so are obvious. On the one hand, line-like objects are not assumed to
be perfectly straight. Some slightly curved lines are also detectable with
path openings and closings. On the other hand, the thickness of line-like
objects is not a critical issue by using path-based structuring elements.
Both thin and thick lines can be found with a same structuring element.
Formally, path openings and closings are dened as follows.
The principle is introduced with binary images. Let G be a directed
graph endowed with an adjacent relation z  z . That means there is a
graph edge from a node z to another z . Here, z is called a predecessor
of z , and z a successor of z. With these denitions, the dilation can be
written as
.
/
({z}) = z G|z  z .

(B.1)

In other words, the dilation of a subset G comprises all points

which have a predecessor in . Furthermore, the following notations
are given. A -path of length L is dened as b = (b1 , b2 , . . . , b L ) with
bk+1 ({bk }) for k = 1, 2, . . . , L 1 . The elements of b are contained
in P (b) = {b1 , b2 , . . . , b L }. The set of all -paths of length L is denoted as
L , in which the part contained in a subset of G is notated with L (),
i.e.,
L () = {b L |P (b) } .

(B.2)

Consequently, the path opening can be dened as

L () =

{P (b)|b L ()} .

(B.3)

97

B Path openings and closings

L () is the union of all paths of length L contained in . Conversely, the

path closing is dened by exchanging the foreground and the background
in a binary map.
Path openings and closings depend on the notion of graph connectivity. Figure B.1 demonstrates the adjacencies dened in [74], which are oriented in four different directions. Combination by supremum (for openings) and inmum (for closings) makes it possible to retain paths oriented
in all possible directions just using these four adjacencies.

(a)

(b)

(c)

(d)

Figure B.1 Adjacencies in four directions. (a) East. (b) Northeast. (c) North. (d)
Northwest.

98

C Miscellaneous
C.1 Roundness
The roundness, also named as the circularity, is the most important measure for graphite classication. As dened in [56], the roundness is expressed as
roundness =

4c

,
(C.1)
D2
where c is the area of the grain, and D denotes the diameter of the smallest
circle that can fully contain the grain. The graphical illustration in Fig. C.1
intuitively explains the meaning of roundness. As a shape parameter, the
roundness is very useful for discriminating round and elongated objects.

Figure C.1 Geometrical illustration of the roundness.

99

C Miscellaneous

C.2 Local adaptive thresholds

Table C.1 Local adaptive thresholds.

Methods

Denition, x W

Bernsen

If max { g (x)} min { g (x)} > 15, T (x) =

0.5Q, where Q = max { g (x)} + min { g (x)},
else the background is segmented at Q 128,
and the foreground is segmented at Q < 128.
The window size is N = 31.

Niblak
Sauvola
Local mean

T (x) = W (x) + W (x) with = 0.2. The

window size is N = 15.


(x)
T (x) = W (x) + 1 + W

1
with =
128
0.5. The window size is N = 15.
T (x) = W (x). The window size is N = 15.

Note: for images digitalized with 256 gray levels, W (x) is the local
mean, and W (x) is the local standard deviation. W denotes the
local window of size N N

100

D Nomenclature
Symbol

Description
Chapter 2: Honed surfaces

Honing angle

v1

Horizontal speed of honing stones

v2

Vertical speed of honing stones

Chapter 3: Image acquisition

Incident angle of primary electron beam in SEM

Chapter 4: Defect detection

Vector for optimization

Objective function

1D or 2D gray-level image

Spatial horizontal coordinate

Spatial vertical coordinate

Vector spatial coordinate ( x, y)T

Local window size

Support of g

Support of the local window

TC

Classic structure tensor

T11 , T12 , T21 , T22

Elements of the structure tensor

GW

Spatial lter kernel

gx

x-derivative of g

gy

y-derivative of g

Gradient of g

||

L1 -norm

101

D Nomenclature
Symbol

Description

The rst eigenvalue of the structure tensor

The second eigenvalue of the structure tensor

det

Determinant of the structure tensor

tr

Trace of the structure tensor

Angle of the rst eigenvector corresponding to 1

Imaginary unit

Amplitude of the squared-gradient for g (x)

Angle of the squared-gradient for g (x)

Non-uniform image intensity

True texture signal in the image

Noise signal in the image

rt

Amplitude of the squared-gradient for t (x)

Angle of the squared-gradient for t (x)

rn

Amplitude of the squared-gradient for n (x)

Angle of the squared-gradient for n (x)

RW

Amplitude of the mean squared-gradient for g (x)

Angle of the mean squared-gradient for g (x)

Rt

Amplitude of the mean squared-gradient for t (x)

Angle of the mean squared-gradient for t (x)

Rn

Amplitude of the mean squared-gradient for n (x)

Angle of the mean squared-gradient for n (x)

Binary edge map

Mean squared-gradient

Bilateral lter output

GR

Range lter

Vector coordinate neighboring x

Normalization factor in the bilateral lter

102

D Nomenclature
Symbol

Description

g0

Reference image for the cross bilateral lter

Basic Gabor-lter in the spatial domain

Basic Gabor-lter in the spatial frequency domain

u, v

Coordinates of 2D Fourier domain

x , y

Standard deviations in f ( x, y)

u , v

Standard deviations in F (u, v)

Index for orientations in the Gabor-lter bank

Index for scales in the Gabor-lter bank

u0

Central spatial frequency of F (u, v)

u1

The lowest cantral frequency of scaled Gaborlters

u N

The highest cantral frequency of scaled Gaborlters

The angle of Gabor lters

Minimum angle interval of Gabor lters

Orientation number of the Gabor-lter bank

Scale number of the Gabor-lter bank

N


x ,y

Scales and rotated x and y coordinats

Scale factor

f ,

Oriented and scaled Gabor-wavelet in the spatial

domain

g ,

Output image ltered by f ,

Scale-invariant Gabor energy

Feature vector composed of all values of g

L2 -norm

G1

Range function relying on | g|

G2

Range function relying on

g

G1 2

Multiplication of G1 and G2

103

D Nomenclature
Symbol

Description

Standard deviation for G1

Standard deviation for G2

Tensor at a pixel location

TE

Edge-aware structure tensor

TB

Bilateral structure tensor

uE

Mean squared-gradient obtained from TE

Subscript for indexing variables

d1

Feature distance for computing G1

d2

Feature distance for computing G2

Sorted array of feature distances

Histogram of d

Indices of d labeled for structures similar to the

window center

Indices of d for ambiguous structures

Indices of d for structures different from the window center

Linear coordinate for array or matrix

lA

Upper limit of A

lS

Upper limit of S

lmax

Maxmum pixel number for sorting the feature distance in local windows

l1 , l2

Inection points in the simplied prole of d

Parameter in the estimation function for 2

Straight line connecting the begin and the end

points of d

Defect signature

Ground truth angle generated by smoothing

Normalization function

104

D Nomenclature
Symbol

Description

Orientation dispersion calculated by the small

eigenvalue

Image area in pixel

Ae

Area of segmented edge regions

Support of segmented edge regions

Standard deviation of image noise

Chapter 5: Graphite detection

lx

Linear coordinate in the neighborhood of l

l0

Observed signal location

Real number domain

yo

Filtered signal by morphological opening

yc

Filtered signal by morphological closing

Output of the top-hat transform

Output of the bottom-hat transform

Scale index

s1

Lower limit of the scale range

s2

Upper limit of the scale range

Structuring element

Bright feature

Dark feature

Observation domain

Complete set of the observation domain

Size of U

Boundary of

Steepness of peaks

Steepness of valleys

s0

Total length of scale intervals where no bright feature is output

105

D Nomenclature
Symbol

Description

s0

Total length of scale intervals where no dark feature is output

Morphological stability

2
1, m
m

Optional expressions of m

Foreground in g

Background texture in g

Support of

Support of

1 ,
m

Rectied m
2
m


Optional expressions of m

g
m

Modied morphological stability

rB

Radius of B

as

Output of the alternating sequential lter at the

s-th scale

Local adaptive threshold

Coefcient adjusting T

Sum of local minimum and local maximum of g

Local mean of image intensities

Local standard deviation of image intensities

Chapter 6: Surface evaluation

Defect severity

Support of segmented defective regions

Morans autocorrelation index

p, q

Indice of grains

w pq

Weight in Morans I

Grain area

c p , cq

A pair of observations of

Mean grain area

106

D Nomenclature
Symbol

Description

Nc

Number of graphite grains

d pq

Distance between a pair of graphite grains

Appendix B: Path openings and closings
Graph node as the predecessor

z
z

Graph node as the successor

Oriented graph dened on the image grid

A subset of G

Node set containing all successors

Node vector denoting a path

bk

Node in the path b

Path length

Path set containing all paths of length L

Output of the path opening

Node set containing all nodes in the path b

Appendix C: Miscellaneous

Diameter of the smallest circle that fully contains a

grain

Coefcient for scaling local adaptive thresholds

107

Bibliography
[1]

Ayres, F. J. and Rangayyan, R. M. Performance analysis of oriented

feature detectors. In: Proceedings of the XVIII Brazilian Symposium on
Computer Graphics and Image Processing. Natal, RN, Brazil, 2005.

[2]

Beyerer, J., Krahe, D., and Puente Le

on, F. Characterization of
cylinder bores. In: Metrology and Properties of Engineering Surfaces
(2001), pp. 243282.

[3]

Beyerer, J. and Puente Le

on, F. Detection of defects in groove textures of honed surfaces. In: International Journal of Machine Tools and
Manufacture 37 (1997), pp. 371389.

[4]

Beyerer, J. and Puente Le

on, F. Suppression of inhomogeneities in
images of textured surfaces. In: Optical Engineering 36.3 (1997), pp. 85
93.

[5]

Beyerer, J. and Puente Le

on, F. Adaptive separation of random lines.
In: Optical Engineering 37.10 (1998), pp. 27332741.

[6]

Bien, H., Parikh, P., and Entcheva, E. Lenses and effective spatial
resolution in macroscopic optical mapping. In: Physics in Medicine and
Biology 52.4 (2007), pp. 941960.

[7]

Boomgaard, B. T. R. van den. Adaptive structure tensors and their

applications. Tech. rep. Saarbrucken,

Germany: Universitat des Saarlandes, 2005.

[8]

Brox, T. Nonlinear structure tensors. Tech. rep. Saarbrucken,

Germany: Universitat des Saarlandes, 2004.

[9]

Chen, J., Sato, Y., and Tamura, S. Orientation space ltering for
multiple orientation line segmentation. In: IEEE Transactions on Pattern
Analysis and Machine Intellegence 22.5 (2000), pp. 417429.

[10]

Coulon, O., Alexander, D. C., and Arridge, S. Diffusion tensor

magnetic resonance image regularization. In: Medical Image Analysis 8
(2004), pp. 4767.

[11]

Davidson, M. W. and Mortimer, A. Optical microscopy. 1999. URL:

http://micro.magnet.fsu.edu/primer/opticalmicroscopy.html.

109

Bibliography

[12]

Decenci
ere, E. and Jeulin, D. Morphological decomposition of the surface topography of an internal combustion engine cylinder to characterize
wear. In: Wear 249.3 (2001), pp. 482488.

[13] Dienwiebel, M. and Scherge, M. Neue Erkenntnisse zur Tribologie von ubereutektischen

AlSiZylinderlaufachen. In: Motortechnische

Zeitschrift (MTZ) 3 (2007), pp. 186189.
[14] Dimkovskia, Z. Characterisation of worn cylinder liner surfaces by
segmentation of honing and wear scratches. In: Wear 217.34 (2011),
pp. 548552.
[15]

Dore, V., Moghaddam, R. F., and Cheriett, M. Nonlocal adaptive

structure tensors application to anisotropic diffusion and shock ltering.
In: Image and Vision Computing 29 (2011), pp. 730743.

[16]

Falconer, K. Mathematical foundations and applications. John Wiley &

Sons, Ltd., 2003.

[17]

Felsberg, M., Kalkan, S., and Kr

uger, N. Continuous dimensionality characterization of image structures. In: Image and Vision Computing
27.6 (2009), pp. 628636.

[18]

Flores, G., Abeln, T., and Klink, U. Funktionsgerechte Endbearbeitung von Zylinderbohrungen aus Gusseisen. In: Motortechnische
Zeitschrift (MTZ) 3 (2007), pp. 181185.

[19]

Fortin, M.-J., Dale, M., and Hoef, J. ver. Spatial analysis in ecology.
In: Encyclopedia of Environmetrics 4 (2002), pp. 20512058.

[20]

Freeman, W. T. and Andelson, E. H. The design and use of steerable

lters. In: IEEE Transaction on Pattern Analysis and Machine Intelligence 13.9 (1991), pp. 891906.

[21]

Fritz, A. H. and Schulze, G. Fertigungstechnik. 7. uberarb.

Au.
Berlin, Heidelberg: Springer, 2006.

[22] Geels, K. Light Microscopy, Image Analysis and Hardness testing. In:
ASTM International, 2006. Chap. Metallographic and materialographic specimen preparation.
[23] Ginkel, M. van. Image analysis using orientation space based on steerable
lters. PhD thesis. Netherlands: University of Delft, 2002.
[24] Gittleman, J. L. and Kot, M. Adaptation: statistics and a null model
for estimating phylogenetic effects. In: Systematic Zoology 39 (1990),
pp. 227241.

110

Bibliography

[25]

Han, J. and Ma, K.-K. Rotationinvariant and scaleinvariant Gabor

features for texture image retrieval. In: Image and Vision Computing 25.1
(2007), pp. 14741481.

[26]

He, K., Sun, J., and Tang, X. Rotationinvariant and scaleinvariant

Gabor features for texture image retrieval. In: 11th European Conference
on Computer Vision. Crete, Greece, 2010.

[27]

Hendriks, C. L. Constrained and dimensionalityindependent path

openings. In: IEEE Transactions on Image Processing 19.6 (2010),
pp. 15871595.

[28]

HOMMEL-ETAMIC IPS100 Innenprufsensor.

Jenoptik AG. URL:

http://www.jenoptik.com.

[29] J
ahne, B. Digtale Bildverarbeitung. Heidelberg: Springer: Springer,
2005.
[30] Jiang, H., Tan, Y., and Lei, J. F. Auto-analysis system for graphite
morphology of grey cast iron. In: Journal of Automated Methods & Management in Chemistry 25.4 (2003), pp. 8792.
[31] Jiang, X. On orientation and anisotropy estimation for online ngerprint
authentication. In: IEEE Transactions on Signal Processing 53.10 (2005),
pp. 40384049.
[32] Jiang, X. Extracting image orientation feature by using integration operator. In: Pattern Recognition 40 (2007), pp. 705717.
[33]

Kamarainen, J.-K., Kyrki, V., and K

alvi
ainen, H. Invariance properties of Gabor lterbased featuresoverview and applications. In: IEEE
Transactions on Image Processing 15.5 (2006), pp. 10881099.

[34]

Kamarainen, J.-K., Kyrki, V., and K

alvi
ainen, H. Local and global
Gabor features for object recognition. In: Pattern Recognition and Image
Analysis 17.1 (2007), pp. 93105.

[35] LaserstrukturierenVerbessern der tribologischen Eigenschaften von

Oberachen. Gehring GmbH. 2004.
[36] Lenhof, U. and Zwein, F. Messtechnik fur
Motorenzylinder. In: Motortechnische Zeitschrift (MTZ) 5 (2002), pp. 360369.
[37] Li, X. Directional Gaussian derivative lter based palmprint authentication. In: International Conference on Computational Intelligence and
Security. Suzhou, China, 2008.

111

Bibliography

[38]

Manjunath, B. and Ma, W. Textural feature for browsing and retrieval.

In: IEEE Transactions on Pattern Analysis and Machine Intelligence 18.8
(1996), pp. 837842.

[39] Maragos, P. Image and Video Processing Handbook. In: 2nd ed. Elsevier Academic Press, 2005. Chap. Morphological ltering for image
enhancement and feature Detection, pp. 135156.
[40] Mester, R. Orientation estimation: conventional techniques and a new
nondifferental approach. In: Proceeding 10th European Signal Processing Conference. 2000.
[41]

Michelet, F. Local multiple orientation estimation: isotropic and recursive oriented network. In: 17th International Conference on Pattern
Recognition. Cambridge, UK, 2004.

[42]

Moran, P. A. P. Notes on continuous stochastic phenomena. In:

Biometrika 37.1 (1950), pp. 1723.

[43] Movellan,
J. Tutorial on Gabor lters.
http://mplab.ucsd.edu/tutorials/pdfs/gabor.pdf.

2008.

URL :

[44] Mukhopadhyay, S. and Chanda, B. A multiscale morphological approach to local contrast enhancement. In: Signal Processing 80 (2000),
pp. 685696.
[45] MVplus
cylinder
inspector.
http://www.wolfsysteme.de.

Wolf

Systeme

AG.

URL :

[46] Nath, S. K. and Palaniappan, K. Adaptive robust structure tensors

for orientation estimation and image segmentation. In: Proceedings of the
First international conference on Advances in Visual Computing. Lake
Tahoe, NV, USA, 2005.
[47] Nikolaidis, N. and Pitas, I. Nonlinear processing and analysis of angular signals. In: IEEE Transactions on Signal Processing 46.12 (1998),
pp. 31813194.
[48] Oberachenscanner. Breitmeier
http://www.breitmeier.de.

Messtechnik

GmbH.

URL :

[49] Osher, J. Bildanalytische Charakteriesierung von Graphit im Grauguss

und Klassikation der Lamellenanordnung. In: Practical Metallography
40.9 (2003), pp. 454473.
[50] Otsu, N. A threshold selection method from graylevel histograms. In:
IEEE Transaction on Systems, Man, and Cybernetics 9.1 (1979), pp. 62
66.

112

Bibliography

[51]

Paris, S. Bilateral ltering: theory and applications. In: Computer Graphics and Vision 4.1 (2009), pp. 173.

[52]

Pehnelt, S., Osten, W., and Seewig, J. Vergleichende Untersuchung

optischer Oberachenmessgerate mit einem Chirp-Kalibriernormal. In:
Technisches Messen 78.10 (2007), pp. 405421.

[53]

Pei, S. C., Lai, C. L., and Shih, F. Y. An efcient class of alternating sequential lters in morphology. In: Graphical Models and Image
Processing 59.2 (1997), pp. 109116.

[54]

Petschnigg, G. Digital photography with ash and noash image pairs.

In: ACM Trans. Graph 23.3 (2004), pp. 664672.

[55]

Pouliquen, F. L. A new adaptive framework for unbiased orientation estimation in textured images. In: Pattern Recognition 38 (2005),
pp. 20322046.

[56]

Prakash, P., Mytri, V. D., and Hiremath, P. S. Classication of

cast iron based on graphite grain morphology using simple shape descriptors. In: International Journal of Engineering and Technology 2.4 (2009),
pp. 3742.

[57]

Prakash, P., Mytri, V. D., and Hiremath, P. S. Digital microstructure analysis system for testing and quantifying the ductile cast iron. In:
International Journal of Computer Applications 19.3 (2011), pp. 2227.

[58]

Puente Le
on, F. Evaluation of cylinder bores. In: Annals of CIRP 51.1
(2002), pp. 503506.

[59]

Puente Le
on, F. An objective measure of the quality of honed surfaces. In: Optical Measurement Systems for Industrial Inspection IV 5856
(2005), pp. 287295.

[60]

Puente Le
on, F. and Beyerer, J. Oberachencharakterisierung durch
morphologische Filterung. In: Technisches Messen 72 (12), pp. 663670.

[61]

Radzikowska, J. M. Metallography and Microstructures. In: vol. 9.

ASM Handbook. ASM International, 2004. Chap. Metallography
and microstructures of cast iron, pp. 565587.

[62]

Reimer, L. and Pfeerkorn, G. RasterElektronenmikroskopie. Berlin:

Springer, 1999.

[63]

Ribeiro, E. and Shah, M. Computer vision for nanoscale imaging. In:

Machine Vision and Applications 17 (2006), pp. 147162.

113

Bibliography

[64]

Robota, A. and Zwein, F. Einuss der Zylindertopographie auf den

Olverbrauch
und die Partikelemissionen eines DIDieselmotors. In: Motortechnische Zeitschrift (MTZ) 4 (1999), p. 246.

[65]

Roos, E. and Maile, K. Werkstoffkunde fur

Ingenieure: Grundlagen,
Anwendung, Prufung.

Berlin, Heidelberg: Springer, 2002.

[66]

Scheib, H. Untersuchung des Zusammenhangs zwischen Erstarrung,

Abkuhlung,

Gefuge
und mechanischen Eigenschaften von dunnwandigen

GJV Gussbauteilen. PhD thesis. Otto-von-Guericke-Universitat

Magdeburg, 2007.

[67]

Schmid, J. Optimiertes Honverfahren fur

Gusseisen-Laufachen. In:
VDI-Tagungsband Zylinderlaufbahn, Kolben, Pleue. 2006.

[68]

Schmid, J. Reibungsoptimierung von Zylinderlaufachen aus Sicht

der Fertigungstechnik. In: Motortechnische Zeitschrift (MTZ) 6 (2010),
pp. 408413.

[69]

Seck, E. and Strobel, J. DiamantFluidstrahl: Ein neues Verfahren zur

Bearbeitung der Zylinderlaufbahnen von Kurbelgehausen aus Grauguss.
In: Motortechnische Zeitschrift (MTZ) 1 (2001), pp. 184189.

[70] Sezgin, M. and Sankur, B. Survey over image thresholding techniques

and quantitative performance evaluation. In: Journal of Electronic Imaging 13.1 (2004), pp. 146165.
[71] Swets, J. A. Signal detection theory and ROC analysis in psychology
and diagnostics: Collected papers. Lawrence Erlbaum Associates, 1995.
[72]

Tomasi, C. and Manduchi, R. Bilateral ltering for gray and color

images. In: Procedings of the Sixth International Conference on Computer
Vision. Bombay, India, 1998.

[73]

Tsai, D. M. Optimal Gabor lter design for texture segmentation using stochastic optimization. In: Image and Vision Computing 19 (2001),
pp. 299316.

[74] Valeroa, S. Advanced directional mathematical morphology for the detection of the road network in very high resolution remote sensing images.
In: Pattern Recognition Letters 31.10 (2010), pp. 11201127.
[75] Voort, G. F. V. Failure analysis and prevention. In: vol. 11. ASM
Handbook. ASM International, 2002. Chap. Metallographic techniques in failure analysis, pp. 498515.

114

Bibliography

[76]

Vovk, U., Pernus, F., and Likar, B. A review of methods for correction
of intensity inhomogeneity in MRI. In: IEEE Transactions on Medical
Imaging 26.3 (2007), pp. 405421.

[77]

Wang, Y., Hu, J., and Han, F. Enhanced gradientbased algorithm for
the estimation of ngerprint orientation elds. In: Applied Mathematics
and Computation 185.1 (2007), pp. 823833.

[78]

Weidner, A., Seewig, J., and Reithmeier, E. 3D roughness evaluation of cylinder liner surfaces based on structure-oriented parameters. In:
Measurement Science and Technology 17.3 (2006), pp. 477482.

[79]

Wong, A. Adaptive bilateral ltering of image signals using local phase

characteristics. In: Signal Processing 88.1 (2008), pp. 16151619.

[80]

Xin, B. Automatische Auswertung und Charakterisierung dreidimensionaler Messdaten technischer Oberachen mit Riefentexturen, PhD
Technologie, 2008.
thesis. Karlsruher Institut fur

[81]

Zhang, L., Zhang, L., and Zhang, D. A multiscale bilateral structure tensor based corner detector. In: 9th Asian Conference on Computer
Vision. Xian, China, 2009.

[82]

Zhang, M. and Gunturk, B. K. Multiresolution bilateral ltering

for image denoising. In: IEEE Transactions on Image Processing 17.12
(2008), pp. 23242333.

[83]

Zhou, J. and Gu, J. A modelbased method for the computation of ngerprints orientation eld. In: IEEE Transactions on Image Processing
13.6 (2004), pp. 821835.

[84]

Zhou, J., Xin, L., and Zhang, D. Scale-orientation histogram for texture image retrieval. In: Pattern Recognition 36 (2003), pp. 10611063.

115

Forschungsberichte
aus der Industriellen Informationstechnik
(ISSN 2190-6629)
Institut fr Industrielle Informationstechnik
Karlsruher Institut fr Technologie (KIT)
Hrsg.: Prof. Dr.-Ing. Fernando Puente Len, Prof. Dr.-Ing. habil. Klaus Dostert
Die Bnde sind unter www.ksp.kit.edu als PDF frei verfgbar oder als Druckausgabe bestellbar.
Band 1

Prez Grassi, Ana

Variable illumination and invariant features for detecting
and classifying varnish defects. (2010)
ISBN 978-3-86644-537-6

Band 2

Christ, Konrad
Kalibrierung von Magnet-Injektoren fr BenzinDirekteinspritzsysteme mittels Krperschall. (2011)
ISBN 978-3-86644-718-9

Band 3

Sandmair, Andreas
Konzepte zur Trennung von Sprachsignalen in
unterbestimmten Szenarien. (2011)
ISBN 978-3-86644-744-8

Band 4

Bauer, Michael
Vergleich von Mehrtrger-bertragungsverfahren und
Entwurfskriterien fr neuartige Powerline-Kommunikationssysteme
zur Realisierung von Smart Grids. (2012)
ISBN 978-3-86644-779-0

Band 5
Kruse, Marco

Mehrobjekt-Zustandsschtzung mit verteilten Sensortrgern
am Beispiel der Umfeldwahrnehmung im Straenverkehr (2013)

ISBN 978-3-86644-982-4
Band 6
Dudeck, Sven

Kamerabasierte In-situ-berwachung gepulster
Laserschweiprozesse (2013)

ISBN 978-3-7315-0019-3
Liu, Wenqing
Band 7

Emulation of Narrowband Powerline Data Transmission Channels
and Evaluation of PLC Systems (2013)

ISBN 978-3-7315-0071-1

Forschungsberichte aus der Industriellen Informationstechnik (ISSN 2190-6629)

Institut fr Industrielle Informationstechnik | Karlsruher Institut fr Technologie (KIT)
Hrsg.:

Prof. Dr.-Ing. Fernando Puente Len, Prof. Dr.-Ing. habil. Klaus Dostert

Band 8

Otto, Carola
Fusion of Data from Heterogeneous Sensors with Distributed
Fields of View and Situation Evaluation for Advanced Driver
Assistance Systems. (2013)
ISBN 978-3-7315-0073-5

Band 9

Wang, Limeng
Image Analysis and Evaluation of Cylinder Bore Surfaces
in Micrographs. (2014)
ISBN 978-3-7315-0239-5

ISBN 978-3-7315-0239-5

ISSN 2190-6629
ISBN 978-3-7315-0239-5

9 783731 502395