My "Grain Motion Detection" Project

Abstract

The purpose of this study is development of software that measures the dimensions of the
grains and their speed and displacement. That is required as in the current study of
sediment transport there are no efficient ways of measuring mentioned parameters.

This work covers the experimental work in order to evaluate the efficiencies of the
currently available methods and provide the new ways of problem solution.

Some popular segmentation techniques were implemented in program to test their
efficiencies in current problem. Mostly those methods are based on the edge-detection,
some other methods were used as well.

To test the mentioned ideas it was required to construct a simplified model of the channel
bed and simulate grain movement in order to test it by current methods.

Written codes were tested under different regulation parameters and best possible results
were presented and discussed. Weaknesses of those methods were identified and those
results were the basis to select further alternative research direction.

After analysis of all the work done new alternative methods were recommended and
details regarding their implementation provided in a very descriptive way.

1

Contents

ABSTRACT............................................................................................................... 1

1.0 INTRODUCTION ........................................................................................... 6

Aims and objectives .................................................................................................................................6

2.0 LITERATURE REVIEW AND BACKGROUND INFORMATION .............. 8

2.1 Sediment definition .....................................................................................................................8

2.2 Subject Relevance .......................................................................................................................9

2.3 Solution of sediment problems ....................................................................................................9

2.4 Sediment Discharge Calculation ............................................................................................... 10

2.5 Bed Load Transport Definition ................................................................................................ 10
2.5.1 Two Schools of thought ............................................................................................................... 10

2.6 Physical description of the bed load transport ......................................................................... 11
2.6.1 General function........................................................................................................................... 11

2.7 Bed Load Transport Relations.................................................................................................. 13

2.8 Possible implementation of theory in software design .............................................................. 13
2.8.1 Direct measure method ................................................................................................................ 13
2.8.2 Estimation method that uses bed load calculation formulas ........................................................ 14

2.9 Digital image properties ............................................................................................................ 14
2.9.1 Digital Image definition ............................................................................................................... 14
2.9.1.1 Raster graphics ........................................................................................................................ 15
2.9.1.2 Important characteristics ..................................................................................................... 16
2.9.1.3 Advantages .......................................................................................................................... 16
2.9.1.4 Disadvantages ..................................................................................................................... 17
2.9.1.5 Formats................................................................................................................................ 17
2.9.1.6 Types of raster images......................................................................................................... 17
2.9.1.7 Grayscale images ................................................................................................................ 17
2.9.2 Image processing ......................................................................................................................... 18

3.0 METHODOLGY ........................................................................................... 19

3.0.1 Package used for software development ................................................................................... 19
3.0.1.1 Matlab Overview ................................................................................................................. 19
3.0.1.2 Image Processing Toolbox .................................................................................................. 20

3.1 Experimental Set up .................................................................................................................. 21
3.1.1 Programming Package features related to software design ........................................................ 23
3.1.1.1 Systems of co-ordinates ...................................................................................................... 23
3.0.1.1 List of Matlab commands and functions used in experiments. .......................................... 23
3.0.1.2 Cycle operators ................................................................................................................... 24

2

3.0.1.3 PARFOR loop ....................................................................................................................... 25
3.0.1.4 Converting image from colour to grayscale ........................................................................ 25

3.2 Proposed software working principles...................................................................................... 26
3.2.1 Proposed program concept Nr. 1 .................................................................................................. 27
3.2.1.1 Description .......................................................................................................................... 27
3.2.1.2 Background ......................................................................................................................... 27
3.2.1.3 Segmentation Method 1 ...................................................................................................... 28
3.0.1.6 Canny edge detector ............................................................................................................ 53
Conclusion on the Program Concept Nr.1 ............................................................................................ 55
3.0.2 Proposed program concept Nr. 2 ................................................................................................. 55
Description............................................................................................................................................ 55
Highlight modified areas ...................................................................................................................... 55
Results .................................................................................................................................................. 56
Conclusion ............................................................................................................................................ 57

DISCUSSIONS ........................................................................................................ 58

RECOMMENDATIONS ......................................................................................... 60

Development of the 3d surface recreation method ................................................................................ 61
3d scanners ................................................................................................................................................ 61
Three-dimensional photo .......................................................................................................................... 62
Analysis of obtained 3d data ..................................................................................................................... 65
Conclusion on proposed recommendation ................................................................................................ 67

Alternative recommendation ................................................................................................................. 67
Development of the segmentation method based on graph theory ............................................................ 67
Method description ............................................................................................................................... 67
Conclusion on proposed method........................................................................................................... 71

CONCLUSIONS OUTLINE.................................................................................... 72

LIST OF REFERENCES ................................................................................................. 73

APPENDIX A : MATLAB CODES OF THE PROPOSED METHODS .................................... 74

1. Function rem_small ........................................................................................................................ 74

2. Function rate_of_change_scan ....................................................................................................... 75

3. Function watershed_segmentation.m ............................................................................................ 76

4. Function horscan_i ......................................................................................................................... 80

5. Function vibr1_1 ............................................................................................................................. 81

6. Function imabsdiffer.m .................................................................................................................. 84

3

List of figures

FIGURE 1: TYPES OF SEDIMENT MOTION ............................................................................................. 8
FIGURE 2 – VECTOR AND RASTER FORMAT IMAGES ...................................................................... 15
FIGURE 3: VARIATIONS OF GRAYSCALE INTENSITIES ................................................................... 18
FIGURE 4: SIMPLIFIED MODEL OF THE CHANNEL ............................................................................ 22
FIGURE 5 -PIXEL SYSTEM OF COORDINATES .................................................................................... 23
FIGURE 6 : ORIGINAL COLOUR IMAGE ................................................................................................ 25
FIGURE 7: IMAGE CONVERTED TO GRAYSCALE .............................................................................. 25
FIGURE 8:PROPOSED ALGORITHM NR.1 .............................................................................................. 26
FIGURE 9: PROPOSED ALGORITHM NR.2 ............................................................................................. 26
FIGURE 12: ORIGINAL IMAGE ................................................................................................................ 30
FIGURE 13: SENSITIVITY PARAMETER “DIFFER” = 7 ........................................................................ 30
FIGURE 14: SENSITIVITY PARAMETER “DIFFER” = 10 ...................................................................... 30
FIGURE 16:NOISE REDUCED BY MEDIAN FILTERING ...................................................................... 31
FIGURE 17: NOISE REDUCED BY REM_SMALL FUNCTION ............................................................. 31
FIGURE 18-ANALYZED PIXEL LOCATIONS ......................................................................................... 32
FIGURE 19: ORIGINAL GRAYSCALE IMAGE ....................................................................................... 34
FIGURE 21: SENSITIVITY PARAMETER “DIFFER” = 7 ........................................................................ 34
FIGURE 22:SENSITIVITY PARAMETER “DIFFER” = 5 ......................................................................... 34
FIGURE 23: SENSITIVITY PARAMETER “DIFFER” = 5 ....................................................................... 35
FIGURE 24: SENSITIVITY PARAMETER “DIFFER” =9 ........................................................................ 35
FIGURE 25: OBTAINED GRAYSCALE PICTURE ................................................................................... 37
FIGURE 26:GRADIENT SEGMENTATION .............................................................................................. 38
FIGURE 27-FOREGROUND MARKING ................................................................................................... 39
FIGURE 28:IMAGE ERODE ....................................................................................................................... 39
FIGURE 29:IMAGE CLOSE ........................................................................................................................ 40
FIGURE 31:IMREGIONALMAX FUNCTION USED................................................................................ 41
FIGURE 32:IMPOSE MARKERS ON THE IMAGE .................................................................................. 41
FIGURE 33: BWAREAOPEN FUNCTION ................................................................................................. 42
FIGURE 34:THRESHOLD OPERATION ................................................................................................... 43
FIGURE 35:IWATERSHED LINES ............................................................................................................ 43
FIGURE 36:BORDERS MARKED .............................................................................................................. 44
FIGURE 37:DISPLAY THE RESULTS ....................................................................................................... 45
FIGURE 38:RESULTS IMPOSE ON ORIGINAL IMAGE ......................................................................... 46
FIGURE 39:ACCIDENTLY HIGHLIGHTED BORDERS .......................................................................... 47
FIGURE 40: GRAIN EDGES OBTAINED USING HORIZONTAL SHIFT .............................................. 50
FIGURE 41: GRAIN EDGES OBTAINED USING VERTICALL SHIFT .................................................. 50
FIGURE 42: GRAIN EDGES OBTAINED USING DIAGONAL(VERTICAL + RIGHT) SHIFT ........... 50
FIGURE 43: GRAIN EDGES OBTAINED USING DIAGONAL(VERTICAL + LEFT) SHIFT .............. 50
FIGURE 44: IMAGE BEFORE NOISE REMOVAL ................................................................................... 51
FIGURE 45: FIG.454235 .............................................................................................................................. 51
FIGURE 46: ORIGINAL GRAYSCALE PICTURE .................................................................................... 52
FIGURE 47: GRAIN BORDERS OBTAINED USING VIBRATION SIMULATION ............................... 52
FIGURE 48: METHOD 1 ............................................................................................................................. 52
FIGURE 49: METHOD 2 ............................................................................................................................. 52
FIGURE 50: ORIGINAL IMAGE ................................................................................................................ 54
FIGURE 51: THRESH = 0.05 ....................................................................................................................... 54
FIGURE 52: THRESH = 0.1 ......................................................................................................................... 54

4

FIGURE 53: THRESH = 0.2 ......................................................................................................................... 54
FIGURE 54: HIGHLIGHTED AREAS WHERE PIXEL INTENSITY VALUES HAVE CHANGED
MORE THAN PRESET CRITICAL LEVEL. ..................................................................................... 56
FIGURE 55: HIGHLIGHTED AREAS WHERE PIXEL INTENSITY VALUES HAVE CHANGED
MORE THAN PRESET CRITICAL LEVEL. ..................................................................................... 57
FIGURE 56: WEAK EDGES, METHOD1 ................................................................................................... 59
FIGURE 57: WEAK EDGES, CANNY EDGE DETECTOR ...................................................................... 59
FIGURE 58: OVER SEGMENTATION, CANNY EDGE DETECTOR ..................................................... 59
FIGURE 59: NOISE CONTAMINATION, METHOD1 .............................................................................. 59
FIGURE 60: STRUCTURE LIGHT PRINCIPLE ........................................................................................ 63
FIGURE 61: AN EXAMPLE OF TRIANGULATION PRINCIPLE ........................................................... 64
FIGURE 62: AN EXAMPLE OF RESTORATION OF THREE-DIMENSIONAL SHAPE OF A
SURFACE USING A METHOD OF THE STRUCTURED ILLUMINATION: INITIAL OBJECT
(A) AND SHAPE RECONSTRUCTION (A VIEW FROM VARIOUS ANGLES)........................... 65
FIGURE 63: AN EXAMPLE OF MODELLING OF THE IMAGE THE WEIGHED GRAPH. ................. 68
FIGURE 64: AN EXAMPLE OF A MATRIX OF PAIRED DISTANCES FOR A DOT
CONFIGURATION. ............................................................................................................................ 69
FIGURE 65: RESULTS OF WORK OF METHOD NORMALIZED CUTS( IMAGE TAKEN FROM ..... 69
FIGURE 66: RESULT OF WORK NESTED CUTS .................................................................................... 70
FIGURE 67: CONSTRUCTION OF A PYRAMID OF THE WEIGHED GRAPHS FOR THE IMAGE. .. 70
FIGURE 68: COMPARISON OF RESULTS OF WORK OF ALGORITHM SWA, ITS UPDATINGS
AND NORMALIZED CUTS .............................................................................................................. 71

5

1.0 Introduction

In a science of sediment transport there is a common difficulty of detection of moving
particles. Considering the fact that it is one of the basic properties in an accurate
description of a sediment transport rate, an efficient method of grain movement detection
needs to be developed.

According to the description of the project, required method is limited to the image
analysis; therefore an image processing techniques must be used with input images to
obtain necessary grain parameters and dimensions.

A common challenge of analysis of the objects on the image is image segmentation.
Splitting of the image into unlike areas of some sign is called image segmentation. It is
supposed that areas correspond to real objects, or their parts, and borders of areas
correspond to borders of objects. Segmentation plays the important role in problems of
processing of images and computer sight. Therefore, major and most important part of
creation of the image analysis techniques is the development of precise and effective
ways to segment an image.

Work required for development of the software, is mainly of experimental nature.
Various methods must be tested and compared in terms of their efficiencies. Several
segmentation techniques are available for testing. Most of them are based on the detection
of the edge, some other are relied on another principles of work.
But even though variety of experiments should be done to get a big picture of the
efficiency of the tested methods, as that may lead to working out an optimal solution of
the problem, based on the mentioned experimental statistics.

Aims and objectives
Main aim of the project is development of the software that will be able to effectively
obtain the required parameters of the grain particles (such as size, displacement and
velocity) in order to use that data to work out the volumetric bed load transport rate.

6

The objectives of the project are:

 Design and construction of a simplified model of a river bed to obtain required
input data for further analysis.
 Experiment with different factors that could positively or negatively influence
further analysis(such as lighting mode, use of reflector, etc.) and identify the best
configuration.
 Development and comparison of different approaches of obtaining the required
grain parameters. Test a variety of the proposed methods and based on the results
of the experiments – evaluate the efficiency and perspectives of method
implementation.
 Consider alternative approaches instead of sticking with a single method
 Analyze the obtained results to work out an optimal solution of the problem

7

2.0 Literature review and Background Information

2.1 Sediment definition
River sediment is firm mineral particles carried by a stream. River deposits are formed of
aeration products, denudation and erosion of rocks and soils. Water erosion, destruction
of a terrestrial surface under the influence of fluid waters, represents the most active
process enriching the rivers by deposits.
There are several phases of sediment movement. (Fig.1)

 When the stream velocity is low, no deposits move, but with the increase of
velocity some of the deposits begin to sporadically move by rolling and sliding.
This type of movement is called “contact load of the stream”.
 If a velocity continues to increase, individual sediments will start to make short
jumps by leaving the bed for a short amount of time, and then returning to rest
after that or continue motion by rolling or further jumping. This type of movement
is called “saltation load of the stream”.
 If a velocity increases even more, the saltation will occur more frequently. Some
of the sediments will be kept in suspension by the upward components of flow
turbulence for appreciable lengths of time. Such type of sediment movement is
called “suspended load of the stream”.

FIGURE 1: TYPES OF SEDIMENT MOTION
(HTTP://WWW.DKIMAGES.COM/DISCOVER/PREVIEWS/796/5103379.JPG)

8

2.2 Subject Relevance
The information of river bedload is extremely necessary for designing and building of
various hydraulic engineering constructions, bridge transitions, channels of different
function, dams and other constructions, to avoid and predict possible negative effects of
the sediment.
Some structures in high-velocity streams are taking damage from the sediment particles
in motion. This process can seriously wear off the surface of such structures or pavements
unless sufficient protection is provided. The damage can be caused by both smaller and
larger particles. Chief damage of this nature is to turbines and pumps where sediment-
laden water causes excessive wear on runners, vanes, propellers, and other appurtenant
parts. (Murthy and Madhaven, 1959)
Deposition in natural or artificial channels and reservoirs can also cause serious problems,
as there is always a need of removing excessive sediment for following reasons. In
navigable channels, excessive sediment needs to be removed to maintain specific depths
that are crucial for safe shipping. Considering open channel hydraulics, excessive
sediment amount in natural streams have a huge impact on channel flood-water capacity
that may result in overflow. To avoid mentioned problems, sediments are removed from
the problematic channels on a regular basis. (H.Garcia ,2008)

2.3 Solution of sediment problems
As it may first appear, optimal solution for preventing problems of sediment transport
would be stopping the erosion source and thus preventing the occurring of new
sediments. But bearing in mind huge lengths of the rivers, even if the erosion source is
stopped, huge amounts of sediment will still remain in the river for considerably long
time .Therefore it is more rational to use protection systems that will filter out the
sediments and store them in special sediment-containers or bypass sediments from the
risk areas. In some cases more radical methods like dredging are used. To obtain a
solution for the mentioned and other sediment problems, and provide an assessment of
how effectively the measure is solving the problem, there should be a clear understanding
of basic principles of sedimentation and hydraulics. (Steven J. Goldman, Katharine
Jackson, Taras A. Bursztynsky, ”Erosion and sediment control handbook”)
Basics of the sediment bedload transport principles are discussed in the next chapter.

9

2.4 Sediment Discharge Calculation
It is crucial to have efficient methods for computing sediment discharge to effectively
plan and design all the construction and maintenance works in rivers and canals. But right
now, available techniques and methods for computing sediment discharge do not allow
efficient predicting and estimating of sediment movement. In practice, engineers cannot
use these methods as main argument in making decisions. Usually they have to rely on
their own experience. There is a certain difficulty for the engineer to select a formula to
be used in calculations, because the results often vary significantly when using different
methods. And it is difficult to judge which formula to use to get the most realistic result
unless some observations and comparison of the discharge are made by an engineer.
Many formulas that engineers treat as the most useful and realistic base only on the
subjective experience of the engineer. So comparison of efficiencies of the formulas is a
big concern. (A.Vanoni 1975)

2.5 Bed Load Transport Definition

2.5.1 Two Schools of thought
There are two main schools of thought in bed load science. The author of one is Ralph
Alger Bagnold and the other one is Professor Hans Albert Einstein.
The Bagnold’s (1956) definition of the bed load transport is that contact of the particles
with the bed is mainly influenced by the effect of gravity, when the suspended particles
are lifted by random upward impulses exerted on a particle by turbulence.
Einstein gives a little bit different definition for the transport bed load. He assumed that
the bed load transport is a movement of the grains in a thin layer with a thickness of a few
grain diameters, where grains move by sliding, rolling and making small jumps for a
distance of a few grain diameters.
Einstein considered that the bed layer turbulence mixing is too small to directly influence
movement of sediments and thus suspension of the particles is not possible in the bed
layer. He assumed that any particle in the bed load travels as a series of movements the
distance of approximately 100 grain diameters independently of the flow, transport rate
and bed characteristics.
He treated saltating grains as the suspended load as the heights of jumps were much
larger than one or two grain diameters. But Bagnold (1956, 1973) believed saltation is the
main process causing the bed load transport.

10

Many further works were based on these two Schools. The key element in the formulation
of Einstein is the entrainment rate of the particles per unit area as a function of different
parameters such as shear stress and other.

2.6 Physical description of the bed load transport
As the main purpose of the project is the design of software, only basic principles that are
necessary for understanding of the topic and successful implementation of these
principles in the software design, were covered below. More detailed descriptions can be
obtained from the referenced sources.

2.6.1 General function

In general the function of volumetric bed load transport rate is a relation of a boundary
shear stress and various parameters of deposits.

= ( , )
Eq.1
It can be defined in several ways:
1. As the multiplication of grain velocity, thickness of the bed load layer and the
grain concentration.

=
Eq.2

2
 – volumetric bed load transport rate ( )

 – velocity of moving particles ( )

 – concentration of particles ( )
−

 – bed load layer thickness ( )

11

2. As the multiplication of grain velocity, grain volume and number of moving
grains per unit area

=
Eq.3

2
 – volumetric bed load transport rate ( )

 – velocity of moving particles ( )

 – volume of particles (3 )
 – number of moving grains per unit area

Velocity of the particles can be also defined as the ratio of the saltation distance and the

period of particle movement T so = /.

= can be expressed as = /
Eq.4, Eq.5

12

2.7 Bed Load Transport Relations

Most of the bed load relations can be described in a universal form:
, where ∗ is the Einstein bed load number(1950).

Eq.6

Its dimensionless form is introduced by: ∗ = , where

Eq.7

∗
∗

= and =

Eq.8, Eq.9
 is volumetric transport rate of bed load
 is the particles Reynolds number. (Transition from laminar to a turbulent
mode occurs after achievement of so-called critical number of Reynolds . At
< flow occurs in a laminar mode, at > occurrence of turbulence
is possible.)
 is submerged specific gravity of sediment.
 is the acceleration of gravity.
 is volumetric sediment entrainment rate per unit area.
 is particle travel distance
 is produced shear stress.

2.8 Possible implementation of theory in software design
Assumptions made about the way software could meet the requirements:

2.8.1 Direct measure method
The software is required to detect motion of the grains and measure the sizes of the
moving grains to work out the bed load transport rate afterwards. Thus if it is functioning
13

ideally it should be able to collect all the data necessary to work out the volumetric
transport rate by detecting each grain displacement and estimating individual moving
grain size from the top projection. (For example it can be estimated by treating each
particle as a sphere or by using Einstein‟s (1950) estimation that distance travelled by
each grain is approximately 100 grain diameters.)

2.8.2 Estimation method that uses bed load calculation formulas
In case program is not able to track the motion of the particles efficiently, it possibly
could measure the size of each grain on the picture and produce grain – size distribution.
Then by combining the grain sizes data and flow data characteristics that are measured
simultaneously, it might be possible to estimate the flow by using a variety of theories
and formulas introduced by different scientists.

2.9 Digital image properties

2.9.1 Digital Image definition
As the software input will be presented as the digital image it is very important to
understand what a digital image is and the way data is stored in digital image.
Digital image is differing by creation of the optical image on a photo sensor control
instead of traditional photographic material. The image presented in a digital way, is
suitable for further computer processing, therefore digital image often concerns to area of
information technologies.
Digital technologies are used in digital cameras and video cameras, fax and copying
devices with the various photo sensor controls, writing down and transferring analog or
digital signals. Achievements in the field of technology of photo sensor controls allow to
create digital cameras which supersede film photo technics from the majority of spheres
of application.

14

There are two basic ways of digital representation of images (fig.2):
 Raster graphics
 Vector graphics

FIGURE 2 – VECTOR AND RASTER FORMAT IMAGES

(http://www.edc.uri.edu/criticallands/raster.html)

2.9.1.1 Raster graphics

As the input for the required software will be given as the raster images, it is important to
understand principles of raster graphics.
The raster image - a grid (raster) which cells are called as pixels. The pixel in the raster
image has strictly certain site and color, hence, any object is represented by the program
as a set of the color pixels. It means that the user, working with raster images, works over
groups of pixels making them.
It is represented in the form of a considerable quantity of points - more points there are,
the better visual quality image has, file size increases accordingly. I.e. one specific picture
can be presented with a good or bad quality according to amount of points per unit of
length - the resolution (usually, points on inch - dpi or pixels on inch - ppi).
There are different types of raster images. They differ from each other in the ways of
representation and storage of color or brightness information of the pixel. Color is formed
as a result of mixing several components which can be set in various color systems (or
color spaces).

15

Term color depth is used for designation of how many bits are necessary for storage of
pixel color information. Depth of color is measured in bits per pixel.
Volume of memory, necessary for storage of raster image, can be calculated by formula:
∗ ∗
=
8
Eq.10
 - number of columns;
 - number of lines;
 - depth of color;

2.9.1.2 Important characteristics
The important characteristics of the raster image are:
 quantity of pixels. The quantity of pixels can separately be specified on width and
height (1024*768, 640*480 ...) or, seldom, total amount of pixels (usually
measured in megapixels);
 quantity of used colors (or depth of color);
 color space RGB, CMYK, XYZ, YCbCr, etc.
Raster image is edited by means of raster graphic editors. Raster images are created by
cameras, scanners, in the raster editor, also by export from the vector editor or in the form
of screenshots.

2.9.1.3 Advantages
 Raster graphics allow to create (to reproduce) practically any figure, without
dependence from complexity, in difference, for example, from vector graphics
where it is impossible to transfer precisely effect of transition from one color to
another (in the theory, certainly, it is possible, but a file with a size of 1 MB in
format BMP will have the size of 200 MB in a vector format in complex images).
 Prevalence - raster graphics is used now practically everywhere: from small
badges up to posters.
 High speed of processing of complex images if scaling is not necessary.
 Raster representation of the image is natural to the majority of devices of
input/output of the graphic information, such as monitor, printer, digital camera,
scanner, etc.

16

2.9.1.4 Disadvantages
 Big size of files with simple images.
 Impossibility of ideal scaling.
Because of these disadvantages, for storage of simple figures it is recommended to use
vector graphics instead of even compressed raster graphics.

2.9.1.5 Formats
Raster images are usually stored in compressed way. Depending on type of compression
it can be possible or impossible to restore the image to a quality what it was before
compression (compression lost-free or compression with losses accordingly). In a graphic
file, additional information can be stored as well: author-related information, a camera
and its adjustments, quantity of points per centimeter at a press, etc.

2.9.1.6 Types of raster images
There are following types of raster images, each of which is intended for the decision of
the certain range of problems:
 binary
 grayscale
 indexed
 true colour

2.9.1.7 Grayscale images
Grayscale type of images is overviewed as the input data will be presented as a set of
grayscale raster images.
So called Grayscale or Intensity images are images, pixels of which can have one of
intensity values of any color in a range from minimal up to the maximal intensity (fig.3).
Usually it is supposed that in Intensity picture, gradation of grey color is stored in a range
from black up to white. Therefore sometimes Intensity pictures are called “grey” or
images in a gradation of grey, and term “brightness of pixel” is used as a synonym of
“intensity”.

17

Currently grayscale pictures with depth of color of 8 bits/pixel have the widest
application. They can store 256 values of brightness (from 0 up to 255). Grayscale
pictures having depth of color from 2 up to 16 bits/pixel are less often used.

FIGURE 3: VARIATIONS OF GRAYSCALE INTENSITIES
(http://www.kumagera.ne.jp/kkudo/grayscale.jpg)

2.9.2 Image processing
As mentioned above, software is required to measure certain parameters of the grains in
the image. To achieve that a method called “image processing” should be used in the
required software.
Image Processing - any form of processing of the information for which the input data is
presented by the image, for example, by photos or the video frames. Processing of images
can be carried out as for getting the image on an output (for example, preparation for
polygraphic duplicating, for teletranslation, etc.), and for getting other information (for
example, recognition of the text, calculation of number and type of cells in a field of a
microscope, etc.). Except for static two-dimensional images to process it is also required
to process the images changing with time, for example video.
The variety of the purposes and problems of processing of images can be classified as
follows:
 Improvement of quality of images;
 Measurements on images;
 The spectral analysis of multivariate signals;
 Recognition of images;
 A compression of images.
(Wikipedia accessed on 03.03.2009, John C. Russ - The Image Processing
Handbook 2006)
18

3.0 Methodolgy

3.0.1 Package used for software development

3.0.1.1 Matlab Overview
MATLAB was selected for writing a code of required software due to its availability and
convenience. It has Image Processing Toolbox that makes working with images much
simpler.
MATLAB as the programming language has been developed at the end of 1970.
MATLAB (from English « Matrix Laboratory ») - the term related to a package of
applied programs for the solution of problems of technical calculations, and also to the
programming language used in this package. MATLAB is used by more than 1 000 000
engineering and science officers, it works on the majority of modern operational systems,
including GNU/Linux, Mac OS, Solaris and Microsoft Windows .
MATLAB is high-level programming language that includes structures based on matrixes
of data, a wide spectrum of the functions, the integrated environment of the development,
object-oriented opportunities and interfaces to the programs written in other programming
languages.
The programs written on MATLAB can be of two types - functions and scripts. Functions
have entrance and target arguments, and also own working space for storage of
intermediate results of calculations and variables. Scripts use the general working space.
Both scripts, and functions are not compiled in a machine code and kept in the form of
text files. There is also an opportunity to keep so-called pre-parsed programs - functions
and the scripts processed in a kind, convenient for machine execution. Generally such
programs are carried out more quickly than usual, especially if function contains
commands of construction of graphs and other figures.
The basic feature of programming language MATLAB is its wide opportunities of
working with matrixes, which developers of language have expressed in the slogan
"Think vectorized".

19

3.0.1.2 Image Processing Toolbox
Image Processing Package gives scientists, engineers and even artists a wide spectrum of
tools for digital processing and the analysis of images. Being closely connected with
environment of development of MATLAB applications, Image Processing Toolbox helps
an engineer to avoid performance of long operations of coding and debugging of
algorithms, allowing to concentrate efforts on the decision of the basic scientific or
practical problems.
The basic properties of a package are:
 Restoration and allocation of details of images
 Work with the allocated site of the image
 The analysis of the image
 Linear filtration
 Transformation of images
 Geometrical transformations
 Increase in contrast of the important details
 Binary transformations
 Processing of images and statistics
 Color transformations
 Change of a palette
 Transformation of types of images
Image Processing Package gives wide opportunities for creation and the analysis of
graphic representations in MATLAB environment. This package provides a flexible
interface, allowing to manipulate images, interactively develop graphic pictures, to
visualize data sets and to annotate results for descriptions, reports and publications.
Flexibility, connection of algorithms of a package with such feature of MATLAB as the
matrix-vector description makes a package very successfully adapted for solving any
problems in development and representation. MATLAB includes specially developed
procedures that allow to raise efficiency of a graphic core. It is possible to note, in
particular, such features:
 Interactive debugging during development of graphics;
 Profiler for optimization of time of algorithm performance;

20

 Construction tools of the interactive graphic user interface (GUI Builder) for
acceleration of development of the GUI-patterns, allowing to adjust it under goals
and problems of the user.
This package allows the user to spend much less time and forces for creation of standard
graphic representations and, thus, to concentrate efforts on the important details and
features of images.

MATLAB and Image Processing Toolbox are highly adapted for development and
introduction of new ideas and methods of the user. For this purpose there is a set of the
interfaced packages directed on the decision of every possible specific problem and
problems in nonconventional statement.
Package Image Processing now is intensively used in thousands companies and
universities worldwide. Thus there is very much a broad audience of problems, which
users solve by means of the given package, for example space researches, military
development, astronomy, medicine, biology, a robotics, materiology, genetics, etc.

3.1 Experimental Set up
A set of high-quality images of sediment had to be taken to test different ideas and
methods during the development of software. In practice, a video sample of bed load
would be made at water channels or rivers for further analysis. But for required
experiments, simplified model can be efficiently used.
Simplified model consists from(fig.4):
 Rectangular fish tank filled with water, and gravel to the point, where whole tank
bottom is uniformly covered with sediment.
 A digital camera
 A tripod
 Two lighting sources (desk lamps)
 A simple white reflector

21

FIGURE 4: SIMPLIFIED MODEL OF THE CHANNEL

A fish tank represents a section of a water channel where sediment transport occurs.
Camera is mounted on the tripod perpendicularly to the surface of water. Tripod ensures
stability of camera and thus each shot will be made from exactly the same position. This
is very important, because often, images will be analyzed in pairs and even minor image
displacements can create unnecessary problems.
Each camera shot is assumed to be one frame of the video that will be analyzed by the
software. To simulate movement of the sediment, some grains are regularly manually
moved between the shots of camera. All the images will be converted from colour to
grayscale as in reality the input images obtained from the high-speed cameras will be
given in grayscale mode.
To efficiently detect the edges of the grains it is important to create a lighting that will
leave shaded areas in gaps between the grains and to have a little effect on those shadows
in gape lighting source is placed almost parallel to the surface of sediment.
A white reflector is placed opposite to the lighting source. It is used to create dissipated
light that will help to highlight the surface of sediments and will not totally exclude
shadows from the gaps. This configuration was obtained in experimental way and proved
to be the most efficient.

22

3.1.1 Programming Package features related to software design

3.1.1.1 Systems of co-ordinates
Two coordinate systems are used in IРТ(Image Processing Toolbox) package : pixel and
spatial. In the majority of functions of a package the pixel system of co-ordinates is used,
in a number of functions the spatial system of co-ordinates is applied, and some functions
can work with both systems. It is possible to use only pixel system of co-ordinates when
writing own scenarios for the reference to values of pixels in Matlab system.
Pixel system of co-ordinates is traditional for digital processing of images. In it an image
is represented as a matrix of discrete pixels. For the reference to pixel of the image "I" it
is necessary to define a number of a line "r" and number of column "c" on crossing of
which the pixel is located: I(r,c). Lines are numbered from top to down, and columns
from left to right (fig. 5,). The top left pixel has co-ordinates (1;1). Only pixel system will
be used in the current software. Information about spatial system of coordinates can be
found in Appendix.

FIGURE 5 -PIXEL SYSTEM OF COORDINATES

3.0.1.1 List of Matlab commands and functions used in experiments.
In this part all the main used Matlab and Image Processing Toolbox functions that are
necessary for understanding the experimental codes, will be discussed:

imread
Reads the image from a file.
Function D = imread (filename, fmt) reads binary, grayscale or indexed image from a
file with a name "filename" and places it in array "D".
If MATLAB cannot find a file with a name "filename" the file with a name "filename"
and expansion "fmt" is searched. Parametres "filename" and "fmt" are strings.

23

imwrite
Writes an image in to a file
Function imwrite (S, filename, fmt) writes down a binary, grayscale or indexed image S
in to a file with a name "filename". The file format is defined by parametre "fmt".
Parametres "filename" and "fmt" are strings.

adapthisteq
Function J=adapthisteq (I) improves contrast of a grayscale picture I by transformation
of values of its elements by a method of contrast limited adaptive histogram equalization
(CLAHE).
Method CLAHE works more effective with small local vicinities of images, than with full
images. Contrast, especially on homogeneous vicinities, should be limited in order to
avoid strengthening of a noise component.

medfilt2
Function q=medfilt2 (q, [A B]) makes a median filtration by a filter kernel with a size of
AхB pixels. The filtration eliminates noise of type "salt-pepper" on the image q in a
following way: all values of pixels in working area of a kernel line up in a row by
increase of brightness and last element of a row is equated to the central.

imabsdiff
Function Z=imabsdiff (X, Y) subtracts each element of image Y from a corresponding
element of the image X and places an absolute difference of these elements in to resultant
variable Z.

3.0.1.2 Cycle operators
Similar and repeating operations are fulfilled by means of cycle operators”for “and
“while”. The cycle for is intended for performance of the predetermined number of
repeating operations, a while - for the operations with unknown number of required
repetitions, but the condition of continuation of a cycle is known.

24

3.0.1.3 PARFOR loop
During experiments a problem arised. Most of the processes in the written functions were
based on the cycles and it took more than 10 hours to process an image using standart
loops. That is the reason why PARFOR cycle was introduced.
General purpose of the PARFOR loop is to run not just one cycle, but divide current cycle
in to independent parts and run them in parallel. This results in significant increase of the
image processing speed. In current project it helped to reduce more than 10 hours
processing time to a satisfactory 3-10 minutes.
More detailed information about parfor loop can be found at Matlab help or the
Mathworks website (http://www.mathworks.com)

3.0.1.4 Converting image from colour to grayscale
As it was previously stated, colour images were made for experiments in the simplified
model of a channel. As high speed cameras that produce grayscale images are going to be
used in real-life conditions, it is necessary to convert obtained colour pictures to the
grayscale ones.
To do that, a function called “mrgb2gray” (written by Kristian Sveen, 10 Sep 2004) was
taken from the Matlab central website. Detailed description of the code can be found on
the hyperlink below.
http://www.mathworks.com/matlabcentral/fileexchange/5855.
Figures below show the original obtained colour image and converted grayscale image.
All the experiments will be done using the grayscale image .(fig.7)

FIGURE 6 : ORIGINAL COLOUR IMAGE FIGURE 7: IMAGE CONVERTED TO GRAYSCALE

25

3.2 Proposed software working principles

Detect the grain Detect the grain Calculate the
edges on first edges on second absolute
image image difference
between frames
Image 1 Image 2
and highlight the
areas where
values have
changed.
Segment and label Segment and label
each grain each grain

Identify which of
the highlighted
Compare two
areas is grain
images and
movement starting
identify grains that
point and where is
have moved
the grain stopping
point

Measure the area
of the moved
grains
Estimate area of
the grain from the
highlight of the
area and
Approximate approximate
volume of those volume of the
grains grain

Work out the Work out the
volumetric volumetric flow
movement rate of rate of the
the sediment sediment

FIGURE 8:PROPOSED ALGORITHM NR.1 FIGURE 9: PROPOSED ALGORITHM NR.2

26

3.2.1 Proposed program concept Nr. 1

3.2.1.1 Description
The main purpose of current experiment is to find out if the segmentation based on object
edge detection is efficient in current situation. Considering the fact that provided image
cannot be classified as the one that is easy to segment, due to its properties(non-
homogeneity of grains surfaces, non-sharp edges, shading, etc.) many different methods
of edge detection are used in a set of experiments.
Additionally a method of a “watershed segmentation” was tested with edge-detectors.

3.2.1.2 Background

Grain detection
In this method, grain detection and segmentation is a key problem that needs to be solved.
When appropriate solution to this problem is found, work on other blocks of the
algorithm can be started.

Segmentation
Image segmentation represents division or splitting the image on regions by similarity of
properties of their points. Primary allocation of required objects on initial grayscale image
by means of segmentation transformation is one of the basic image analysis stages. Most
widely used transformations are brightness and planimetric. Some researchers include as
well textural segmentation as one of the basic methods . According to this classification,
allocation of areas in the process of segmentation is carried out proceeding from the
conformity estimation, by some criterion of similarity of values: either brightness of each
point, or the first derivative of brightness in some specified vicinity of each point, or any
of the textural characteristics of distribution of brightness in the specified vicinity of a
point. (C.Gonsales,2004)

Edges
Edges are such curves on the image along which there is a sharp change of brightness or
its derivatives on spatial variables. Such changes of brightness which reflect the important
features of a represented surface are most interesting. Places where surface orientation
varies in steps concern them, or where one object blocks another, or where the border of

27

the rejected shade lays down on the object, or there is no continuity in reflective
properties of a surface, etc.
It is quite natural, that noisy brightness measurements limit possibility to allocate the
information on edges. Contradiction rises between sensitivity and accuracy and thus
short edges should possess higher contrast, than long that they could be distinguished.
Allocation of edges can be considered as addition to image segmentation as edges can be
used for splitting of images into the areas corresponding to various surfaces.

3.2.1.3 Segmentation Method 1

Description

This experiment was the first one and its primary purpose was to check how efficient is
the simpliest method of edge detection and secondary objective was to test how fast
matlab package is processing images using simplest algorithms.
The idea is that ideal borders have a rapid change in grayscale intensity (brightness).
Concept of the method is – comparison of gray intensities of each coherent pair of pixels
in one image in vertical , horizontal and diagonal directions. If certain predetermined
difference amount is reached – pixels are marked as the ones that belong to the grain
edge.

function horscan_i

Input

 “T” - an image or the video frame that needs to be analyzed
 “differ”- maximum intensity value difference between coherent pixels.
(sensitivity of the edge scanner)

Output

 “marked” - A variable that contains an image of the grain borders obtained by the
current method.

28

Basic steps

1) Image is read in to the variable:
T = imread('imname.bmp')
2) Image size is being measured for setting the cycle step amount later on:
siz = size(T)
3) Critical difference value between coherent pixels is set.
differ = 8
4) All the pairs of coherent pixels have to be checked for reaching critical intensity
difference value described in previous step.

a) Checking process is a cycle where each two coherent pixels are being processed
in sequence. When whole horizontal line is fully processed, next line is analyzed.
for hor = 1:(siz(1,2))
parfor ver = 1:(siz(1,1)-1)

b) The difference between coherent pixels in horizontal direction is computed, and
checked if it reaches the pre-set “differ” value. If so, pixel is marked as the edge
in a variable that stores the coordinates of the grain edges.
if (max(T(ver,hor), T(ver,hor + 1)) - min(T(ver,hor), T(ver,hor + 1))) >= differ
marked(ver,(hor)) = 1

c) The difference between coherent pixels in vertical direction and checked if it
reaches the pre-set “differ” value. If so, pixel is marked as the edge:
else if (max(T(ver,hor), T(ver + 1,hor)) - min(T(ver,hor), T(ver + 1,hor))) >= differ
marked((ver),hor) = 1

d) The difference between coherent pixels in diagonal direction and checked if it
reaches the pre-set “differ” value. If so, pixel is marked as the edge:
else if (max(T(ver,hor), T(ver + 1,hor + 1)) - min(T(ver,hor), T(ver + 1,hor + 1))) >=
differ

marked((ver),hor) = 1

29

Results

FIGURE 10: ORIGINAL IMAGE FIGURE 11: SENSITIVITY PARAMETER “DIFFER” = 7

FIGURE 12: SENSITIVITY PARAMETER “DIFFER” = 10 FIGURE 13: SENSITIVITY PARAMETER “DIFFER” = 13

As it is seen from the figures above, method works to some extent. It captures some of the
grain borders and the amount of captured borders depends on the “differ parameter”. By
decreasing this parameter, sensitivity increases and thus, more borders are captured. But
by increasing sensitivity more noise comes along with better borders.
Noise might be acceptable in this case, as long as the amount of noise particles allows
efficiently excluding it and distinguishing noise from the borders. Noise can possibly be
excluded by size, solidity and other parameters that matlab can work with.
Figure 16 below is an example of significant noise reduction even on images obtained on
high sensitivity („‟differ=7‟‟). In this case it was reduced by smoothing the original image
using function medfilt2. (Appendix A) On the Figure 16, an example of noise reduced

30

by the method of excluding objects by their area using function called rem_small is
shown. Function is written by the author of the report. (Function code can be found in
Appendix A)

FIGURE 14:NOISE REDUCED BY MEDIAN FILTERING FIGURE 15: NOISE REDUCED BY REM_SMALL
FUNCTION

It is seen that some of the grains are over contaminated by noise independently from
sensitivity parameter. The reason for that is non-homogenous surface of those grains (this
is clearly seen on the original image.

Conclusion

Despite of all the disadvantages of the method it captures some of the borders and might
find an application. It might efficiently work in combination with other methods of grain
border detection. It can either be used with low sensitivity and thus – little or no noise
particles, or with higher sensitivity, more noise but with efficient measures that separate
grain borders from noise particles.

31


Description
The idea of this experiment is detection of grain borders by comparing change rates of
pixel intensity values around the control pixel. It was assumed that rate of change of
grey-intensities changes rapidly at the point of object border.

FIGURE 16-ANALYZED PIXEL LOCATIONS

First of all, a rate of change of intensity from pixel A1 to central pixel B1 is calculated by
finding a numerical difference between current pixel values. After that, the same process
is repeated between pixels B1 and C1. Finally the rate of change on the left from the
middle pixel is compared to the rate of change on the right side of the middle pixel. If
rates of change have a reasonable difference (manually defined by the user), the middle
pixel B1 is marked as a pixel belonging to the border. The same sequence is used to
calculate vertical rates of change.

Matlab algorithm based on the proposed method

function rate_of_change_scan

Input

 T - an image or the video frame that needs to be analyzed
 differ- maximum rate of change difference, reaching which,centra pixel is
marked as the edge . (sensitivity of the edge scanner)

Output

 marked - A variable that contains an image of the grain borders obtained by the
current method.

32

Basic steps

T = imread('imname.bmp')
siz = size(T)
3) Critical rate of change of intensity is set. Less value it has – more sensitive current
edge detector is.
differ = 8
4) With the aid of cycles,
for ver = 2:(siz(1,1) - 1) ;
parfor hor = 2:(siz(1,2) - 1);
each pixel one by one becomes a control pixel and is checked for the rates of change with
coherent pixels.
a) Rate of change is calculated with the pixel on the left
ROC_left = T(ver,hor) - T(ver,hor - 1)
b) Rate of change is calculated with the pixel on the left
ROC_right = T(ver,hor) - T(ver,hor + 1)
c) Rate of change is calculated with the upper pixel
ROC_up = T(ver,hor) - T(ver + 1,hor)
d) Rate of change is calculated with the pixel on the left
ROC_up = T(ver,hor) - T(ver - 1,hor)
5) Difference of intensity change rates around the control point is calculated

a) In the horizontal plane first
ROC_diff_hor = imabsdiff(ROC_left,ROC_right)
b) Then in the vertical plane
ROC_diff_vert = imabsdiff(ROC_up,ROC_down)

6) Difference obtained in previous step is compared with the critical difference value set
by user. If the critical value is exceeded, the central control pixel is marked in a
variable “marked” that highlights the pixels belonging to the border.

33

if ROC_diff_hor >= differ
marked(ver,hor) = 1;

elseif ROC_diff_vert >= differ

Results

FIGURE 17: ORIGINAL GRAYSCALE IMAGE FIGURE 18: SENSITIVITY PARAMETER “DIFFER” = 10

FIGURE 19: SENSITIVITY PARAMETER “DIFFER” = 7 FIGURE 20:SENSITIVITY PARAMETER “DIFFER” = 5

34

It is seen on the images above that without preliminary preparation of the processed
image (such as smoothing, contrast improvement etc.), the given method highlights edges
efficiently only with big amount of noise, at high sensitivity set by the user. Situation is
very similar with results of the "Experiment 1" - by increasing sensitivity, amount of
noise increases as well, and thus similar measures need to be applied to cope with the
noise.
After a number of tests some methods of the noise reduction proved to be relatively
efficient in this case:
 Preliminary original image smoothing using function medfilt2. Map used in the
filter had size 6x6 pixels. (figure 23)
 A combination of contrast enhancement using function adapthisteq and image
smoothing using function medfilt2. (figure 24)

FIGURE 21: SENSITIVITY PARAMETER “DIFFER” = 5 FIGURE 22: SENSITIVITY PARAMETER “DIFFER” =9

Conclusion
If compared with the method described in “Experiment 1” both of these methods have
similar quality of grain edge capturing capabilities.
If using this method solely, obtained grain quality does not allow segmenting the grains
with at least a satisfactory quality. Thus current method cannot be used independently but
again, it might be combined with other edge scanners and provide those parts of edges
that an edge-detector of a different concept could not capture.

35


Description
This method is called “Marker-Controlled Watershed Segmentation”.
Development of technologies of processing of images has led to occurrence of new
approaches to the decision of problems of segmentation of images and their application at
the decision of many practical problems.

In this experiment rather new approach to the decision of a problem of segmentation of
images will be considered - a watershed method. Shortly the name of this method and its
essence wil be explained.

It is offered to consider the image as some district map where values of brightness
represent values of heights concerning some level. If this district is filled with water then
pools are formed. At the further filling with water, these pools unite. Places of merge of
these pools are marked as a watershed line.

Division of adjoining objects on the image is one of the important problems of processing
of images. Often for the decision of this problem the so-called Marker-Controlled
Watershed Segmentation is used. At transformations by means of this method it is
necessary to define "catchment basins" and "watershed lines" on the image by processing
of local areas depending on their brightness characteristics.


During this experiment, instructions described on the website given below were used.
Parts of the code were copied and used.
http://www.mathworks.com/products/image/demos.html?file=/products/demos/shipping/i
mages/ipexwatershed.html

36

function watershed_segmentation

Basic steps

1) Reading of the colour image and its transformation to the grayscale.

Reading data from a file
rgb = imread (' G:Matlabtested_image.jpg');
And present them in the form of a grayscale picture.
I = rgb2gray(rgb);
imshow(I)
text(732,501,'Image courtesy of
Corel(R)',...'FontSize',7,'HorizontalAlignment','right')

FIGURE 23: OBTAINED GRAYSCALE PICTURE

2) Use value of a gradient as segmentation function.

For calculation of value of a gradient Sobel edge mask, function imfilter and other
calculations are used. The gradient has great values on borders of objects and small (in
most cases) outside the edges of objects.

hy=fspecial (' sobel ');
hx=hy ';

37

Iy=imfilter (double (I), hy, ' replicate ');
Ix=imfilter (double (I), hx, ' replicate ');
gradmag=sqrt (Ix. ^ 2+Iy. ^ 2);
figure, imshow (gradmag, []), title (' value of a gradient ')

FIGURE 24:GRADIENT SEGMENTATION

3) Marking of objects of the foreground.

For marking of objects of the foreground various procedures can be used. Morphological
techniques which are called "opening by reconstruction" and "closing by reconstruction"
are used. These operations allow analyzing internal area of objects of the image by means
of function imregionalmax.

As it has been told above, at carrying out marking of objects of the foreground
morphological operations are also used. Some of them will be considered and compared.
At first operation of disclosing with function use imopen will be implemented.

se=strel (' disk ', 20);
Io=imopen (I, se);
figure, imshow (Io), title (' Io ')

38

FIGURE 25-FOREGROUND MARKING

Further opening using functions imerode and imreconstruct will be calculated.

Ie=imerode (I, se);
Iobr=imreconstruct (Ie, I);
figure, imshow (Iobr), title (' Iobr ')

FIGURE 26:IMAGE ERODE

The subsequent morphological operations of opening and closing will lead to moving of
dark stains and formation of markers. Operations of morphological closing are analyzed
below. For this purpose function imclose is used first:

Ioc=imclose (Io, se);
figure, imshow (Ioc), title (' Ioc ')

39

FIGURE 27:IMAGE CLOSE

Further function imdilate is applied together with function imreconstruct. For
implementation of operation "imreconstruct" it is necessary to perform operation of
addition of images.

Iobrd=imdilate (Iobr, se);
Iobrcbr=imreconstruct (imcomplement (Iobrd), imcomplement (Iobr));
Iobrcbr=imcomplement (Iobrcbr);
figure, imshow (Iobrcbr), title (' Iobrcbr ')

FIGURE - IMDIALATE

Comparative visual analysis Iobrcbr and Ioc shows, that the presented reconstruction on
the basis of morphological operations of opening and closing is more effective in
comparison with standard operations of opening and closing.

Local maxima Iobrcbr will be calculated and foreground markers recieved.

40

fgm=imregionalmax (Iobrcbr);
figure, imshow (fgm), title (' fgm ')

FIGURE 28:IMREGIONALMAX FUNCTION USED

Foreground markers imposed on the initial image.

I2=I;
I2 (fgm) =255;
figure, imshow (I2), title (' fgm, imposed on the initial image ')

FIGURE 29:IMPOSE MARKERS ON THE IMAGE

Some latent or closed objects of the image are not marked. This property influences
formation of result and such objects of the image will not be processed from the
segmentation point of view. Thus, in ideal conditions, foreground markers display
borders only of the majority of objects.

41

Some of the foreground markers cross the edges of the grains. Markers should be cleaned
and shrunk to allow further processing. In particular, it can be morphological operations.

se2=strel (ones (5, 5));
fgm2=imclose (fgm, se2);
fgm3=imerode (fgm2, se2);
As a result of carrying out of such operation the separate isolated pixels of the image
disappear. Also it is possible to use function bwareaopen which allows to delete the set
number of pixels.

fgm4=bwareaopen (fgm3, 20);
I3=I;
I3 (fgm4) =255;
figure, imshow (I3)
title (' fgm4, imposed on the initial image ')

FIGURE 30: BWAREAOPEN FUNCTION

4) Calculation of markers of a background.

Now operation of marking of a background will be performed. On image Iobrcbr dark
pixels relate to a background. Thus, it might be possible to apply operation of threshold
processing of the image.

bw=im2bw (Iobrcbr, graythresh (Iobrcbr));
figure, imshow (bw), title (' bw ')

42

FIGURE 31:THRESHOLD OPERATION

Background pixels are dark, however it is impossible to perform simple morphological
operations over markers of a background and to receive borders of objects which are
segmented. Background will be "thinned" so that in order to receive an authentic skeleton
of the image or, so-called, the grayscale picture foreground. It is calculated using
approach on a watershed and on the basis of measurement of distances (to watershed
lines).

D=bwdist (bw);
DL=watershed (D);
bgm=DL == 0;
figure, imshow (bgm), title (' bgm ')

FIGURE 32:IWATERSHED LINES

43

5) Calculate the Watershed Transformation of the Segmentation Function.

Function imimposemin can be applied for exact definition of local minima of the image.
On the basis of it function imimposemin also can correct values of gradients on the image
and thus specify an arrangement of markers of the foreground and a background.

gradmag2=imimposemin (gradmag, bgm | fgm4);
And at last, operation of segmentation on the basis of a watershed is carried out.

L=watershed (gradmag2);
Step 6: Visualization of the processing result

Displaying the imposed markers of the foreground on the initial image , markers of a
background and border of the segmented objects.

I4=I;
I4 (imdilate (L == 0, ones (3, 3)) |bgm|fgm4) =255;
figure, imshow (I4)
title (' Markers and the borders of objects imposed on the initial image ')

FIGURE 33:BORDERS MARKED

As a result of such display it is possible to analyze visually a site of markers of the
foreground and a background.

44

Display of results of processing by means of the colour image is also useful. The matrix
which is generated by functions watershed and bwlabel, can be converted in the
truecolor-image by means of function label2rgb.

Lrgb=label2rgb (L, ' jet ', ' w ', ' shuffle ');
figure, imshow (Lrgb)
title (' Lrgb ')

FIGURE 34:DISPLAY THE RESULTS

Also it is possible to use a translucent mode for imposing of a pseudo-colour matrix of
labels over the initial image.

figure, imshow (I), hold on
himage=imshow (Lrgb);
set (himage, ' AlphaData ', 0.3);
title (' Lrgb, imposed on the initial image in a translucent mode ')

45

FIGURE 35:RESULTS IMPOSE ON ORIGINAL IMAGE

A combination of contrast enhancement function adapthisteq and image smoothing
function medfilt2 was used as preliminary processing as under current conditions
segmentation result proved to be the best from all the tested ones.

Conclusion

It is seen from the Fig.38 that only few of all grains are segmented properly. Some of the
captured grains are over segmented. Such unimpressive grain capture capability can be
explained by non uniform lighting, absence of the well-defined background and non-
homogenous surface of the most grains.
Method might work with different efficiency, depending on variety of conditions (such as
lighting, background etc.)


Description

This method is very interesting as it was discovered by accident.
During experiments on the method of the pixel value difference (method is described
starting from page 55), when comparing 2 coherent frames and highlighting the areas

46

values of which have changed, it was noticed that the grain borders are highlighted at
some point as well (fig.39).

FIGURE 36:ACCIDENTLY HIGHLIGHTED BORDERS

After analysis of the reasons that caused the highlight of grain borders, it was noticed that
processed coherent frames were made from a little bit different perspectives. Camera was
accidently moved by few millimeters when pressing an image-capture button and that
caused vibration of the secondary image in relation to the first image.
It was assumed that such vibration can be simulated in matlab environment to construct
an edge detector based on that idea.
The main idea is to shift the same image for a few pixels, compare it with original image
and highlight the difference of the pixel values that are exceeding critical value set by the
user. If picture will be smoothed preliminary, the grains will be relatively homogenous,
and the grain edge lines could be highlighted with a little noise..


function [ dif ] = vibr1_1(imnam1,pix_mov,pix_difference,rem_area)

Input
imnam1 = image name
rem_area = area of particles to be filtered out

47

pix_move = distance to move pixel
pix_difference = critical pixel difference for highlighting, when comparing two frames.
Output
dif = variable containing grain borders obtained by current method

Basic steps

Im1= imread(imnam1)
siz = size(T)
3) Critical difference value between coherent pixels is set.
differ = 8

4) Shifted images are created and saved in to predetermined variables.

a) The process of the image shifting is a cycle that one by one shift pixels in all
possible directions and saves them into different variables to further analyze and
combine the borders obtained using all the directions of image shift.

for ver = (pix_move+1):(ver_size - pix_move)
pix_move_ver_down = ver - pix_move
pix_move_ver_up = ver + pix_move
parfor hor = (pix_move+1):(hor_size - pix_move)

b) Image shifted in horizontal direction:
im2(ver,hor + pix_move) = img1(ver,hor);
c) Image shifted in vertical direction:
im3(pix_move_ver_up,hor) = img1(ver,hor);
d) Image shifted in diagonal direction (vertically + right):
im4(pix_move_ver_down,hor + pix_move ) = img1(ver,hor);
e) Image shifted in diagonal direction (vertically + left):
im5(pix_move_ver_up,hor + pix_move) = img1(ver,hor);

48

5) Absolute pixel differences are calculated between original and shifted images
a) Between original and horizontally shifted image
im_difference1_2 = imabsdiff(im1,im2);
b) Between original and vertically shifted image
c) Between original and diagonally(vertically + right) shifted image
d) Between original and diagonally(vertically + left) shifted image
6) Difference between images is checked for reaching the critical value. If so the pixel is
marked as the border one.

a) Current process is a loop,
where pixels are sequentially checked for reaching of critical value predetermined by the
user.
Variable pix_move is used to exclude grain border doubling that is caused by image
shifting.
b) Check differences of all the images for for reaching the critical value and mark
the required pixels.
if im_difference1_2(ver,hor) >= pix_difference
bi_im1_2(ver,hor - pix_move) = 1;

elseif im_difference1_3(ver,hor) >= pix_difference
bi_im1_3(pix_move_ver_down,hor) = 1;

bi_im1_4(pix_move_ver_up,hor - pix_move) = 1;

bi_im1_5(pix_move_ver_down,hor - pix_move) = 1;
49

On the figures below(Fig.40,Fig.41,Fig.42,Fig.43) grain edges obtained by different shift
directions can be seen.

FIGURE 37: GRAIN EDGES OBTAINED USING FIGURE 38: GRAIN EDGES OBTAINED USING
HORIZONTAL SHIFT VERTICALL SHIFT

FIGURE 39: GRAIN EDGES OBTAINED USING FIGURE 40: GRAIN EDGES OBTAINED USING
DIAGONAL(VERTICAL + RIGHT) SHIFT DIAGONAL(VERTICAL + LEFT) SHIFT

7) From the figures above, it is seen that obtain edge part images contain some amount
of noise. As all of those grain edge parts will be put together later on it is preferred to
remove the noise to avoid noise accumulation on the final image.

a) To do that, a function rem_small was written(Appendix A).
50

 First of all this function labels all the objects in the image
 Then it measures each objects area
 Finally it excludes the objects with area that is less than predetermined value

b) Noise is removed from each image using function rem_small described above
bi_im1_2_mod = rem_small(bi_im1_2,rem_area)

Example of the noise removal result can be seen on the images below. Fig.44 is image
before noise removal and Fig.45 is the resultant image.

FIGURE 41: IMAGE BEFORE NOISE REMOVAL FIGURE 42: FIG.454235

8) All the grain edge parts now have been cleaned from the noise and are put together in
one image. The resultant image is cleaned from the noise again.

a) Simple summation of the matrices is used to create a final image containing grain
borders
bi_im_final = bi_im1_2_mod + bi_im1_3_mod + bi_im1_4_mod + bi_im1_5_mod

b) Noise is removed from the resultant image using function rem_small
bi_im_final = rem_small(bi_im_final,rem_area)

51

Results
Original grayscale and combined final border images are shown at figures below

FIGURE 43: ORIGINAL GRAYSCALE PICTURE FIGURE 44: GRAIN BORDERS OBTAINED USING
VIBRATION SIMULATION

FIGURE 45: METHOD 1 FIGURE 46: METHOD 2

It is seen that proposed method capture some of the grains ideally, some of the grain
borders are not fully highlighted though. Amount of noise is very small. If compared with
best quality edges captured with method 1 and method 2 (Fig. 42,43) , it is seen that
edges captured by current method (fig.47) are quite stronger and have significantly less
noise.

52

Conclusion
Method produced comparatively good edge detection results but still not enough to
segment grains with enough quality to use the segmentation in further analysis of
sediment discharge. It still might provide a reasonable segmentation if used in
combination with other methods. From the experiments described above it is possible to
assume that efficiency of current method in terms of detected edge strength and amount
of noise, is higher than Method 1 and Method 2 efficiencies.

3.0.1.6 Canny edge detector

Description
It is quite complex method consisting of the big number of stages. The method essence is
searching for local sites with brightness differences. Brightness differences are searched
by means of a filtration along each of axes by the one-dimensional Laplacian-Gaussian
filter. In Canny method for classification of differences on "weak" and "strong" two
thresholds are used - bottom and top. "Weak" borders are marked in resultant image, only
if they are connected to "strong". For noisy images the given method provides the best
detection of borders in comparison with other methods but it demands more time.
[J.Cаnny, 1986. А computаtional Аpproach to Еdge Detectiоn, IEEЕ Trаns. Pаttern
Anаlysis аnd Mаchine Intelligеnce].

Matlab function BW=edge(I, 'canny', thresh)
BW = еdgе(I,'cаnny',thrеsh)
The parameter thresh can be a two-element vector. In this case the first element of a
vector sets value of the bottom threshold, and the second element - value of the top
threshold. If the parameter "thresh" is scalar value "thresh" sets value of the top threshold,
and for the bottom threshold value- (0.4*thresh) is used. If the parameter "thresh" is not
given or "thresh" is an empty array, values of thresholds are defined automatically.

Results
Images obtained by Canny edge detector are either over segmented (fig. 51,52) or with
weak edges( fig,5). Oversegmentation is probably caused by the high efficiency of the
scanner. In experimental way it was figured out that there is no “thresh” sensitivity value
53

that would allow to avoid oversegmentation and capture the grain borders at satisfactory
level.

FIGURE 47: ORIGINAL IMAGE FIGURE 48: THRESH = 0.05

FIGURE 49: THRESH = 0.1 FIGURE 50: THRESH = 0.2

Conclusion
Based on this experiment results, it can be stated that Canny filter is too efficient for the
image of such type. If sensitivity parameter “thresh” reaches level when a grain edges can
be detected efficiently, apart from that each single detail on the grain is segmented as
well. This causes oversegmentation that gives no possibility for further analysis of the
picture.
54

Conclusion on the Program Concept Nr.1

On a flow diagram on page (fig.9) it is seen that primary step of the proposed algorithm is
detection of the grain borders. Methods of detection that work to some extent were found
during a set of experiments described above. Unfortunately quality of the obtained
borders and segmentation does not allow proceeding to the next step of the proposed
algorithm. Probable reason for that is combination of conditions that are seriously
complicating the segmentation.(lighting conditions, non – homogenous surface of the
grains, absence defined background, shadows etc.)
It is not stated that grain segmentation and edge detection is impossible task in this case,
but based on the results, it is probably achievable only using extremely complicated
image processing techniques.
Before tackling such serious task it is recommended to consider other methods that might
be easier to achieve.

3.0.2 Proposed program concept Nr. 2

Description
It is assumed that the areas where the grains have moved will have their intensity values
relatively different, when comparing coherent frames where movement has occurred. T
The idea is to capture and highlight such areas as ideally these areas have the contours of
the moved grains. After obtaining the direction of grain movement and estimation of
volume, volumetric bed load could be estimated.

Highlight modified areas

1) Absolute difference is obtained using imabsdiff function
im3 = imabsdiff(im1,im2)
2) Values obtained in previous step with aid of cycles:
for ver = 1:(siz (1,1))
parfor hor = 1:(siz(1,2))

55

3) Each pixel is sequentially compared to a critical value pix_difference
predetermined by the user. If value is exceeded a pixel is highlighted in a variable
bi_im
if im3(ver,hor) >= pix_difference
bi_im(ver,hor) = 1

Results

During the experiment, only one grain was displaced manually to simulate the effect of
grain movement in order to check the possibility of capturing grain contour efficiently.

FIGURE 51: HIGHLIGHTED AREAS WHERE PIXEL INTENSITY VALUES HAVE CHANGED MORE THAN PRESET
CRITICAL LEVEL.

56

FIGURE 52: HIGHLIGHTED AREAS WHERE PIXEL INTENSITY VALUES HAVE CHANGED MORE THAN PRESET
CRITICAL LEVEL.

Based on the results (on figures54,55) it is seen that highlighted difference areas( in red
circles) at some point, have contours similar to the contours of the grains.

Conclusion
Considering amount of noise, coming with highlighted contours there will be difficulties
obtaining area of the grains efficiently. The most probable reason for noise is a change of
shading, caused by grain movement and non homogenous surface of the grains.
Thus it is not likely that a moved grain area could be approximated by this method.

Alternative application of the method
It was concluded that this method may be used in a way it was not designed for:
Method does not allow obtaining precise grain area information easily, but it is surely
efficiently marks the areas where changes have occurred, and thus there is no need to
process whole image, when area of interest can be reduced significantly using this
method.

57

That reduction of interest area can significantly simplify the image processing
(segmentation, edge detection, etc) and identification of the moved grains and their
position.
For example, instead of trying to identify a moved grain on the image of 1000 grains, this
number could be reduced to 10 grains and logically it would be much easier to identify
the required grain in that case. As on the smaller area of interest there are less possible
variants of a moved grain, thus the probability of the incorrect grain identification (error)
is also significantly reduced.
Method also might find an application when detecting the grains that did not move, but
only produced small “shakes” as all the displaced grains will be highlighted.

Accidental discovery
It should be also mentioned that when taking pictures for that experiment, a vibration of
the camera was accidently induced by pressing the image capturing button. As a result,
when highlighting the difference between frames, most of grain borders were noticed to
be highlighted with a relatively good quality. (it can be seen on Fig.39)
Mentioned accidently discovered useful effect of the vibration became the basic working
principle for the grain edge-detector described starting from page 44.

Discussions

During analysis of the experiment results described above, a following pattern was
noticed -during tests of the segmentation methods based on edge detection it was
observed that when analyzing the images at relatively low sensitivities at the beginning of
experiments, exactly the same parts of the grain edges often could not be captured even
when different methods of grain detection are used. Method 1 on figure 56 and canny
filter on figure 57 are shown as an example of the edges obtained using low edge-detector
sensitivities)
This might be an evidence of extremely weak preciseness of undetected edges.

58

FIGURE 53: WEAK EDGES, METHOD1 FIGURE 54: WEAK EDGES, CANNY EDGE DETECTOR

FIGURE 55: OVER SEGMENTATION, CANNY EDGE FIGURE 56: NOISE CONTAMINATION, METHOD1
DETECTOR

After further increase of each methods sensitivity, there was a moment when excessive
amount of noise or over segmentation (depending on the method) appeared on the image.
(Fig.58 and 59)

Even though the images were over contaminated by noise or excessively
segmented(fig.58,59), some edges were still unrevealed. Considering the fact, that
mentioned observations were obtained using methods based on completely different
working principles (Method 1, Canny edge detector), this might be another confirmation
of the mentioned excessively weak edges hypothesis.

59

Based on the experimental results, majority of the tested segmentation and grain detection
methods proved to be inefficient in such conditions. The reason for that might be a huge
combination of negative image and grain features that complicate the analysis, for
example
 Non homogeneity of grain surface
 Absence of the background below the sediment
 Input data is presented only in grayscale mode
 Shadows induced by other grains
 Lighting conditions
 And many other negative features

A proposed “difference highlight” method (function “imdiffer.m”), appeared to be
inefficient in the way it was planned to be used. But presumably it might find an
application as the method that marks the area of interest and thus – significantly simplify
current problem. Efficiency of the method still needs to be tested in a set of experiments
by combining it with other algorithms. Possible implementation of the current algorithm
is discussed in the recommendation part on the page 65.
Considering the inefficiency of the techniques described above it is possible to assume
that very sophisticated methods of image processing need to be used to solve current
problem efficiently. Before starting very energy and time consuming process of the
sophisticated methods development it is rational to make that absolutely no other options
are available.
This was a reason to re-analyze the problem and consider different approach that might
help to avoid development of unnecessary complicated image processing techniques. A
totally different approach to the problem is discussed in the next section. Some advices
will be given about the further steps that will most likely solve the problem in most
optimal and energy-saving way.

Recommendations
As it was mentioned, results of the described experiments at some point can testify that
grain parameters that an image of the grain contains (Parameters like : top grain
projection, grain colour information, features of the grain surface, etc.) are insufficient to

60

process grains (segment grains, detect edges, obtain grain parameters,etc.) with high
degree of certainty by means of a simple tehniques.
Unlike the situation with images, in real life sediment grains have many more parameters,
such as:
 Grain width, length and heigth
 Grain colour information
 Grain density
 Friction factor of the grain surface
 etc.
As the main idea of the project is processing of the images, it may first appear that
analysis is limited to the mentioned grain parameters available from the 2 dimensional
image. Logically, the grain analysis would be much more efficient and simpler if
grain data would contain and could be classified by more grain parameters apart from
those that are currently contained by the images.
An alternative approach to the problem is suggested. It is both aimed on getting an
additional grain parameter that would be helpful in solving the problem, and on
sticking with a requirement of using only images for analysis.
It is proposed to recreate a 3 dimensional surface of the analysed area, by obtaining
the only missing parameter for a 3d representation of the object – height information.
Thus some sort of 3d scanner have to be used to recreate the surface.

Development of the 3d surface recreation method

3d scanners
3D scanner - is the device that analyzes a physical object and on the basis of received data
is creating its 3D model.
3D scanners have 2 major types of a scanning method:
 Contact, such method is based on direct contact of the scanner with investigated
object.
 not Contact
Not contact scanners in turn can be classified in 2 separate categories:
 Active scanners
 Passive scanners

61

Active scanners radiate some directed waves (more often light, a laser beam) on the
object and find out its reflection for the analysis. Possible types of used radiation include
light, ultrasound or X-rays.
Passive scanners do not radiate anything on object, and instead rely on detection of the
reflected surrounding radiation. The majority of scanners of such type detect visible light
- readily available surrounding radiation.
3D models received by a method of scanning can be further processed and used in
engineering calculations.

Considering the fact that project is limited to image processing and those images(frames)
are obtained by a video recording, it is impossible to use scanners that involve any
moving parts , therefore it is proposed to use a scanner based on the principle of a
structured light. The method of structured illumination is described in a following
chapter.

Three-dimensional photo
It is known, that the traditional photo provides reception of two-dimensional projection of
three-dimensional objects, the information on object along the third co-ordinate is thus
lost. In some cases it is possible to restore the lost information, for example, using laws of
distribution of brightness in pictures of semitones , however generally it does not provide
unambiguity of received results.
Methods of definition of the shape of three-dimensional surfaces on the basis of optical
stereosystems which can be classified as "passive systems" are known. Along with
advantages (simplicity and rather low cost), such systems have serious lacks - dependence
of results on character of object and illumination conditions, sensitivity to influence of the
background flares, complexity of mathematical methods and computer algorithms of
processing, etc.
During last years, systems of "active" type were widely used, in which the object was
projected with a source of radiation with known properties that allowed to overcome the
lacks of traditional methods specified above. Use of various methods of modulation of
radiation, more often on amplitude or frequency, allows to realise a mode of a laser range
finder for restoration of a three-dimensional relief of a surface.

62

Active development of computer technologies and perfection of technology of
multielement radiation recievers (videocameras) have allowed to replace
electromechanical scanning by electronic scanning and has created preconditions of
working out of methods of restoration of the three-dimensional form of objects by more
simple means on the basis of «projection of lines» principle, generated in coherent or not
coherent light and «the structured light» - illumination of objects by not coherent
radiation with the set spatial distribution of brightness.
The principle of formation of images using structured light is shown On Fig.60

FIGURE 57: STRUCTURE LIGHT PRINCIPLE

Restoration of three-dimensional form of objects using structured illumination is based on
a triangulation principle: at a known mutual angular arrangement of axes of a source
(projector) and a videocamera, predefined pattern is projected on the object in the
cameras field of view.

63

FIGURE 58: AN EXAMPLE OF TRIANGULATION PRINCIPLE

At illumination of object by a set of "patterns" with known distribution of the brightness,
each "pattern" is deformed according to a surface relief, and the distance to each point of
a surface is represented as binary code.
Decoding operation allows to receive an estimation of distance to each point of a surface.
At a projection of parallel strips the picture of strips changes according to the form of a
covered surface. In the right part of fig. 61 an example of function of conformity for one
horizontal line is shown.
Not-monotony of a relief of the real objects, caused, for example, by sharp jumps or
shading of separate sites, can lead to ruptures of the function of conformity. For
overcoming of this problem conformity function is analyzed on its separate segments.
Methods of the structured illumination allow to restore the shape of three-dimensional
objects of a difficult kind. One of examples of restoration is illustrated on fig. 62

64

FIGURE 59: AN EXAMPLE OF RESTORATION OF THREE-DIMENSIONAL SHAPE OF A SURFACE USING A
METHOD OF THE STRUCTURED ILLUMINATION: INITIAL OBJECT (A) AND SHAPE RECONSTRUCTION (A
VIEW FROM VARIOUS ANGLES)

It is recommended to use current method to recreate the surface of sediment layer for
further analysis.

Analysis of obtained 3d data
A purpose of the required software is – obtaining the volumetric bed load. It is proposed
to experiment on implementation of a following sequence of 3 dimensional data analysis:
1) 3d surfaces are obtained from the coherent frames made by the high speed
camera.
2) Coordinates of the areas, where the grain started and stopped their movement, are
obtained.
Start and stop coordinates can be derived by highlighting the areas where the
heights of a corresponding points have decreased or increased in their values.
For that purpose it is advised to slightly modify and use a function “imdiffer.m”
discussed on previous pages. Current function was designed to compare gray
intensity values of the points of coherent frames and coincidently , the gray
intensity values are stored in a same format that heights are stored in Matlab

65

environment and therefore if provided input will contain heights data instead of
colour-intensity data, it will provide same type of output results.

Comment:

As it was assumed on pages 55-56 function”imdiffer.m” is likely to find an
application in an alternative way.

3) Each highlighted grain volume is calculated by summing up the height values of
the highlighted points in respect to the ordinance level.

= Σ
Eq. 11
 - volume of the grain
 Σ - sum of heights at all the points of the grain

4) If several grains have moved at the same time, it is important to identify which
start location is corresponding to which grain stop location.
This can be done by either:
a) Comparing the volumes of the moved grains on the coherent pictures
b) If a weak accuracy of the scanner does not allow comparing grains by
volume only, it possibly can be additionally compared by the mean colour
of the grain. This might help to reduce the probability of incorrect
identification of the grain.

5) Distances of travel of each grain are measured between movement start and stop
points obtained in step 2 and using output obtained in step 4.

6) Finally, all the required grain motion parameters (moved grain volume, travelled
distances of the grains, times of grain travels) are obtained and volumetric bed
load can be directly obtained using them. (see pages 4-6 for details of volumetric
bedload transport rate description).
66

Conclusion on proposed recommendation

Major advantage of the described method is that it is transforming input information in a
type of information requested on the software output. (volume on input, volumetric bed
load rate on output), therefore it becomes possible to directly analyse the volumetric input
data without any prior assumptions and approximations made about the volumes of the
grains. Therefore, accuracy of the bed load measurements will not be lost because of such
simplifications. Logically, accuracy of the method is only limited to a quality and
precision of the proposed scanner.

Alternative recommendation
As it was mentioned above – 3d surface recreation method is targeted on avoiding the
development of complicated 2 dimensional image analysis techniques. Theoretically it
should be the optimal solution for the problem. But as no experiments have been done on
the method it is useful to have an alternative proposal in case unpredicted difficulties are
met during the implementation of the described above method.
As in current situation the basic and most important task is to efficiently segment the
grains to be able to further analyse it and use the obtained grain parameters to determine
the required bed load.

Development of the segmentation method based on graph theory

Method description
Methods of the theory of graphs - one of most actively developing directions in
segmentation of images.
The general idea of methods of this group is the following. The image is represented in
the form of the weighed graph, with tops in image points. The weight of an edge of the

67

graph reflects similarity of points in some way (distance between points under some
metrics). Image splitting is modelled by cuts of the graph.

FIGURE 60: AN EXAMPLE OF MODELLING OF THE IMAGE THE WEIGHED GRAPH.

Usually in methods of the theory of graphs the functional of "cost" of the cut, reflecting
quality of the received segmentation is entered. So the problem of splitting the image on
homogeneous areas is reduced to an optimizing problem of search of a cut of the
minimum cost on the graph. Such approach allows besides uniformity of colour and a
structure of segments to operate also the shape of segments, their size, complexity of
borders, etc.
Various methods are applied to search of a cut of the minimum cost: "greedy" algorithms
(on each step such edge when total cost of a cut is minimum is selected), methods of
dynamic programming (it is guaranteed, that, choosing on each step an optimum edge,
will provide an optimum way), etc.
Method of Normalized Cut (J. Shi, J. Malik 1997). The normalised functional of quality
of a cut is entered in a way to simultaneously maximise distinction of points between
classes and to minimise distinctions of points in a class. Optimisation of the normalised
functional is reduced to a problem of search of own values of a matrix of paired distances
between all points of the image.

68

FIGURE 61: AN EXAMPLE OF A MATRIX OF PAIRED DISTANCES FOR A DOT CONFIGURATION.

Complexity of effective algorithm of search of own values of the rarefied matrix is linear
by number of points of the image. However the method demands storage of a matrix of
the size n*n, where n - number of points of the image and thus, in an initial kind is
inapplicable to the big images.

FIGURE 62: RESULTS OF WORK OF METHOD NORMALIZED CUTS( IMAGE TAKEN FROM

(http://cgm.computergraphics.ru)
Method of the Nested Cuts (Olga Veksler, 2000). The Main principle of this method
consists of branching of each point of the image from a special point outside of the image
bya a cut of the minimum cost. At such approach the image shares on not crossed
segments. It is shown, that the size of segments of the image can be operated by imposing
restrictions on cut cost.

69

FIGURE 63: RESULT OF WORK NESTED CUTS


The method of segmentation SWA (Segmentation by Weighted Aggregation) is based on
grouping of similar points of the image.The basic idea of a method consists in
construction of a pyramid of the weighed graphs, each of which is received from previous
by association of similar tops.

FIGURE 64: CONSTRUCTION OF A PYRAMID OF THE WEIGHED GRAPHS FOR THE IMAGE.


On each step weights of connections are recalculated. In the course of pyramid
construction the various statistics characterising the form, colour, a structure of regions
are calculated, these statistics are used for calculation of a measure of similarity of
regions. Then, following ideology of methods of the theory of graphs, for the received
graph the functional of cost of a cut is entered and the cut of the minimum cost is found.
70

In updating of algorithm,on each following step the result of the previous aggregation is
analyzed and corrected, and also the information on borders of the received segments is
used.

FIGURE 65: COMPARISON OF RESULTS OF WORK OF ALGORITHM SWA, ITS UPDATINGS AND NORMALIZED
CUTS


Quality of work of methods of the theory of graphs strongly depends on a metrics choice.
Therefore for a choice of the optimum metrics machine training is applied. The basic
problems of methods of the theory of graphs are a low speed of work and the big use of
memory. The majority of methods demand then storage of a matrix of paired distances
between the image points which size is equal to a square of number of points. Such
restrictions make grah methods practically inapplicable for huge images.

Conclusion on proposed method
As it is seen from the proposed method description, the proposed method is extremely
complicated and the process of its implementation to the problem of sediment bed load
measurement would be a very time-consuming process. Therefore it is recommended to
consider current method only if other more simple methods (including the 3d surface
reconstruction method described above) are proved to be insufficiently effective to
produce the qualitative output.

71

Conclusions Outline

 A simplified model of the analyzed river bed was designed, and the required input
data was obtained and analyzed.
 An optimal configuration of factors (such as lighting angle and strength, use of
reflector, etc.) was obtained in experimental way and described in the report.
 Many approaches mostly of image segmentation (mostly based on edge detection)
were developed and successfully tested and analyzed.
 The efficiency of each of the method was obtained and the possible
implementation of each method was considered and discussed.
 Obtained statistics of the results of the experiments helped to make assumptions
about the extremely weak edge areas of the image.
 It was concluded that image segmentation could not be produced efficiently on the
image with such combination of properties (lighting, shading, colors, etc.) by
means of the relatively simple and most widely used analysis techniques and
segmentation of the 2d image.
 Therefore the ideas of t alternative ways of program implementations were
developed and precisely described in the recommendations part.
 Relatively big amount of theories and different approaches were implemented and
tested. As a result, efficiency and the limit of each tested method is known. Based
on those results, possibly, most optimal direction for the problem solution was
identified.

72

List of References

 K.M. Clayton. “Weathering”. 1969. Edinburgh: Oliver & Boyd. P 551.3.053 ONE

 Steven J. Godman & Katharine Jackson & Taras A. Bursztynsky. “Erosin and
sediment control handbook”. 1986. New York; London: McGraw-Hill. R
631.459.GOL

 Walter Hans Graf. “Hydraulics of sediment transport”. 1971. New York:
McGrraw-Hill. X 627.15 GRA

 Gregory A. Baxes. “Digital image processing: a practical primer”. 1984.
Englewood Cliffs, N.J.: Prentice-Hall.

 Chang Wen Chen, Ya-Qin Zhang. “Visual information Representation,
Communication, and Image Processing”. 1999. W 621.391.61 BAX

 Kujawinska M., Weigiel M., Sitnik R. “Real-time 3D shape measurment based on
color structure light projection”.

 W. Osten and W.Jueptner. “Workshop on Automatic Processing of Fringle
Patterns”. 2001. P.324-331

 Salvi J., Batle J. “Pattern codification strategies in structured light systems”. 2004.
P. 827-849.

 Shtuchkin A., Gurov I. “Structured light range sensing using color and two stage
dynamic programming”. 2004.

 Marr D., Hildreth E. “Theory of edge detection”. 1980. Series 13, Vol. 207, pp.
187-217.

 Ziou D., Tabbone S. “Edge detection techniques-an overview”. 1998. Vol. 4, pp
537-559.

 Baker S. “Design and evaluation of feature detectors”. 1998.

 Maccarone M. C. “Fuzzy Mathematical Morphology: Concepts and Applications”.
1996. Vol. 40. Part 4, pp 469-477.

73

Appendix A : matlab codes of the proposed methods

1. Function rem_small

function [ img_small_removed ] = rem_small( imge, siz_rem )
%This function removes particles with the area defined in the input, from the image

% First of all this function labels all the objects in the image

[L,num] = bwlabel(imge ,8)

feats = regionprops(L, 'Area')

Area1 = zeros(num:1)

% Then it measures each objects area

parfor i = 1:1:num

Area1(i) = feats(i).Area
end

% Finally it excludes the objects with area that is less than predetermined value

idx = find(Area1 > siz_rem)

img_small_removed = ismember(L,idx)

end

74

2. Function rate_of_change_scan

function rate_of_change_scan

T = imread('G:Matlabtested_image.jpg'); %original picture
T = colr_to_BW(T); %convert image to BW

siz = size(T);

%smooth the image
T = medfilt2(T, [6 6])

T=adapthisteq(T)

%rate of change diference to mark pixel as a border
differ = 9;

%create an empty image - variable with a same size as original image

marked = zeros(siz(1,1),siz(1,2));

for ver = 2:(siz(1,1) - 1) ;

% Rate Of Change of a central and left pixel
ROC_left = T(ver,hor) - T(ver,hor - 1)
% Rate Of Change of a central and right pixel
ROC_right = T(ver,hor) - T(ver,hor + 1)
% Rate Of Change of a central and upper pixel
ROC_up = T(ver,hor) - T(ver + 1,hor)
% Rate Of Change of a central and bottom pixel
ROC_down = T(ver,hor) - T(ver - 1,hor)

ROC_diff_hor = imabsdiff(ROC_left,ROC_right)
ROC_diff_vert = imabsdiff(ROC_up,ROC_down)

if ROC_diff_hor >= differ

elseif ROC_diff_vert >= differ

end
end
end
figure,imshow(T)
figure,imshow(marked)

end

75

3. Function watershed_segmentation.m

function watershed_segmentation

1) Reading of the colour image and its transformation to the grayscale.

Reading data from a file
rgb = imread (' G:Matlabtested_image.jpg');
And present them in the form of a grayscale picture.
I = rgb2gray(rgb);
imshow(I)
text(732,501,'Image courtesy of Corel(R)',...'FontSize',7,'HorizontalAlignment','right')

2) Use value of a gradient as segmentation function.

For calculation of value of a gradient Sobel edge mask, function imfilter and other
calculations are used. The gradient has great values on borders of objects and small (in
most cases) outside the edges of objects.

hy=fspecial (' sobel ');
hx=hy ';
Iy=imfilter (double (I), hy, ' replicate ');
Ix=imfilter (double (I), hx, ' replicate ');
gradmag=sqrt (Ix. ^ 2+Iy. ^ 2);
figure, imshow (gradmag, []), title (' value of a gradient ')

3)Marking of objects of the foreground.

For marking of objects of the foreground various procedures can be used. Morphological
techniques which are called "opening by reconstruction" and "closing by reconstruction"
are used. These operations allow analyzing internal area of objects of the image by means
of function imregionalmax.

As it has been told above, at carrying out marking of objects of the foreground
morphological operations are also used. Some of them will be considered and compared.
At first operation of disclosing with function use imopen will be implemented.

se=strel (' disk ', 20);
Io=imopen (I, se);
figure, imshow (Io), title (' Io ')

Further opening using functions imerode and imreconstruct will be calculated.

76

Ie=imerode (I, se);
Iobr=imreconstruct (Ie, I);
figure, imshow (Iobr), title (' Iobr ')

The subsequent morphological operations of opening and closing will lead to moving of
dark stains and formation of markers. Operations of morphological closing are analyzed
below. For this purpose function imclose is used first:

Ioc=imclose (Io, se);
figure, imshow (Ioc), title (' Ioc ')

Further function imdilate is applied together with function imreconstruct. For
implementation of operation "imreconstruct" it is necessary to perform operation of
addition of images.

Iobrd=imdilate (Iobr, se);
Iobrcbr=imreconstruct (imcomplement (Iobrd), imcomplement (Iobr));
Iobrcbr=imcomplement (Iobrcbr);
figure, imshow (Iobrcbr), title (' Iobrcbr ')

Comparative visual analysis Iobrcbr and Ioc shows, that the presented reconstruction on
the basis of morphological operations of opening and closing is more effective in
comparison with standard operations of opening and closing.

Local maxima Iobrcbr will be calculated and foreground markers recieved.

fgm=imregionalmax (Iobrcbr);
figure, imshow (fgm), title (' fgm ')

Foreground markers imposed on the initial image.

I2=I;
I2 (fgm) =255;
figure, imshow (I2), title (' fgm, imposed on the initial image ')

Some latent or closed objects of the image are not marked. This property influences
formation of result and such objects of the image will not be processed from the
segmentation point of view. Thus, in ideal conditions, foreground markers display
borders only of the majority of objects.
Some of the foreground markers cross the edges of the grains. Markers should be cleaned
and shrunk to allow further processing. In particular, it can be morphological operations.

se2=strel (ones (5, 5));
fgm2=imclose (fgm, se2);
fgm3=imerode (fgm2, se2);

77

As a result of carrying out of such operation the separate isolated pixels of the image
disappear. Also it is possible to use function bwareaopen which allows to delete the set
number of pixels.

fgm4=bwareaopen (fgm3, 20);
I3=I;
I3 (fgm4) =255;
figure, imshow (I3)
title (' fgm4, imposed on the initial image ')

4) Calculation of markers of a background.

Now operation of marking of a background will be performed. On image Iobrcbr dark
pixels relate to a background. Thus, it might be possible to apply operation of threshold
processing of the image.

bw=im2bw (Iobrcbr, graythresh (Iobrcbr));
figure, imshow (bw), title (' bw ')

Background pixels are dark, however it is impossible to perform simple morphological
operations over markers of a background and to receive borders of objects which are
segmented. Background will be "thinned" so that in order to receive an authentic skeleton
of the image or, so-called, the grayscale picture foreground. It is calculated using
approach on a watershed and on the basis of measurement of distances (to watershed
lines).

D=bwdist (bw);
DL=watershed (D);
bgm=DL == 0;
figure, imshow (bgm), title (' bgm ')

5) Calculate the Watershed Transformation of the Segmentation Function.

Function imimposemin can be applied for exact definition of local minima of the image.
On the basis of it function imimposemin also can correct values of gradients on the image
and thus specify an arrangement of markers of the foreground and a background.

gradmag2=imimposemin (gradmag, bgm | fgm4);
And at last, operation of segmentation on the basis of a watershed is carried out.

L=watershed (gradmag2);
Step 6: Visualization of the processing result

Displaying the imposed markers of the foreground on the initial image , markers of a
background and border of the segmented objects.

78

I4=I;
I4 (imdilate (L == 0, ones (3, 3)) |bgm|fgm4) =255;
figure, imshow (I4)
title (' Markers and the borders of objects imposed on the initial image ')

As a result of such display it is possible to analyze visually a site of markers of the
foreground and a background.

Display of results of processing by means of the colour image is also useful. The matrix
which is generated by functions watershed and bwlabel, can be converted in the
truecolor-image by means of function label2rgb.

Lrgb=label2rgb (L, ' jet ', ' w ', ' shuffle ');
figure, imshow (Lrgb)
title (' Lrgb ')

Also it is possible to use a translucent mode for imposing of a pseudo-colour matrix of
labels over the initial image.

figure, imshow (I), hold on
himage=imshow (Lrgb);
set (himage, ' AlphaData ', 0.3);
title (' Lrgb, imposed on the initial image in a translucent mode ')

79

4. Function horscan_i

function horscan_i

%237 - krasnij cvet

T = imread('G:Matlabtested_image.jpg'); %original picture

T = colr_to_BW(T); %convert image to BW

siz = size(T);

%sensetivity (the smaller parameter is, the smaller should be pixel
%intensity diference to mark pixel as a border
differ = 7;

%create an empty image - variable with a same size as original image

marked = zeros(siz(1,1),siz(1,2));
T = wiener2(T,[5 5])

for ver = 1:(siz(1,1) - 1) ;

if (max(T(ver,hor), T(ver,hor + 1)) - min(T(ver,hor), T(ver,hor + 1))) >= differ;

marked(ver,(hor)) = 1;

else if (max(T(ver,hor), T(ver + 1,hor)) - min(T(ver,hor), T(ver + 1,hor))) >= differ;

marked((ver),hor) = 1 ;

else if (max(T(ver,hor), T(ver + 1,hor + 1)) - min(T(ver,hor), T(ver + 1,hor + 1))) >=
differ;

marked((ver),hor) = 1 ;

end
end
end
end

end
marked = rem_small(marked, 18 )
imshow(marked)
figure, imshow(T)
end
80

5. Function vibr1_1

function [ dif ] = vibr1_1(imnam1,pix_mov,pix_difference,rem_area)
%This function simulates vibration of the camera, in a way that will be
%useful to highlight the edges of the grains in the image

%Input:
%imnam1 = frame 1 name
%rem_area = area of particles to be filtered out
%pix_move = distance to move pixel
%pix_difference = critical pixel difference for highlighting

%Output:
%dif = variable containing grain borders obtained by current method

global im1 marked marked_after_hor_dif_sc marked_mod ver_size hor_size bin_im siz
ver_siz hor_siz

% im1 = original frame number one

% im2 = original frame number two

% marked = modified image after different operations

% marked_after_hor_dif_sc = original image after marking the horizontal borders with
this method

% marked_mod = original image modified also by this method, after all previous
modifications

% ver_size ; hor_size = vertical and horizontal sizes of the image

im1 = imread(imnam1)

im1= colr_to_BW(im1) %written function that converts colour image to black/white

im1 = adapthisteq(im1)

%smooth the image to reduce noise
im1 = medfilt2(im1, [6 6])

im_start %written function that measures the size of the image

81

%part that is simulating vibration of one pixel to the right side of frame

im2=im1
im3=im1
img1 = im1
im4=im1
im5=im1

pix_move = pix_mov+1


im2(ver,hor + pix_move) = img1(ver,hor); %Image shifted in horizontal direction
im3(pix_move_ver_up,hor) = img1(ver,hor); %Image shifted in vertical direction:
im4(pix_move_ver_down,hor + pix_move ) = img1(ver,hor); %Image shifted in
diagonal direction (vertically + right):
im5(pix_move_ver_up,hor + pix_move) = img1(ver,hor); %Image shifted in diagonal
direction (vertically + left):

end
end

bi_im1_2 = zeros(siz(1,1),siz(1,2)); % create an empty matrices to highlight borders
there in a binary format
bi_im1_3 = zeros(siz(1,1),siz(1,2));

im_difference1_2 = imabsdiff(im1,im2); %calculates absolute difference between the
two images

two images

two images

two images


82


if im_difference1_2(ver,hor) >= pix_difference
bi_im1_2(ver,hor - pix_move) = 1;

bi_im1_3(pix_move_ver_down,hor) = 1;

bi_im1_4(pix_move_ver_up,hor - pix_move) = 1;

bi_im1_5(pix_move_ver_down,hor - pix_move) = 1;

end
end

end

bi_im1_2_mod = rem_small(bi_im1_2,rem_area) %this part removes all the noise from
the resultant edge images
bi_im1_3_mod = rem_small(bi_im1_3,rem_area) %function
rem_small[img_name,area_of_noise_particle]

%this part of code puts all the defined edges into 1 image

bi_im_final = bi_im1_2_mod + bi_im1_3_mod + bi_im1_4_mod + bi_im1_5_mod

bi_im_final = rem_small(bi_im_final,rem_area) %this part removes all the noise from the
resultant edge images

figure, imshow(bi_im_final)

end

83

6. Function imabsdiffer.m
function [im_d] = imabsdiffer(im_one,im_two,pix_difference)
% global np1 np2 im1 im2 bin_im

im1 = imread(im_one)
im2 = imread(im_two)
%
% im1 = np1;
% im2 = np2;

%coverting to B/W

im1= colr_to_BW(im1)
im2= colr_to_BW(im2)


siz = size(im1) %gets the size of the image

im3 = imabsdiff(im1,im2) %calculates absolute difference between the two images

bi_im = zeros(siz(1,1),siz(1,2))

for ver = 1:(siz (1,1))
parfor hor = 1:(siz(1,2))

if im3(ver,hor) >= pix_difference
bi_im(ver,hor) = 1

end
end
end

% bi_im = im2bw(bin_im)

bin_im = bi_im

im_d = bi_im

figure, imshow(bi_im)

figure, imshow(im1)
figure, imshow(im2)
end

84

My "Grain Motion Detection" Project

More Related Content

What's hot (20)

Similar to My "Grain Motion Detection" Project (20)

My "Grain Motion Detection" Project