Lane Detection and Pedestrian Tracking System

Submitted in partial fulfilment for the requirements of the Degree

BACHELOR OF ENGINEERING IN COMPUTER ENGINEERING

Submitted By

Mohammed Aaris Tabraiz Kazi 18CO28

Mohd Yusuf 18CO39

Mohammed Altamash Swaleh Khan 18CO32

Talha Asjad Fakih 18CO22

Under the guidance of

Prof. Mubashir Khan
(Lecturer, Computer Engineering. Dept)
Department of Computer Engineering

Anjuman-I-Islam's Kalsekar Technical Campus

School of Engineering,

Plot No. 2 & 3, Sector - 16, Near Thana Naka,

Khandagaon, New Panvel, Navi Mumbai, Maharashtra 410206
(AY 2020-21)
 CERTIFICATE

This is to certify that the Mini Project entitled “Lane Detection and Pedestrian

Tracking System” is a Bonafide work being submitted to the University of Mumbai

in partial fulfilment of the requirement for the award of the degree of “Bachelor of

Engineering” in “Computer Engineering” And is a record of original Bonafide

carried out under our guidance and supervision. The results contained in this respect

have not been submitted in part or full to any other university or institute for the

award of Degree certificate.

Aaris Kazi

Mohd Yusuf

Altamash Khan

Talha Fakih.

(Internal Examiner) (External Examiner)

Prof. Mubashir Khan

(Project Supervisor, Guide)

Prof. Tabrez Khan. Dr. Abdul Razzak Honnutagi

(Head of Department) (Director, AIKTC)

i
 Mini Project Approval

This Mini Project entitled “Lane Detection and Pedestrian Tracking System” by

Name of students (Roll No.) mention below is approved for the degree of Bachelor

of Engineering in Computer Engineering. I represents my ideas in my own words and

where others' ideas or words have been included, I have adequately cited and

referenced the original sources. I also declare that I have adhered to all principles of

academic honesty and integrity and have not misrepresented or fabricated or falsified

any data/fact in my submission. I understand that any violation of the above will be

cause for disciplinary action by the Institute and can also evoke penal action from the

sources which have thus not been properly cited or from whom proper permission has

not been taken when needed.

Aaris Kazi

Mohd Yusuf

Altamash Khan

Talha Fakih

Date:

Place: Panvel

ii
 ABSTRACT
LANE DETECTION AND PEDESTRIAN TRACKING SYSTEM: Lane
Detection and Pedestrian Tracking System project explains about developing a
Machine based car system which detects the lanes and cars and moving object to
ensure safety on road. This helps the drivers to maintain the lane prevent the collision,
and as a result it helps the driver on the road and can prevent accidents. In present
trend education procedures from online had gradually increased which is also helping
faculties and institutions to save time and cost.

This system makes the time less consuming. Over time, problems meant to be solved
by software engineering have grown more and more complex. Today, The Faculties
are in problem due to the manual process of the Exam system. Teamwork is Must,
Time is consuming in manual process.

The purpose of Lane Detection and Pedestrian Tracking System– explains about
developing a system vehicle crashes remain the leading cause of accident death and
injuries in Malaysia and Asian countries which claiming tens of thousands of lives
and injuring millions of people each year. Most of these transportation deaths and
injuries occur on the nation’s highways. The United Nations has ranked India
accounts for 11 percent among countries with the highest number of fatal road
accidents, registering an average of 4.5 deaths per 10,000 registered vehicles.

The system is best based on today’s generation. It’s necessary to make an automated
system for the road traffic and discipline of the lane.

Lane Detection and Pedestrian Tracking System (LDPTS) require the ability to model
the shape of road lanes and localize the vehicle with respect to the road. Although, the
main reason to build intelligent vehicles is to improve the safety conditions by the
entire or partial automation of driving tasks. Among these tasks, the road detection
took an important role in driving assistance systems that provides information such as
lane structure and vehicle position relative to the lane.

iii
 Acknowledgement

I consider myself lucky to work under guidance of such talented and

experienced people who guided me all through the completion of my dissertation.

I express my deep sense of gratitude to my guide Prof. Mubashir Khan,

Lecturer of Computer Engineering Department, and Prof. Mubashir Khan, for his
generous assistance, vast knowledge, experience, views & suggestions and for giving
me their gracious support. I owe a lot to them for this invaluable guidance in spite of
their busy schedule.

I am grateful to Dr. Abdul Razzak Honnutagi, Principal for his support and
co-operation and for allowing me to pursue my Degree Programme besides permitting
me to use the laboratory infrastructure of the Institute.

I am thankful to my H.O.D Prof. Tabrez Khan and Prof. Mubashir Khan

(Project Coordinator) for his support at various stages.

Last but not the least my thanks also go to other staff members of Computer
Engineering Department, Anjuman Islam Kalsekar Technical Campus, Panvel,
library staff for their assistance useful views and tips.

I also take this opportunity to thank my Friends for their support and
encouragement at every stage of my life.

Date:

iv
 List of Figures

1 Main Algorithm 6

2 Code Example 9

3 Threshold table 10

4 Gaussian blurring 11

5 Canny Edge Detection 12

6 Canny vs Segmented Canny Image 13

7 Example of curve Roads 18

8 Orientation of light 19

9 Gantt Chart 19

10 Lane detection methodology 22

11 Algorithm 23

12 Waterfall model 23

13 DFD LEVEL 0,1,2 24-25

14 Use case Model 25

15 Sequence Diagram 26

16 Main Function 27

17 Region of interest 28

18 Sequence Diagram 28

19 Colour segmentation 30

20 Final Output 30

v
 List of Abbreviations

ITS Intelligent Transportation Systems

NGPP Near Ground Point Projection

SNR Signal to Noise Ratio

LDA Lane Detection Algorithms

LOIS Likelihood of Image Shape

ROI Region of Interest

HMM Hidden Markov Model

CPU Central Processing Unit

ITS Intelligent Transport Systems

IBM International Business Machines

RGB Red, Green, Blue

CMYK Cyan Magenta Yellow and Key (black)

HIS Hue Saturation and Intensity

SHT Standard Hough Transform

SVM Support Vector Machines

vi
 Contents

Abstract iii
Acknowledge iv
List of Figures v
List of Abbreviations vi

1 INTRODUCTION 1
1.1 Introduction 2
1.2 Motivation 3
1.3 Problem Statement and Objectives 5
1.4 Organization of the Report 6

2 LITERATURE SURVEY 15
2.1 Survey of Existing System 16
2.2 Limitation Existing system or research gap 17
2.3 Mini Project Contribution 19

3 PROPOSED SYSTEM 21
3.1 Introduction 22
3.2 Architecture/ Framework 22
3.3 Algorithm and Process Design 23
3.4 Use-case Diagram 25
3.5 DFD 24
3.6 Sequence Diagram 26
3.7 Details of Hardware & Software 26
3.8 Experiment and Results 27
3.9 Conclusion and Future work. 31
References 32

vii
 CHAPTER 1

INTRODUCTION

1
1.1 Introduction

Everybody in this world is concerned about safety. The people those who go out
from one place to other, expect to reach safely. Without any sudden incidents which
may come through externally by road accidents while travelling. We can avoid the
road accidents by using improved driving assistances. Vehicle crashes remain the
leading cause of accidental death and injuries in most traffic congested countries e.g.,
UK, USA, and Asian countries claiming tens of thousands of lives and injuring
millions of people each year. Most of these transportation deaths and injuries occur on
the nation’s highways. Therefore, a system that provides a means of warning the
driver to the danger has the potential to save a considerable number of lives. One of
the main technologies involve in these tasks is computer vision, which become a
powerful tool for sensing the environment and has been widely used in much
application by the intelligent transportation systems (ITS).

In order to increase safety and reducing road accidents, people are spending lots
of money for the advancement in the driving techniques which ensures the safety. The
technology makes man to think more to improve the safety to save the lives. The
automobiles are more conscious of providing safety feathers like seat belts, air bags
and strong body structures which provide the passive safety that may reduce the
effects of an accident. Avoiding accidents and saving lives are one of great interest
that all researchers and Automobile companies work on.

In Advanced Driver Assistance Systems in order to achieve the desired safety on

roads, the complex and challenging tasks of future road vehicles are road lanes
detection or boundaries detection (white and black lines on roads) and Obstacle's
detection (cars, pedestrians, trees, etc) especially for moving object detection is a key
component of collision avoidance in driving assistance systems.

Many sensors can be used for lane detection and obstacle detection, such as laser,
radar and vision sensors. Detecting all kinds of obstacles on the roads mainly include
IPM (inverse perspective mapping) method. The system acquires the front view using
a camera mounted on the vehicle then applying few processes in order to detect the
lanes and objects. A versatile methodology is used in order to detect the lanes and
objects. Cars equipped with intelligent system like road lane detection and obstacle

2
detection makes vehicles safer, which is vital in decrease number of victims or injured
people by car accidents. Principal approaches to detection are using vision system on
the vehicle.

In our research we have developed a simple heuristic method to improve the

robustness of lane detection and object detection and tracking in relation to intelligent
transportation system. In Heuristic method clustering methodology is used to group
the detected points and a best fit line in the mean square sense to detect the lanes.
Which are compared with other methods gives better lane detection. This method is
briefly explained in Chapter-3. From which you can assume the lane detection plays a
vital role for safety of lives in moving vehicles on roads.

On the other hand object detection and tracking the proposed heuristic method is
more effective for detect and track of single or multiple vehicles at a time without any
means of distortion and collision. This is compared with other important methods are
explained briefly in Chapter -4. Object detection and tracking with respect to distant is
other important aspect to ensure the safety on roads.

1.2 Motivations
There are many researchers who have worked and are working on creating and
developing many techniques in intelligent transportation systems with advanced
driving assistances system which are able to ensure the safety in the roads and
congested traffic conditions. The road accidents are the main causes for the sudden
death in this world. Even though we have many good and advanced techniques in this
world, we are left over with something to make it better than before. There are
chances from different angles. The road lane detection and object detection is also the
other important way that we can improve the safety in roads.

Vehicle crashes remain the leading cause of accident death and injuries in Indians
and Asian countries claiming tens of thousands of lives and injuring millions of
people each year. Most of these transportation deaths and injuries occur on the
nation’s highways. The United Nations has ranked Malaysia 30th among countries
with the highest number of fatal road accidents, registering an average of 4.5 deaths
per 10,000 registered vehicles[1]. It is not only limited to one country most of the

3
traffic congested countries like U.S, India, other Asian countries have many
calculation of deaths and injuries.

In intelligent transportation systems with improved technologies, the vehicles are

made more sophisticated with better infrastructure. But the way to move on the roads
by means of lane and object detection aspect is neglected by many automobile
companies and the ways to improve these aspects does not change from many years.
Lane detection and object detection plays vital role for accidents. For human vision
and human intelligence the task of lane detection and object detection changes due to
variations in the road conditions. Sometimes it is very easy to detect with the human
eyes but in some conditions due to externals effects the human intelligent have
detection problems.

Due too many external conditions that appears for the lane detection and obstacle
detection which may lead for the accidents. They are conditions such as appearances
such as change of Light conditions at Night vision, shadows caused by building and
trees, existence of surrounding objects, Mismatching of lanes, and lane changes in
curved roads.

So in our research we provide the way to improve the lane detection and object
detection in vehicles is import ants then rest of the other categories that may avoid the
many road accidents. Lane should have to be detected clearly even with the external
factors in consideration. The object detection will provide driving person confidences
even in the different lighting and different environments situations by improved
techniques to detect the objects. Thought you can provide the safety in roads to
achieve a safer environment and in traffic congested conditions.

In our thesis we have motivated to improve the intelligent vehicle assistances

with improving the SNR quality for lane and object detections as an important aspect
to avoid the road accidents and improving the safety on roads.

4
1.3 Problem Statement & Objectives
This document gives the detailed functional and nonfunctional requirements for
Lane Detection and Pedestrian Tracking System The purpose of this document is
that the requirement mentioned in it should be utilised by software developers to
implement the system.Identifying lanes on the road is a common task performed by
all human drivers to ensure their vehicles are within lane constraints when driving, so
as to make sure traffic is smooth and minimize chances of collisions with other cars
due to lane misalignment.

Similarly, it is a critical task for an autonomous vehicle to perform. It turns out

that recognizing lane markings on roads is possible using well known computer vision
techniques.

In earlier days of Automation Industry The Cars and The Trucks are made to
travel far and fast. The Cars were manually Transition and they were lack Human
Safety such as Seatbelts, and Airbags. Now cars have automatic Transition, Seatbelts
and Airbags to keep the driver safe from the collision of Impact, but does not prevent
accident. We can prevent the accident by making car smart and Intelligent by
Implementing:

• Lane-Detection Lane Detection is used to help the driver to be on road by

maintaining between the lane. By using Computer vision we can make sure the
vehicle which set on course will be inside the lane.

• Pedestrian Tracking Pedestrian Tacking helps the driver to detect any humans on
the road to avoid any collision on the road. This will help to reduce the accident
happens with vehicles on the road.

• Object Detection Any Obstacle or any Object in Comes under the Area of Interest
or Vision of Interest should be detect and warn the driver for the upcoming obstacle.

• Number Plate Capturing Computer Vision can be used as Surveillance tool to

capture the related number plate on the road to identify any missing vehicles

5
1.4 Organization of the Report

Fig Main Algorithm

Functional and Non-Functional Requirements

• Functional Requirements:

To identify the lanes and the edges of the road and identify the video feed is
coming to the program. The program identifies the video of roads and identifies the
lanes of White and Yellow color lanes. To be identified as a road.

To identify the walking pedestrian of any shape and size in the vision of the lane
and prevent the collision if the pedestrian is close.

In software engineering, a functional requirement is used to define a function

regarding or its component, in which function is declared as specification of behavior
between input and output.
6
Functional requirement includes:

o Test feeds of Video and Images of Road.

o Test feeds of Video and Images of Pedestrian.

o Library of Computer Vision.

Performance:

o The performance should be higher to navigate over the road for the fast
reflexes.

Maintainability:

o Updating the program over the air to maintaining for the future
upgrades

Reliability:

o The program should be accurate in 95% all cases

• Non-Functional Requirements

These are the basic quality constraint that the system must satisfy the system must
satisfy to the contract.

The priority or extent to which these factors are implemented varies from one
project to another, they are also called non-behavioral requirements. They are
basically deal with issues:

o Portability

o security

o Maintainability

o Scalability

5. Functions Used

The Pipeline

In this part, we will cover in detail the different steps needed to create our

pipeline, which will enable us to identify and classify lane lines. The
7
pipeline itself will look as follows:

• Convert original image to HSL

• Isolate yellow and white from HSL image

• Combine isolated HSL with original image

• Convert image to grayscale for easier manipulation

• Apply Gaussian Blur to smoothen edges

• Apply Canny Edge Detection on smoothed gray image

• Trace Region Of Interest and discard all other lines identified by our previous
step that are outside this region

• Perform a Hough Transform to find lanes within our region of interest and
trace them in red

• Separate left and right lanes

• Interpolate line gradients to create two smooth lines

The input to each step is the output of the previous step (e.g., we apply Hough
Transform to region segmented image).

Threshold Function

In some CV Applications it is often necessary to convert color images to

grayscale, since only edges and shapes end up being important.

Similarly, some applications only require a binary image showing general

shapes. Thresholding is fundamentally a method of segmenting an image into
different parts. Thresholding will convert an image to consist of only two values,
white or black.

retval, dst=cv.threshold(src, thresh, maxval, type[, dst])

8
Parameters:

src input array (multiple-channel, 8-bit or 32-bit floating point).

dst output array of the same size and type and the same number of channels as

src.

thresh threshold value.

maxval maximum value to use with the THRESH_BINARY and

THRESH_BINARY_INV thresholding types.

type thresholding type (see ThresholdTypes).

Code Example:

9
Adaptive Threshold Function
Threshold table:

dst =cv.adaptiveThreshold( src,maxValue,

adaptiveMethod,thresholdType,blockSize, C[, dst])
Parameters:

src Source 8-bit single-channel image.

dst Destination image of the same size and the

same type as src.

maxValue Non-zero value assigned to the pixels for which

the condition is satisfied

adaptiveMeth Adaptive thresholding algorithm to use, see

od AdaptiveThresholdTypes. The
BORDER_REPLICATE| BORDER_ISOLATED is
used to process boundaries.

thresholdType Thresholding type that must be either

THRESH_BINARY or THRESH_BINARY_INV, see
ThresholdTypes.

blockSize Size of a pixel neighbourhood that is used to calculate a

threshold value for the pixel: 3, 5, 7, and so on.

10
 C Constant subtracted from the mean or weighted mean
(see the details below). Normally, it is positive but may
be zero or negative as well.

Gaussian Blurring
In this method, instead of a box filter, a Gaussian kernel is used. It is done with
the function, cv.GaussianBlur(). We should specify the width and height of the
kernel which should be positive and odd. We also should specify the standard
deviation in the X and Y directions, sigmaX and sigmaY respectively. If only sigmaX
is specified, sigmaY is taken as the same as sigmaX. If both are given as zeros, they
are calculated from the kernel size. Gaussian blurring is highly effective in removing
Gaussian noise from an image.

If you want, you can create a Gaussian kernel with the function,
cv.getGaussianKernel().

The above code can be modified for Gaussian blurring:

Result:

Gaussian blurring

11
Canny Edge Detection:
OpenCV puts all the above in a single function, cv.Canny(). We will see how to
use it. First argument is our input image. Second and third arguments are our minVal
and maxVal respectively. Third argument is aperture size. It is the size of Sobel kernel
used to find image gradients. By default it is 3. Last argument is L2gradient which
specifies the equation for finding gradient magnitude. If it is True, it uses the equation
mentioned above which is more accurate, otherwise it uses this function:

Edge_Gradient(G)=|Gx|+|Gy|

By default, it is False.

Canny Edge Detection

Region Of Interest
Our next step is to determine a region of interest and discard any lines outside of
this polygon. One crucial assumption in this task is that the camera remains in the
same place across all these images, and lanes are flat, therefore we can identify the
critical region we are interested in.

Looking at the above images, we “guess” what that region may be by following
the contours of the lanes the car is in and define a polygon which will act as our
region of interest below.

We put the canny and segmented images side by side and observe how only the
most relevant details have been conserved:

12
 Canny vs Segmented Canny Image

1.1 Software and Hardware Requirements

Following Requirement for running the program successfully:
a. OS: Linux or Windows

b. Processor: i3 or Higher

c. Ram: 4GB or Higher

d. Diskspace: 500MB

e. Language python

f. modules: opencv, numpy, imutils, ffmpeg-python, matplotlib

1.2 Software Quality attributes of the projects

The quality of the product should be accurate enough to detect the road and lane

and maintain the driver to be between the lane and detect the pedestrian between the
vision of the computer to prevent the collision if the object is close enough.

• Accuracy hence reliable

• Performance

• Fast speed of computing

• Compatibility of any type of camera or any video feed.

13
8. References and Acknowledgements
• Tsukamoto, Yukihiro, et al. "Multi-Lane Detection and Tracking Using
Vision for Traffic Situation Awareness." 2020 16th International
Conference on Wireless and Mobile Computing, Networking and
Communications (WiMob)(50308). IEEE, 2020.
• Gaussian Blur:
https://docs.opencv.org/master/d4/d13/tutorial_py_filtering.html

9. Software Quality attributes of Project

The quality of the product should be accurate enough to detect the road and lane and
maintain the driver to be between the lane and detect the pedestrian between the
vision of the computer to prevent the collision if the object is close enough.
• Accuracy hence reliable
• Performance
• Fast speed of computing
• Compatibility of any type of camera or any video feed

14
 CHAPTER 2

LITERATURE SURVEY

15
2.1 Survey of Existing System

Lane detection is a well-researched area of computer vision with applications to

autonomous vehicles and driver assistance systems. This is partly because, despite the
apparent simplicity of the white markings on a dark road, making it very difficult to
identify the markings on different types of roads. These difficulties are of an
occlusion in the shadow of other vehicles, changes in the roadway itself, and different
types of road markings. A lane detection system must collect all types of markers
roads

confusion and filtered to give a reliable estimate of the path of the vehicle's position.

Lane detection plays an important role in driver assistance systems. In general,

the steps of lane detection localize lane boundaries in the images of the specified path,
and can help to estimate the geometry of the floor and lateral position ego vehicle on
the road, Lane detection in intelligent cruise control environments for Lane Departure
Warning, modeling the way, and so on.

Lane detection algorithms detect lane markings and the edges of the road, and
estimate the vehicle position in the lane. Lane detection provides a framework for the
support of many other single-camera based Mobil eye functions as vehicle detection;
in this case, it contributes to the correct position of the vehicle in the same lane.
Provided that the road markings visible and that their testimony is not hindered by the
presence of clutter, acknowledge shadows, rain, snow or other disturbances on the
road, the LDA recognizes the majority of white, blue and yellow markings across the
world, and is Mobil eye system is approximately 99% of cases[6].

Different types of marks, such as solid, dashed, Bott points are double and triple
road markings validated and integrated into production successfully. In addition,
recognizing the LDA roadside (road edges) unmarked, such as grass or gravel banks,
for more information on the adjacent track to support the strategy of caution and
refine the OEM requirements. Also developed a system of permits for better
separation of ambiguous markings, road markings double, triple, markings, etc., and
the system has been refined and adapted to meet the variation found in different
countries correctly. The authorization mechanism can also use the color information
for better separation. The LDA was tested in a series production programs in Europe,
16
North America, Africa, the Middle East and Asia and has been validated on several
continents and in a wide range of scenarios, including bright sunlight and weather
around the world. In construction areas where there are many overlapping brands, the
system is not available. Lane markers of different colors (e.g. blue markings Korean)
has successfully developed and operated on the same input a monochrome imager as
all other functions.

2.2 Limitation Existing System or Research Gap

Object Detection

SENSING of vehicle detection and traffic conditions while driving preceded

important safety driving, accident avoidance, and automatic driving and pursuit
detecting and tracking vehicle forward with a video camera in the vehicle. Introduced
the HMM to vehicle identification during tracking, so that the likelihood of a decision
framework that follows less affected by temporary thresholds. The joint probability of
the picture and velocities to separate the dynamic HMM estimated from the target
vehicles in the background-way positions. The use of coherence functions temporal
motion extends the identification and tracking of vehicles.

A novel system for detecting and tracking vehicles from a camera mounted on the
vehicle presented. The heart of the system is high vision algorithms to control the WB
detector and the TLD tracker, real-time process scheduling.[32] The system is running
in real-time (10Hz) on a single CPU core. New record for system evaluation board is
designed to monitor featured of the vehicles. All data were collected on the Italian
motorways, which includes a variety of light conditions and signaling the amount of
data which show in below figure 4, has an approximate ground truth estimated from
laser scans with visual data. The data and the reality of difficult terrain will be
available to the public.

17
 Fig Example of curve Roads

Fig Example of Long Road

SNR Improvement for visibility of Road

Visibility is an important factor for road safety. Every year a number of accidents
occur due to poor visibility and high speeds. Due to poor vision in bad weather,
considerable number of airborne particles with the significant size and distribution of
media participants that absorb and scatter light environment and is reflected by
significant items, and to the point display position is not as clear as if there are no
object presence, and in foggy weather, people actually tend to show that excessive
speed can cause overestimation.

Recently, perception sensors (cameras, radar, etc.) are introduced for monitoring
the surveillance system. These sensors are designed to operate in a variety of
situations and conditions (light, weather, etc.) with a pre-determined threshold rate of
change. When using perception sensors rather than the meteorological visibility
18
sensor, through dealing with hazy or fog image, has a great hot topic in the study.
Many countries and institutions, such as Japan, USA, began to study on the subject.
AerotechTelub and Delarme University under the Swedish National Road Agency
have done some associated research on the topic.

Fig Orientation of light

2.3 Mini Project Contribution

Fig Gantt Chart

19
 Name Task
Aaris Kazi To gather the information, research and
Development of the program
Making Raw Program
Research
Build
Mohd Yusuf To gather the information, research and
Development of the program
To gather alternate source
Contribute in raw function
Research and Implementation
And Matrix understanding
Altamash Khan To test the program and contribution in
coding
To test the expected output with actual
output
To make changes where required
Research and Implementation
Talha Fakih To test the program and covert into a
applications
Py to exe
UI in kivy
Research and Implementation

20
 CHAPTER 3

PROPOSED SYSTEM

21
Introduction

For lane detection this search was thus limited to IEEE database with the
keywords: “lane tracking in video” and “Object tracking in video” in May 11th, 2013.
This search retrieved a total of 99 papers, from which 14 were journal papers and the
remaining 85 are conference papers. After reading all the titles and abstracts, 51
papers were considered to be of interest, from which 4 are journal papers and 47 are
conference papers. Publication years range from 1992 to 2012. From this research
various method have been described, but our research mainly focus on latest research
Hough based transform by using gray-scale method and colour segmentation method
which is a part of our subject Digital image processing, same video is used for both
Hough based transform and proposed method and will see their experimental results
and comparison in case of lane detection.

Architecture

22
Algorithm and Process Design

Fig Algorithm

Process Model

Iterative Waterfall Model In a practical software development project, the

classical Waterfall Model is hard to use. So, Iterative waterfall model can be thought
of as incorporating the necessary changes to the classical waterfall model to make it
usable in practical software development projects. It is almost same as the classical
waterfall model except some changes are made to increase the efficiency of the
software development. The iterative waterfall model provides feedback paths from
every phase to its preceding phases, which is the main difference from the classical
waterfall model.

23
DFD

DFD LEVEL 0

DFD LEVEL 1

24
DFD LEVEL 2

Use case Diagram

Fig Used Case Diagram

25
Sequence Diagram

Fig Sequence Diagram

26
Experiment and Results

Fig Main Function

Hough-based Detection

In this section we will explain the lane detection by using grey scale method and
edge detection and will examine the results by using this method. The Hough based
detection includes the four parts as shown in the grey block diagram.

Defining and Conversion of ROI into Gray Scale

In this step, the captured color image is converted to grey scale to make method
faster, less computational, and less sensitive to scene condition [44]. In our proposed
method, captured series of images received from a camera on top of a car would be
processed. The camera is adjusted in a way that the vanishing point of road should be
placed on the top of ROI, shown in figure 10. Based on camera place adjustment, only
27
part of the bottom of the captured image would be valuable for processing and it
causes short time processing and less memory usage.

Fig Range of Interest

Canny Edge Detection

By applying the optimum global thresholding to selected part of image, ROI, we

have binary image as the input for this step. In this step, to find lane boundaries in the
image we use one of the edge detection methods called Canny Edge Detection and the
detected boundaries are shown in figure 11.

Fig. Canny Edge Detection

28
 Canny Edge detector most commonly used for step edges due to Optimal, then is
corrupted by white noise. The objective is the edges detected must be as close as
possible to the true edges the number of local maxima around the true edge should be
minimum.

Canny edge detection basically uses gradient vector of an intensity image. Lane
boundaries have high contrast in the image, and this feature yields high values of
gradient vector by which we can find the edge direction, which is orthogonal to
gradient vector. Many edge detection methods are based on this principle, but the
efficiency levels are different. One of the best and efficient methods is canny edge
detection [8]. The most important characteristics of canny method are that the error
rate of this method is low because this algorithm uses double thresholding, hysteresis
thresholding [8]. Hysteresis threshold, double thresholding, suppresses the pixels that
are not related to edges. Therefore, the detected edge is really close to true place. We
should also mention that canny edge detector is very sensitive to noise; therefore, we
smooth the image by a low pass filter to reduce the effect of noise.

Colour Segmentation

The process that an image is divided to multiple segments is called segmentation.

Colour segmentation helps us to identify the boundaries and objects in an image based
on desired color. Color images could be modeled with many color space like RGB,
CMYK, and HSI. Every color space could be converted to other by using some

29
 Fig. Colour segmentation

This can be solved easily for the required road plane vector n k , in figure 20 a
typical result of our proposed algorithm is shown. Nearly all lane markers – although
lying in the road plane - have been detected as object so the difference image of the
left and the compensated image which has been calculated from the right image with
the final collineation parameters P are presented.

Fig. Final output

30
Conclusion and Future Work

Robust lane detection and object tracking is an important application of

Intelligent Transport System. To avoid victims and number of accidents in heavy
traffic countries like USA, China, India, Malaysia, UK, where it becomes difficult for
the driver to exact location and detection of line and cars especially during cloudy
environment than it is important to make Intelligent Transport System more robust
and as well in other way lane detection and object tracking is one of important future
application of auto drive Vehicle.

Up till Now so many different vehicle companies and researchers have used
different ways and develop different algorithms under different conditions to make the
Intelligent Transport System more robust to noise and detection but they usually
operate under certain type of scene conditions and more complex to implement under
different conditions.

As in case of lane detection we described and implemented the Hough-based

detection, and had a look on the results. We saw that Hough based algorithm is not
efficient enough and some lines were not detected. Then, we described the color-
based detection algorithm and extracted some more information of lines based on
their color information to make Hough-based detection more efficient. In this case
some unwanted lines are detected although with complexity reduction. As in case of
object detection the use Fish eye camera brings a large field of view resulting great
image distortion and in case of Gabor Filter optimization some unwanted objects are
detected.

For instance, in our research we have developed a Heuristic Algorithm which is

more robust in case of lane detection when compared with other methods of lane
detection with reduced complexity, more tolerant to scene condition and also easy to
implement in any noisy environment. In the same manner it is also in object tracking.

Multiple vehicles are detected on the same time without any distortions and
overcome all the drawbacks when compared with other methods. This method is very
effective in all the conditions and more robust in object tracking with reduced
complexity and easy to implement under different scene conditions, that significantly
gives more strengthen to Intelligent Transport System.
31
 Our proposed algorithm was implemented in MATLAB R2012a on a DELL
computer with CPU of Intel ® CoreTM 2 Duo with the processor frequency of 2.0
GHz and RAM of 4.00 GB. We have processed captured images collected from PHD
student at BTH Karlskrona who has already used this video for another algorithm that
are in RGB24 320x240 with 1807 total number of frames.

References

[github] https://github.com/MaybeShewill-CV/lanenet-lane-detection

[youtube] https://www.youtube.com/watch?v=rvnHikUJ9T0

[analytic vidya] https://www.analyticsvidhya.com/blog/2020/05/tutorial-real-time-

lane-detection-opencv/

[medium] https://medium.com/analytics-vidhya/finding-road-lane-lines-using-

opencv-python-bd1c8eb34652

[opencv] https://docs.opencv.org/master/d6/d00/tutorial_py_root.html

32