A Survey On IoT Based Road Traffic Surveillance and Accident Detection System (A Smart Way To Handle Traffic and Concerned Problems)
A Survey On IoT Based Road Traffic Surveillance and Accident Detection System (A Smart Way To Handle Traffic and Concerned Problems)
Abstract—Road Traffic is one of the most vital problem in our various types of data for completion of some specific objective.
hastily developing world. This paper presents of study of IoT has been quite efficient in phrasing technically smart
different aspects and issues related to the problem. This paper concepts for various domains connected to the real world such
emphasizes on using prominent technology -Internet of Things as smart city, smart home, smart agriculture, and smart security
(IoT) for developing smart system to monitor various parameters etc.
related to road traffic and using it for effective solution. The
survey of the existing systems and concerned techniques related This paper presents in depth description of issues and
to the problem area are discussed. Different issues like vehicle research problems related to traffic surveillance and accident
detection, occlusion detection, lane projectile detection, accident detection system. We also present analysis and findings from
detection and related methods to solve these issues are explored. the study of the literature related to the problem. It also
We propose our ”IoT based traffic monitoring and accident presents our proposed system and future work.
detection system” consisting of Raspberry pi, and pi camera as
hardware which will use live video as input and process it for The proposed system based on IoT is framed with the help
gathering information about the live traffic. The system of a processing board to process the data and a camera module
generates information regarding the traffic such as number of to provide the live video as input. Raspberry pi board will be
vehicles, emergency accident situations, and improper projectile used as processing module and pi camera module will provide
of vehicle. The generated information can be used to handle and the input data in video raw format h.264 to the Raspberry pi.
divert the live traffic as needed to avoid the problems related to The system will detect the number of vehicle passing by,
road traffic. accident and predict the lane projectile of the vehicles on the
road. The background subtraction using Gaussian mixture
Keywords—Internet of Things (IoT); Raspberry pi; Pi camera; model and edge detection using canny edge is been executed
Traffic Monitoring; on Raspberry pi.
I. INTRODUCTION A. Motivation
Rising road Traffic is one of the biggest problem tackled by Major cities of India like Mumbai, Surat, and Ahmedabad
the world in day to day life. People suffer every day in some or are facing the problem of road traffic badly. The people
the other way due to heavy road traffic. It is much important to remain stuck in traffic jams and deadlocks for hours. There are
consider and improve the factor of road safety while thinking many emergency situations on road due to traffic which needs
about the problem of road traffic. The efficient solution to this instant attention. These emergency chaotic situations can
problem is to use smart digital technology for handling the real prove to be vulnerable. Therefore it is necessary to regulate
time traffic. There has been various research work carried out the traffic to ensure swift flow of the vehicles on the road.
to find the solution for this problem. But still there is a need of Hence in this growing world of technology, the idea to use
efficient and compatible system that can be practically tested
smart technological systems with wide future scope can be
and implemented.
much more effective.
Traffic surveillance can be very useful for future planning
such as pollution control, traffic signal control, infrastructure II. RESEARCH PROBLEMS
and road planning, accidental blockage avoidance. The data
obtained from this system can be useful for the prediction of A. Research Problems Related to System in Live Environment
the traveling time and path. There is very wide scope of improvement for the system to
The idea of using Internet of Things (IoT) can be very be implemented and tested practically. Many researchers have
helpful for developing such a smart system. IoT is having proposed various techniques like canny edge detection [8],
interconnected network of physical components and devices Spatial-Temporial analysis with Hidden Markov Model
connected for the collection, exchange and processing of (HMM) [1], thresholding and background subtraction [12], 3-
1
International Conference on Innovations in Power and Advanced Computing Technologies [i-PACT2017]
D model & neural network [13] for vehicle tracking, accident 5) Cost effectiveness
detection and lane projectile. They have used varying range of Economically feasible system will provide much better
hardware like stratix III FPGA [8], cameras such as pan-tilt & practical and feasible approach. If we think of bigger scenario
zoom camera [5], standard camera [2] [1], and V8 camcorder where this kind of system is needed to be used and maintained
[3] at different varying frame rate and resolutions but on large number of roads of the city, cost becomes one major
somehow the architecture of the system developed lack the factor. There arises a need of building system with least cost
following factors and needs to be evaluated. hardware and software.
1) Practical compatibility
B. Resource constraints for IoT based system
Researchers and technocrats have proposed many ideas
and developed many combinations of hardware and software 1) Processing Power
to provide best possible solution they can. But still the systems The proposed system will have a limited resources for
lack to provide practically compatibility with the real world processing which will affect the processing power. Raspberry
for practical usage. Some researchers have worked only with pi has only 1 gigabytes of ram and 1.2 gigahertz of Quad Core
offline video data [1], [4], [8], [13], which does not give processor. These resources need to be managed and handled in
insight about working of their proposed work in real world, a way to perform the various video processing operations.
While some research work have been tested in real world Hence we are bound to work according to this limitation.
which needs more exploration about its functioning in 2) Power supply
different vivid real time situations for practical approach with The propose system requires continuous power supply but at
flexible implementation with live traffic. present situation we plan to use a battery. So we will have a
2) Environment and background problem limited power supply for implementing and testing our system.
The background and environment effects in several ways 3) Atmospheric effect
to the working of the devices. The obstacles in the background The hardware components of the proposed system will not
such as tree, plants, people etc. are factors that can affect to have any protection against the atmospheric situations like
the video processing algorithm and hence can provide inferior heat, rain, cold etc. which can affect the normal working of the
results. Frame differential technique used in [12] can have system.
altered unusual heavy processing on Raspberry pi board if
there is too much heavy traffic and background is heavily
dense and if the frame rate is not set to work with to the III. RELATED WORK
unusual situation. Hence the surrounding environment The previous work carried out by different researchers have
captured in the frame of the camera is an important point to be been analyzed and studied considering various factors and
considered while taking the practical approach for this system. parameters to generate the necessary findings of the
methodology, techniques and criteria needed for developing the
3) Software and hardware compatibility proposed system. The literature review has been prepared into
As described earlier there are many software and hardware five table representing the analysis of techniques used for
that has been combine for several proposed work. It is very vehicle detection, accident detection and vehicle tracking,
vital for software to mutually work precisely with the dataset used and type of the camera used. The quality of the
hardware. Raspberry pi board is not designed to work with system is judged based on the performance parameter along
x86 operating system [21] which includes windows and some with the physical hardware that has been used for development
of the Linux based OS. We preferred to use Raspbian OS, of the system. The tables are prepared based on criteria whether
which is a specially designed and preferred operating system the work of authors has been implemented in live environment
for Raspberry pi and python is the promoted language that is or tested in offline mode.
preferable to work with Raspberry pi. The behavior of tools
and software will differ for different hardware, for example A. Vehicle Detection
the installation and working of the OpenCV library by Intel The methods Spatio-temporal analysis [1] [2], Kalman filtering
according to the programming language on IoT devices for [5], Frame difference [6] [10], Gaussian mixture model [20]
video processing will differ unlike in normal configured have been used for getting the extraction of the shape of
computer. Hence it is necessary to find best possible vehicle. The Gaussian mixture model has been more accurate
combination of both elements which will be suitable to work to separate out the shape of moving vehicle and it also works
in live environment with good performance. for adaptive background. Spatio-temporal difference and
Kalman filtering methods need complex processing. Binary
4) Independency Frame difference is less complex but provides less correctness
Most of the current systems which are been observed and compared to Gaussian mixture model.
tested in the live environment needs formation of the design to
work as an independent system with remote access and control
over the network to provide it more compact ability and
feasibility to be implemented in real world.
2
International Conference on Innovations in Power and Advanced Computing Technologies [i-PACT2017]
TABLE I. OFFLINE VEHICLE DETECTION APPROACHES this unusual behavior can lead us to finding the accident. The
No. Camera Object Performance Data set researchers have used Hidden Markov Model [1], Neural
details detection measure network [13] [17], Support Vector Machine [16], Feature based
technique [claimed method [6] for accident detection. This methods have been
accuracy] used in different conditions, some have used the recorded video
[1] Standard Spatial- Vehicle Recorded while others have manually created the environment using
camera temporal tracking with video from
10 fps Markov random occlusion camera model to demonstrate the condition of the accident. Hence
field model [93-96%] there are different situations and parameters according to which
[3] V8 camcorder Susan algorithm Vehicle count Recorded authors preferred to use these methods.
video from
camera
[4] NA 2-D Token- Vehicle NA TABLE III. ACCIDENT DETECTION APPROACHES
Based Tracking tracking
No. Camera Accident Performance Data set
Using Kalman [99%]
details detection measure/claim
Filtering
technique ed accuracy
[5] pan–tilt-zoom Kalman filter Vehicle Recorded
[1] Standard Hidden NA Recorded
camera 10fps- tracking video from
camera Markov Model video from
25fps camera at
10 fps camera
176x144
[3] V8 Contour NA Images
352x288
camcorder Based recorded at
[8] Standard Thresholding Vehicle count NA
320x240 Chiao Tung
camera with Canny
University
edge
[6] Charge Feature based Accident Video
[11] Cctv camera Harris-Stephen Vehicle count Live video
couple -acceleration prediction at road
320x240 corner detector & speed feeds from
device -area intersection intersection
29.95fps cctv over
camera -position at
freeways
-direction Louisville
[13] NA 3D model based NA Simulated
[13] NA Neural Network Probability of Simulated
[16] Standard Background Vehicle count NA accident environmen
camera subtraction t model
[20] Standard Gaussian NA NA [16] Standard Support vector Accident NA
camera mixture model camera machine prediction
TABLE II. VEHICLE DETECTION APPROACHES IN LIVE ENVIRONMENT [17] Standard Fuzzy-self NA 3D model
camera organizing Images
No. Camera Object Performance Data set Neural network
details detection measure/claimed
technique accuracy C. Vehicle Tracking
[2] Standard Spatial- Vehicle count Real road
camera 25/30 temporal [94%] traffic Vehicle tracking gives the lane or the path of the moving
fps analysis scene at vehicle. The lane of the vehicle can be used to judge and
Modena, monitor the regular flow of the traffic. Vehicle tracking is
Bologna, mainly divided into two categories trajectory based and optical
Ferrara flow. Optical flow based method requires low computation
[6] Charge couple Frame Vehicle count Video
device camera difference from
compared to trajectory based. Based on the survey Lucas-
road Kanade method [7] based on optical flow and K-means
intersecti clustering [5] based on trajectory is found more efficient.
on at Comparatively Lucas-Kanade needs less computation and
Louisville provides high accuracy.
[10] Pi camera Frame No of correct Video -
1280x720 difference vehicle pixels Lobby,
30fps detected fountain,
[89.32-98.52%] water
surface, TABLE IV. VEHICLE TRACKING APPROACHES IN LIVE ENVIRONMENT
[12] Pi camera Frame Vehicle detection Live
No. Camera Vehicle Performance Data set
640x480 difference [93.2%] video
details tracking measure/claimed
30fps feed
technique accuracy
[14] Standard Temporal NA Live
[7] Standard Optical flow True alarm Video from
camera 54- difference video
camera based [98%] Interstate I-35,
70fps feed
Lucas- US-53,
[21] Web camera Frame [99%] Live
Kanade Minnesota
difference video
HW
feed
B. Accident detection
The process of accident detection is connected with some
anomaly or unusual behavior of the vehicle. The prediction of
3
International Conference on Innovations in Power and Advanced Computing Technologies [i-PACT2017]
TABLE V. OFFLINE VEHICLE TRACKING APPROACHES The important analysis and findings that are obtained from
No. Camera Vehicle Performance Data set the study of the previous work is represented in Fig. 1. The
details tracking measure/claimed analysis about various techniques/methods was carried out to
technique accuracy find best techniques. The study of the previous work helped us
[3] V8 Contour NA Images to get insight about the existing system and their methodology.
camcorder based recorded at It helped us in understanding how to design and propose more
320x240 Kalman Chiao Tung
filtering University efficient system.
[4] Standard Trajectory vehicle tracking NA
camera using [99%] D. Available Open Source Data Set for Experimentation
centroid
[5] pan–tilt– Trajectory Vehicle tracking Recorded 1) VIRAT video data set [22]
zoom based – [NA] video from This data set contains raw video files of 8.5 hours in
camera K means camera at .mp4 format having resolution of hd 720pixel. The videos in
10fps & clustering 176x144 the data set are recorded from stationary camera and contains
25fps 352x288
resolution ground information with mixture of humans walking and
[9] Standard Optical NA NA vehicles on the road.
camera flow
Median 2) MIT Traffic Video Data set [23]
filtering & The MIT traffic data set contains 90 min video sequence
centroid recorded at 720x480 using a standard camera. The data set
[11] cctv camera HS Vehicle speed Live video contains daily life activities on the road including humans. This
320x240 corner [NA] feeds from
29.95fps detector cctv over
data set can perfectly be used for testing and experimentation
centroid freeways purpose
[13] NA 3D model NA Simulated
and neural environmen
network t model
[16] Standard Ramer Trajectory NA
camera Douglas detection [NA]
Peucker
Fig. 1. Classification of Vehicle Detection, Accident Detection, and Vehicle Tracking Approaches..
4
International Conference on Innovations in Power and Advanced Computing Technologies [i-PACT2017]
Programming tools:-
IV. PROPOSED SYSTEM
After the study of the related work and analysis of the 5) Python
existing system we decided to design a system which helps to Python is the official supported language for Raspberry pi.
overcome and tackle some of the existing research problems There is also wide community working on Raspberry and
discussed earlier. The scope and objective of the research based python. The Raspberry pi has been strongly supported by the
on the factors and resources available are enlisted below. python programming. Hence we choose to work with Python
IDE.
A. Scope of Research 6) Opencv (Intel open source library)
The system will be composed of raspberry pi and pi camera OpenCV is an open source library for video and image
as hardware devices. The programming of the system will be processing and it is very efficient and effective for various
done using Raspbian OS. The programming language used will operations related to our proposed system. OpenCV is
be python and openCV library. The system will work with lightweight in terms of processing.
video obtained in raw format h.264 from raspberry pi camera at
1296 x 730 resolution 1-49fps or 640 x 480 at 1-90fps. The 7) Eclipse Ide for remote debugging
system will detect heavy vehicles including trucks, buses, and Eclipse can be used for remote debugging. We can
cars and lane of the vehicle will be plotted. The vehicle connect remote raspberry and Eclipse on local computer for
detection will be experimented in live environment. The debugging the programs on raspberry pi. As Eclipse IDE has
system will detect anomaly and accidents or unusual behavior good debugging support hence it provides us ease to work on
of the vehicle. The accident detection will be performed in Raspberry pi.
modelled/simulated environment or offline data due to
limitation of resources. C. Architectural diagrams
B. Objective of Research
To survey analyze and study different techniques related to
vehicle, lane and accident detection for traffic surveillance. To
test the various techniques on the proposed combination of
hardware and check the output according to different
parameters. Selection of technique for steps described in Frame
processing diagram Fig 2 according to the survey or tested
results of experiments. To develop system with those selected
methods/algorithm for detecting number, lane and accident of
vehicles with help of input video captured from camera at
specific needed parameters. To try implementing and testing
this system in live environment for vehicle detection.
Hardware:- Fig. 2. Architecture of Proposed System
1) Raspberry pi
Raspberry pi is a very small sized computer. It has an
ARM computable processor of 1.2 GHZ and ram of 1
Gigabyte. It has various operating system as option to
develop various IoT projects. Raspberry pi also contains
GPIO pins which are also programmable for various other
operations. Due to small size can easily be fit for our
proposed system.
2) pi Camera
Pi camera is a camera module that can be used with
raspberry pi. There is slot in Raspberry pi to directly
connect the pi camera. Pi camera provides input video in
resolution of 640, 720, or 1080 HD. Hence it can be easily
used for our purpose. Different factors like brightness,
intensity can be adjusted for Pi camera. Fig. 3. Block Diagram of Video Frame Processing
3) Battery as power source The Fig. 2 presents the architecture of the proposed system.
The input is obtained from Pi camera which is attached to
4) Internet dongle for remote access Raspberry pi, the input is then forwarded to the Python code
which is executed on Raspbian os. The python program will
5
International Conference on Innovations in Power and Advanced Computing Technologies [i-PACT2017]
V. CONCLUSION
An in-depth survey of approaches used by researchers for
vehicle detection, vehicle tracking, and accident detection is
presented. The available open source video datasets, useful for
offline experimentation, are also discussed. From survey, we
conclude that (i) Hidden Markov Model and Neural Network
methods are more efficient and compatible to experiment with
accident detection as these methods give good accuracy, (ii)
Lucas-Kanade, a technique based on optical flow, and K-
means clustering, a technique based on centroid calculation of
Fig. 5. Output obtained using three techniques on MIT Traffic dataset the moving object, are useful for vehicle tracking. The
experiments on VIRAT and MIT traffic datasets were
conducted and it is concluded that for vehicle detection
Gaussian mixture model and frame difference techniques
prove more effective. The canny edge detection technique
6
International Conference on Innovations in Power and Advanced Computing Technologies [i-PACT2017]
performed well for finding the edge of the vehicle. The Transportation Systems Conference Toronto Canada, pp. 1507-1512,
weather conditions, power supply and processing power are September 2006
[16] M. Isaloo and Z. Azimifar, “Anomaly Detection on Traffic Videos
critical parameters to be considered while designing IoT based
Based on Trajectory Simplification”, IEEE 8th Iranian Conference on
traffic surveillance system. In future, we plan to develop IoT Machine Vision and Image Processing (MVIP), pp. 200-203, 2013
based traffic surveillance system. We aim to test the system in [17] W. Xiao, D. Xie, and T. Tan, “Traffic Accident Prediction Using
live environment with a vision of its practical implementation Vehicle Tracking and Trajectory Analysis”, IEEE Conference
in daily life. Intelligent Transportation Systems, pp. 220-225, 2003
[18] N. Kanhere, S. Birchfield and W. Sarasua, ”Vision Based Real Time
REFERENCES Traffic Monitoring”, United States Patent, Feb 2013
[19] Canny, "A computational approach to edge detection," IEEE
[1] S. Kamijo, Y. Matsushita, K. Ikeuchi and M. Sakauchi. "Traffic
Transactions on Pattern Analysis and Machine Intelligence, no.6, pp.
monitoring and accident detection at intersections." IEEE. Trans. on
679-698, 1986
Intelligent transportation systems, vol. 1, no. 2, pp. 108-118, June 2000
[20] P. KaewTraKulPong and R. Bowden, “An Improved Adaptive
[2] R. Cucchiara, M. Piccardi, and P. Mello, ” Image Analysis and Rule-
Background Mixture Model for Real-time Tracking with Shadow
Based Reasoning for a Traffic Monitoring System” IEEE transactions
Detection”, Proc. 2nd European Workshop on Advanced Video Based
on Intelligent transportation systems, vol. 1, No. 2, pp. 119-130, June
Surveillance Systems, Sept 2001
2000
[21] A. Ansari, M. Sedky, N. Sharma, A. Tyagi, “An Internet of Things
[3] C. Lin, J. Tai and K. Song, “Traffic Monitoring Based on Real-Time
Approach for Motion Detection using Raspberry Pi”, IEEE
Image Tracking”, IEEE International Conference on Robotics &
International Conference on Intelligent Computing and Internet of
Automation, Taiwan, pp. 2091-2096, September 2003
Things (IC1T), pp. 131-134, 2015
[4] Y. Jung, K. Lee, and Y. Ho, “Content-Based Event Retrieval Using
[22] "A Large-scale Benchmark Dataset for Event Recognition in
Semantic Scene Interpretation for Automated Traffic Surveillance”
Surveillance Video" by Sangmin Oh, Anthony Hoogs, Amitha Perera,
IEEE transactions on Intelligent transportation systems, vol. 2, no. 3,
Naresh Cuntoor, Chia-Chih Chen, Jong Taek Lee, Saurajit Mukherjee,
pp. 151-163, September 2001
J.K. Aggarwal, Hyungtae Lee, Larry Davis, Eran Swears, Xiaoyang
[5] J. Melo, A. Naftel, A. Bernardino, and J. Santos-Victor, “Detection and
Wang, Qiang Ji, Kishore Reddy, Mubarak Shah, Carl Vondrick, Hamed
Classification of Highway Lanes Using Vehicle Motion Trajectories”,
Pirsiavash, Deva Ramanan, Jenny Yuen, Antonio Torralba, Bi Song,
IEEE transactions on Intelligent transportation systems, vol. 7, no. 2,
Anesco Fong, Amit Roy-Chowdhury, and Mita Desai, in Proceedings
pp. 188-200 June 2006
of IEEE Comptuer Vision and Pattern Recognition (CVPR), 2011,
[6] Y. Ki, and D. Lee, “A Traffic Accident Recording and Reporting
release 2.0
Model at Intersections”, IEEE. Trans. on Intelligent transportation
[23] X. Wang, X. Ma and E. Grimson, Unsupervised Activity Perception in
systems, vol. 8, no. 2, June 2007
Crowded and Complicated scenes Using Hierarchical Bayesian Models,
[7] R. Sharma, G. Taubel, and J. Yang, “An Optical Flow and Hough
IEEE Transactions on Pattern Analysis and Machine Intelligence
Transform Based Approach to a Lane Departure Warning System”,
(PAMI), Vol. 31, pp. 539-555, 2009
IEEE International Conference on Control & Automation (ICCA), pp.
688-693, June 18-20, 2014
[8] Z. Moutakki, T. Ayaou, K. Afdel, A. Amghar, “Prototype of an
embedded system using Stratix III FPGA for vehicle detection and
traffic management”, IEEE International Conference on Multimedia
Computing and Systems (ICMCS), pp. 141-146, April 2014
[9] P. S.S, V. Vibha and C. Manisha, “Vehicle Detection and Tracking
using the Optical Flow and Background Subtraction”, Elsevier Proc. of
Int. Conf. on Advances in Computer Science and Application, pp. 741-
747, 2013
[10] G. Cocorullo, P. Corsonello, F. Frustaci, L. Guachi, S. Perri,
“Embedded Surveillance System Using Background Subtraction and
Raspberry PI”,IEEE AEIT International Annual Conference (AEIT),
2015
[11] N. Chintalacheruvu, V. Muthukumar, “Video Based Vehicle Detection
and Its Application in Intelligent Transportation Systems”, Scientific
Research Journal of Transportation Technologies, vol. 2, pp. 305-314,
October 2012
[12] M. Kochláßn, M. Hodoßn, L. ßCechovißc, J. Kapitulík, M. Jureßcka,
“WSN for traffic Monitoring using Raspberry Pi Board”, Proc. of the
2014 Federated Conference on Computer Science and Information
Systems, vol. 2, pp. 1023–1026, 2014
[13] H. Weiming, X. Xiao, D. Xie, T. Tan, and S. Maybank. "Traffic
accident prediction using 3-D model-based vehicle tracking." IEEE
Transactions on Vehicular Technology, pp. 677-694, vol. 53, no. 3,
May 2004.
[14] Z. Yu, Y. Chen, “A real-time motion detection algorithm for traffic
monitoring systems based on consecutive temporal difference”, Proc. of
the 7th Asian Control Conference, Hong Kong, China,, pp. 1594-1599,
August 2009
[15] J. Ploetner and Mohan M. Trivedi, “A Multimodal Framework for
Vehicle and Traffic Flow Analysis”, IEEE Conference on Intelligent