Bapuji Educational Association

Bapuji Institute of Engineering and Technology

Department of Computer Science and Engineering

 Progyan 3.0: Inhouse project exihibition

ON
"Video-Based Suspicious Human Behaviour
Recognition System"
Project Guide: Project Co-ordinator :
Dr. Chetana Prakash  M.Tech., Ph.D., Dr. Roopa G M Ph.D
Department of CS&E. B.I.E.T Department of CS&E, B.I.E.T

Presented by:
Head of the Department:
Chandana
   M                 4BD17CS033
Dr. Nirmala C R Ph.D.
Deeksha J Udasi           4BD17CS038
Department of CS&E
Gousiya Banu I             4BD17CS044 B.I.E.T. 
Aneesa Banu                 4BD17CS122
 ABSTRACT

In recent years, the number of surveillance cameras installed to monitor

private and public spaces has increased rapidly. The demand has raised for
smarter video surveillance of public and private spaces using intelligent
vision systems which can differentiate between 'suspicious' and
'unsuspicious' behaviors according to the human observer. Generally, the
video streams are constantly recorded or monitored by operators. In these
cases, an intelligent system can give more accurate performance than a
human. We have proposed a method called motion influence map under
machine learning for representing human activities. The key feature of the
proposed motion influence map is that it effectively reflects the motion
characteristics such as movement speed, movement direction, and size of the
objects and their interactions within a frame sequence. It further extracts
frames of high motion influence values and compares with the testing
frames to automatically detect suspicious activities.
 • Introduction
• LiteratureSurvey
• Problem statement 
• Proposed solution
• Objective
• Functional and Non-Functional
CONTENTS requirements
• Software and Hardware
requirements
• Model design
• Implementation Details
• Results
• Conclusion
• Future Scope
 INTRODUCTION
  Video cameras are extensively used in
surveillance application to examine
public areas, such as train stations,
airports and shopping centers.
  When crowds are intense,
automatically tracking individuals
becomes a difficult task. 
 Human behavior detection is also
known as outlier detection, which is
applicable in a variety of applications.
  Crowded environments are very complicated to monitor by human observer,
whether live or by means of video surveillance, because the optical patterns
are highly recurring and the difficulty of the movement characterize the
scene is often overpowering.
Applications 
Surveillance 
Cameras installed in areas that may need
monitoring such as banks, airports, military
installations, and convenience stores.
Currently, surveillance systems are mainly
for recording. 
The Aim for activity detection using
CCTV's is to monitor suspicious activities
for real-time reactions like fighting and
stealing.

Sports play analysis

Analyzing the play and deducing
the actions in the sport
LITERATURE SURVEY
 AUTHORS AND
SI
NO LITERATURE SURVEY
PUBLISHED
YEAR
TITLE METHOD REMARKS

Gatt T, Seychell D,and Detecting human abnormal robotized camera- human body key points extricated
Dingli A[1] behavior through a video based framework that from a conventional camera can
1 generated model can detect be utilized to distinguish unusual
unpredictable human human conduct
conduct

2 Liu C, Ying J, Han F Abnormal Human Activity human movement bayes classifier and
and Ruan M[2] Recognition using Bayes acknowledgment Convolutional Neural Network
Classifier and technique are utilized in the framework to
Convolutional Neural recognize four practices,
Network including moving, and thumping.

3 . Jain C and Gautam Abnormal behavior constant item's This work can be utilized to build
D[3] detection at traffic conduct discovery is up a reconnaissance arrangement
junctions using Lucas executed utilizing of static camera and mechanical
Kanade and Harris Corner Lucas Kanade and robotization visual frameworks.
detector Harris Corner based
methodology

4 . Hsu SC, Chuang CH, A video-based abnormal strange human An ordinary conduct can be
Huang CL, Teng R and human behavior detection conduct recognition portrayed by the spatial and
Lin MJ[4] for psychiatric patient framework for fleeting highlights of human
monitoring observing mental exercises.
patient
SI AUTHORS AND TITLE METHOD REMARKS
NO PUBLISHED YEAR

Lahiri D, Dhiman C and Abnormal human Average Energy Image strange human action
Vishwakarma DK [5] actionrecognition based element descriptor acknowledgment by coordinating
5 using average is intended for strange HOG and PCA with AEI
energy images human action
acknowledgment by
coordinating HOG and
PCA with AEI

6 Ashokan V and Murthy Comparative HOG method

OR[6] evaluation of Highlight Extraction is finished by
classifiers HOG method for one of the SVM or
forabnormal KNN classifiers
event detection
in ATMs

7 Lung FB, Jaward MH and Spatio-temporal Spatio-Temporal a Spatio-Temporal Descriptor (STD)

Parkkinen J[7] descriptor Descriptor in view of spatio-transient highlights
forabnormal of a picture arrangement.
human activity
detection

8 Nandhini K, Pavithra M, Anamoly irregular detectors 98% F1 score

Revathi K and Rajiv A[8] detection for
safety monitoring
PROBLEM STATEMENT

As the importance of security increases in human life, a great

number of surveillance cameras have been installed in private and
public places. However, the plethora of video sequences available is
overwhelming the human resources monitoring them. They perform
well but are limited to detecting only those specific types of
predefined anomalies. To overcome this problem, there has been
significant interest in a smart surveillance system that can
automatically detect unusual or suspicious activities.
 PROPOSED SOLUTION

Our project comes forward as an attempt to provide solution

to such a problem as the model developed is a smart
surveillance system which can detect unusual or suspicious
activity automatically. A method for representing the motion
characteristics is described for detection and localization of
unusual activities in a crowded scene on a generalized
framework. 
 OBJECTIVES

• To collect the datasets in the form of videos and perform pre-

processing.

• To identify the heed points in the videos which acts as features

and extracted through motion influence map generation
technique.

• To develop a highly coherence suspicious human activity

detection system which are the behaviour representation and
modelling.

• To calculate the metrics for performance evaluation with our

datasets. 
FUNCTIONAL AND NON-FUNCTIONAL
REQUIREMENTS

THE FUNCTIONAL REQUIREMENTS 

 THE SYSTEM SHOULD ALLOW USERS TO UPLOAD VIDEOS AS INPUT.
 THE SYSTEM MUST PERFORM PRE-PROCESSING OF VIDEO.
 THE SYSTEM SHOULD DETERMINE THAT VIDEO CONTAINS SUSPICIOUS
OR UNSUSPICIOUS BEHAVIOR AS OUTPUT.

NON-FUNCTIONAL REQUIREMENTS 
§ FLEXIBILITY
§ ECONOMIC FEASIBILITY
§ RELIABILITY
§ MODIFIABILITY
§ PORTABILITY 
§ SCALABILITY
SOFTWARE AND HARDWARE REQUIREMENTS

SOFTWARE REQUIREMENTS
•    OPERATING SYSTEM         : WINDOWS 10.
•    PLATFORM                       : PYTHON TECHNOLOGY
•    FRONT END                     : HTML, CSS, JAVASCRIPT
•    TOOL                              : PYTHON 2.7 AND ABOVE

HARDWARE REQUIREMENTS
•     SYSTEM                         : PENTIUM IV 2.4 GHZ.
•    HARD DISK                    : 40 GB.
•    MONITOR                       : 15 INCH VGA COLOR
•    RAM                             : 4GB
Flow Diagram
of System
MODEL DESIGN
• Firstly as input, a video file with extension .avi which consists of both video
and audio is inserted. 
• Then the input data is subjected to data pre-processing and a video is treated as
sequence of images called frames and these frames are processed sequentially.
• After the pre-processing step, feature extraction is done where motion influence
map of the scene is built, and can find the mass that contains a suspicious event
where it has a characteristic motion beam. Each scene will be divided into a
group of mega blocks each containing the motion effect.
• For each mega block, K-means clustering using the spatio-temporal features is
performed, Here, one should note that in the training stage, only video clips of
normal activities are used. Therefore, the codewords of a mega block model and
the patterns of usual activities can occur in those respective areas only. 
K-means Algorithm
The working of K-Means algorithm: 

Step 1 − First, the number of clusters must be

specified, K, need to be generated by this
algorithm.

Step 2 − Next, randomly select K data points

and assign each data point to a cluster. In
simple words, classify the data based on the
number of data points.

Step 3 − Now it will compute the

cluster centroids.

Step 4 − Next, keep iterating the

following until optimal centroid is
found which is the assignment of data points
to the clusters that are not changing any more.
DATASET DESCRIPTION

 Two datasets are used : UMN and UCSD

• The dataset consists of 11 video clips of crowded escape scenarios from two
different indoor and outdoor scenes. 
• Each video clip starts with people walking around the scene. In this case,
walking was considered a normal activity. 
• In each clip, each of the subjects make a sudden quick running movement
away from the scene, which is considered unusual in this dataset.
 UMN Datasets

Unsuspicious Behaviour

Suspicious Behaviour
UCSD Datasets

Unsuspicious Behaviour suspicious Behaviour

 IMPLEMENTATION DETAILS

Our implementation consists of five modules: 

• Video Pre-processing
• Optical flow of blocks
• Motion influence generator
• Mega block generator
• Training
• Testing
Video Pre-processing
The video file is given as an input to the system, which is
subjected to pre processing. A video is treated as sequence of
images called frames and these frames are processed sequentially.
An RGB frame is first converted to gray scale.

Optical flow of blocks 

Optical flow for each frame in the video is calculated for each pixel
in a frame after the pre-processing step. Optical flow is the pattern
of apparent motion in a visual scene of objects, surfaces, and edges
caused by the relative motion between an observer and the scene.
 
Motion influence generator

Motion influence map is used to describe the interaction characteristic of the

objects considering the factors of speed, direction, distance, visibility and the
information transfer modes. On the basis of the constructed motion influence
map, the detection and localization of the suspicious behaviors were realized
simultaneously with the help of  minimum-distance matrix

Megablock generator
The development of Mega blocks frames is divided into non-overlapping mega
blocks, every one of which is a blend of different squares of control on
movement. A Mega block's Motion Influence value is the summation of the
values of motion influence of all the smaller blocks which constitute a larger
block. 
Training

We only use video clips of normal activities during our training stage.  
Testing
Testing is a critical element which assures quality and effectiveness of the
proposed system in meeting its objectives. 
The Test cases in unit testing are as follows:

Module 1: Pre-Processing  

Test case Id Pre-processing

Input Videos
Output Optical Flow of frames is developed
Description  Frames undergoes a series of pre-processing
to form a one dimensional vector
Passed(?) yes
Module 2: Feature Extraction

Test case Id Feature-Extraction

Input Optical flow of each blocks
Output Feature is extracted
Description Motion influence map, a block in which an unusual activity
occurs, along with its neighbouring blocks, has unique
motion influence vectors
Passed(?) Yes

Module 3: Training

Test case Id Training

Input Features
Output A set of combinations learnt
Description we perform clustering using the spatio-temporal features
and set the centers as codewords
Passed(?) Yes
Module 4: Testing

Test case Id Testing

Input Learn combination set
Output Output the time in which the abnormality is detected
Description Use the combination to reconstruct test frame and
check the error for abnormality
Passed(?) Yes
Demonstration
RESULTS
The model detects the suspicious behaviour of the subject in the frame and
creates a red  bounding box around the same which says the action is 
suspicious with accuracy of 98%
Performance graph 
Confusion matrix 
CONCLUSION
With the growing number of surveillance cameras installed in private
and public areas, there has been a demand for the automatic and
intelligent analysis of video sequences using computers to detect
suspicious activities. In this project, we proposed a method for
representing the motion characteristics within a frame to detect unusual
human activities in a video. Owing to the representational power of the
proposed motion influence map for both space and time, we can
classify a frame as suspicious or unsuspicious. For a real application, a
smart surveillance system needs to efficiently detect unusual activities
within a unified framework. In our datasets, i.e., the UMN and UCSD,
we validated the effectiveness of the proposed method, which
outperformed other competing methods from the literature.
 FUTURE WORK

• The main focus of this work is to detect suspicious activities within a

surveillance video scenes, for which the cameras usually cover a wide
area, resulting in small objects being present in the scene without
significant perspective changes. 
• Also our experiments were limited to a fixed viewpoint, and there is a
limitation in the applicability of the approach for surveillance cameras
with pan, zoom, or tilt functionality. 
• At this moment the proposed method deals only with static cameras.
However, it can be easily extended to high resolution cameras using
localization results.
REFERENCES
1.Gatt T, Seychell D,and Dingli A,"Detecting human abnormal behavior  through a video
generated model". In 2019 11th International Symposium on Image and Signal Processing and
Analysis (ISPA) 2019  Sep 23 (pp. 264-270). IEEE.

2. Liu C, Ying J, Han F and Ruan M,"Abnormal Human Activity Recognition using Bayes
Classifier and Convolutional Neural Network". In 2018  IEEE 3rd International Conference on
Signal and Image Processing  (ICSIP) 2018 Jul 13 (pp. 33-37). IEEE.

3. Jain C and Gautam D," Abnormal behavior detection at traffic junctions using Lucas Kanade
and Harris Corner detector". In 2018  4th International Conference on Recent Advances in
Information Technology (RAIT)  2018 Mar 15 (pp. 1-5). IEEE.

4. Hsu SC, Chuang CH, Huang CL, Teng R and Lin MJ,"A video-based abnormal human
behavior detection for psychiatric patient monitoring". In 2018 International Workshop on
Advanced Image Technology (IWAIT) 2018 Jan 7 (pp. 1-4). IEEE.
5. Lahiri D, Dhiman C and Vishwakarma DK ," Abnormal human actionrecognition
using average energy images". In 2017 Conference onInformation and
Communication Technology (CICT) 2017 Nov 3 (pp.1-5). IEEE.

6. Ashokan V and Murthy OR, "Comparative evaluation of classifiers forabnormal

event detection in ATMs". In 2017 International Conference on Intelligent
Computing, Instrumentation and Control Technologies (ICICICT)  2017 Jul 6 (pp.
1330-1333). IEEE.

7. Lung FB, Jaward MH and Parkkinen J," Spatio-temporal descriptor forabnormal

human activity detection". In 2015 14th IAPR International Conference on Machine
Vision Applications (MVA) 2015 May 18 (pp.471-474). IEEE.

8. Nandhini K, Pavithra M, Revathi K and Rajiv A, "Anamoly detection forsafety

monitoring". In 2017 Fourth International Conference on Signal Processing,
Communication and Networking (ICSCN) 2017 Mar 16(pp. 1-6). IEEE.
THANK YOU