Synopsis Final
Synopsis Final
Weapon detection is a very serious and intense issue as far as the security
and safety of the public in general, no doubt it’s a hard and difficult task fur-
thermore, its troublesome when you need to do it automatically or with some
of the AI model. Video Surveillance plays an important role in every aspect
of life like theft detection, unusual happenings in crowded places, monitoring
the suspicious activities of each individual to provide a secure and hassle free
environment. Footage of closed circuit television (CCTV) camera is taken as
an evidence to track the suspicious act. It is very tough to operate surveil-
lance cameras with human intervention to detect abnormal activities. Fully
automating surveillance with smart video capturing capabilities using deep
learning technique is one of the most advanced means of remotely monitoring
strange activities with exact location, time of event occurred along with fa-
cial recognition of criminal. Finding misdemeanor activity in a public place
is very difficult to observe, as many objects are involved in the real time sce-
nario. An uncommon or doubtful incidents in public places are captured in
CCTV cameras which promotes police force to safeguard people before any
mishap happens. It helps police to reach that spot on time and rescue victim.
All these are meant to be achieved by using YOLO (You Only Look Once)
object detection models and its variants like YOLO V1, V2, V3, V4 and
latest V5 which is 88system helps in identifying weapons held by a person as
well as face recognition to identify the suspicious user. Using YOLO v5, it is
very simple to track objects like weapons in a crowd. Low resolution images,
far away and out of focus in the scene can also be captured and identified
accurately.
Contents
2.0 Introduction
At present weapon detection at all public places includes sensors for detecting
suspicious objects. Sensors are very expensive, not secure, and not efficient
and also it cannot cover large area under surveillance. To overcome the
drawbacks of conventional system, we focus on machine learning algorithms
for object detection whose efficiency is better than using sensors alone. The
algorithm is applied to all regions of input image and finds the highest score
as the region of detection, which is a time consuming task for processing
large number of images but with deep learning based You Only Look Once
(YOLO) algorithm the object detection process is simplified by applying the
algorithm to entire input image and the region of interest is highlighted with
bounding box concept to detect different real time images with higher accu-
racy. Through deep learning by using YOLO V5 algorithm. Misdemeanor
activities are detected easily and precisely in a Crowd. Through this algo-
rithm high and low level objects like weapons, unusual things irrelevant to
the situation is recognized and identified. This also enhances the localization
tasks. In addition to this, facial recognition is also implemented through
which it increases the speed by eliminating different object categories and
replacing them with facial features. Thus the irregularity through webcam is
monitored and is been prevented before cause. You Only Look Once (YOLO)
algorithm is very popular for real time object detection. It uses YOLOv5
which is an advanced version of YOLO. YOLOV5 is fast and accurate when
compared to earlier version of YOLO.The Difference between Yolov5 and
Yolov4 is as follows Yolov5 is very small which weighs around 27 megabytes
whereas yolov4 is 244 megabytes with DarkNet architecture. Yolov5 is 90can
be measured by using following metrics i) mAP-mean average precision, 2)
p-precision, 3) R-recall. This paper focuses particularly on three things i)
weapon detection such as a person holding a knife, gun, pistol and rifles
in public places ii)face detection of a person holding weapon will also be
observed by surveillance camera as a suspicious even by extracting features
from each segment. iii) It also monitors any suspicious activities like rising
the arms suddenly, bending down, and other abnormal activities performed
by a person will be discovered so it is a multiclass classification problem.
Identification of specific object among several real time objects is very tough
by using Video surveillance which covers multiple real time objects.
Figure 1: DETECTION OF OBJECTS USING YOLO
• Weapon Detection
• Weapon Classification
• Surveillance Systems
• Behavior Monitoring
• Alert Generation
4.0 Domain of Project
To avoid strange activities happening in society, detecting weapons, grouping
of people and threatening activities is achieved by using YOLOV5 algorithm.
-
10.0 List of Features
1. Weapon Detection 2. Tracking 3. Suspicious activity identification
As shown below in the figure, there are three phases of architecture Object
detection, Analysis and Action. Each phase has its own steps including
training and analysis of datasets, detecting and identifying objects from the
video and lastly alerting the system of any suspicious and abnormal activity
or any detected deadly weapons.
12.0 List of Modules and Functionality
1. Data Acquisition and Preprocessing:
Collect data from various sensors (visual cameras, thermal cameras, au-
dio sensors). Preprocess data to ensure uniform quality, correct format,
and remove noise.
2. Weapon Detection Module:
Detect potential weapons in captured images or video streams. Utilize
machine learning models (such as CNNs) trained to identify weapon
signatures. Output the location and type of detected weapons.
3. Feature Extraction:
Extract relevant features from sensor data for both weapon detection
and behavior analysis. For weapon detection, features might include
shape, color, and thermal signatures. For behavior analysis, features
might include motion vectors, body keypoints, and social interactions.
4. Integration and Fusion:
Combine outputs from weapon detection and behavior analysis mod-
ules. Create a comprehensive situational awareness by considering both
potential threats and context.
5. Anomaly Detection:
Compare detected behaviors against expected patterns. Flag behaviors
that are unusual or potentially threatening.
6. Alert Generation and Prioritization:
Generate alerts based on the outputs of weapon detection and behavior
analysis. Assign priorities to alerts based on the perceived threat level.
7. Human Verification and Intervention:
Allow human operators to review alerts and verify potential threats.
Provide real-time visual feeds and summaries of detected behaviors.
Enable operators to take appropriate actions, such as alerting security
personnel.
8. Feedback Loop and Learning:
Continuously improve the system’s accuracy and performance based
on operator feedback. Update machine learning models and behavior
patterns to adapt to evolving scenarios.
9. Continuous Improvement and Adaptation:
Regularly update and refine machine learning models to stay effective
against evolving threats. Adapt the system to changes in the environ-
ment and user needs.
• Objectives :
14.0 Methodology
1. Dataset
Raw images are not appropriate for analysis purposes and need to be
converted into the processed format, such as jpeg, jpg, and tiff for
further analysis. The image size is reconstructed into a square image.
The images were resized into 416px x 416px resolution to reduce the
computational time and then the images were then retained in the RGB
format. Dataset is created by collecting the good pixel weapon images
and making them ready for the creation of the dataset.
– Python3 Libraries:
– OpenCV
– NumPy
– TensorFlow or PyTorch
– Scikit-learn
– Dlib
– Imutils
• Hardware requirements:
18.0 Algorithms
In behavior analysis in public spaces using OpenCV, several algorithms and
techniques are commonly employed to detect and analyze human behaviors.
Some of the key algorithms used are:
• Object Detection Algorithms:
Object detection algorithms, such as YOLO (You Only Look Once),
Faster R-CNN, and SSD (Single Shot Multi-Box Detector), are com-
monly used to locate and identify objects in image or video frames,
including weapons.
• Convolutional Neural Networks (CNNs):
CNNs are crucial for feature extraction in object detection. They rec-
ognize patterns and features in image data.
• Deep Learning for Image Classification:
Deep learning models are trained to classify objects, including weapons,
based on the visual features extracted from images using TensorFlow,
PyTorch These models may include CNNs and fully connected layers.
• Real-time Video Processing Algorithms:
Algorithms for real-time video processing, such as optical flow algo-
rithms, frame differencing, and background subtraction by using RTSP,
are used to efficiently analyse video streams with OpenCV.
• Preprocessing Algorithms:
Image and video preprocessing techniques are used to enhance the qual-
ity of input data. These can include techniques for noise reduction,
contrast adjustment, and resizing using ResNet,MobileNet.
• Alerting Algorithm:
An algorithm is needed to generate alerts when a weapon is detected.
This typically involves sending notifications to security personnel or
relevant authorities through email, SMS, or other communication chan-
nels.
• Weapon detection
• Weapon classification
• Object detection
• Alert generation
21.0 References
[1] T. O’Shea and J. Hoydis, “An introduction to deep learning for the phys-
ical layer,” IEEE Trans. Cogn. Commun. Netw., vol. 3, no. 4, pp. 563–575,
Dec. 2017, doi: 10.1109/TCCN.2017.2758370.
[2] G. Aceto, D. Ciuonzo, A. Montieri, and A. Pescapé, “Mobile encrypted
traffic classification using deep learning: Experimental evaluation, lessons
learned, and challenges,” IEEE Trans. Netw. Service Manag., vol. 16, no.
2, pp. 445–458, Feb. 2019, doi: 10.1109/ TNSM.2019.2899085
[3] A. Taha, H. H. Zayed, M. E. Khalifa, and E.-S. M. El-Horbaty, “Exploring
behavior analysis in video surveillance applications,” Int. J. Comput. Appl.,
vol. 93, no. 14, pp. 22–32, May 2014, doi: 10.5120/16283-6045
[4] Dong-Gyu Lee, Hrung-II Suk, Sung-Kee Park, Seong-Whan Lee “Motion
Influence Map for Unsual Human Activity Detection”, 2015.
[5] Arun Kumar Jhapate, Sunil Malviya, Monika “Unusual Crowd Activity
Detection using OpenCV and Motion Influence Map”,2020.
[6] P.Bhagya Divya, S.Shalini, R.Deepa, Baddeli Sravya Reddy,“Inspection
of suspicious human activity in the crowdsourced areas captured in surveil-
lance cameras”,International Research Journal of Engineering and Technol-
ogy (IRJET), December 2017.
[7] Jitendra Musale,Akshata Gavhane, Liyakat Shaikh, Pournima Hagwane,
Snehalata Tadge, “Suspicious Movement Detection and Tracking of Human
Behavior and Object with Fire Detection using A Closed Circuit TV (CCTV)
cameras ”, International Journal for Research in Applied Science Engineering
Technology (IJRASET) Volume 5 Issue XII December 2017.
[8] H. Kuehne, H. Jhuang, E. Garrote, T. Poggio, and T. Serre, “HMDB:
A large video database for human motion recognition,” in Proc. Int. Conf.
Comput. Vis., Nov. 2011, pp. 2556–2563.
[9] E. Bermejo, O. Deniz, G. Bueno, and R. Sukthankar, “Violence detection
in video using computer vision techniques,” in Computer Analysis of Images
and Patterns. Berlin, Germany: Springer, 2011, pp. 332–339.
[10] D. Freire-Obregón, M. Castrillón-Santana, P. Barra, C. Bisogni, and
M. Nappi, “An attention recurrent model for human cooperation detection,”
Comput. Vis. Image Understand., vols. 197–198, Aug. 2020, Art. no.
102991.
[11] T. Simon, H. Joo, I. Matthews, and Y. Sheikh, “Hand keypoint detec-
tion in single images using multiview bootstrapping,” in Proc. IEEE Conf.
Comput. Vis. Pattern Recognit. (CVPR), Jul. 2017, pp. 1145–1153.
[12] JIANYU XIAO SHANCANG L, (Member, IEEE), QINGLIANG XU11
School of Computer Science and Engineering at Central South University,
China”Video-based Evidence Analysis and Extraction in Digital Forensic In-
vestigation”, IEEE Access (2020).
[13] An Improved YOLOv3-based Neural Network for De-identification Tech-
nology Ji-hun, Dong-hyun Lee, Kyung-min Lee, Chi-ho Lin Won School of
Computer Semyung University Chungcheongbuk-do, Korea,IEEE 2018.
[14] ] Francisco Luque Sanchez, Isabella Hupont, Siham Tabik,Francisco Her-
reraF, “Revisiting crowd behavior analysis through deep learning: Taxonomy
anomaly detection, crowd emotions, datasets, oppurtunities and prospects.”,
ELSEVIER (2019).
[15] Elizabeth Scaria, Aby Abahai T and Elizabeth Isaac, “Suspicious Ac-
tivity Detection in Surveillance Video using Discriminative Deep Belief Net-
wok”, International Journal of Control Theory and Applications Volume 10,
Number 29 -2017.