Volume 10, Issue 3, March – 2025 International Journal of Innovative Science and Research Technology

ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25mar088

Smart Video Monitoring: Advanced Deep

Learning for Activity and Object Recognition
Shashikumar D R1; Tejashwini N2; K N Pushpalatha3; Anurag Kumar4;
Om Chavan5; Atharva Mishra6
1
Computer Science and Engineering Sai Vidya Institute of Technology Bengaluru, Karnataka, India
2
Computer Science and Engineering Sai Vidya Institute of Technology Bengaluru, Karnataka, India
3
CSE (Data Science) Sai Vidya Institute of Technology Bengaluru, Karnataka, India
4
Computer Science and Engineering Sai Vidya Institute of Technology Bengaluru, Karnataka, India
5
Computer Science and Engineering Sai Vidya Institute of Technology Bengaluru, Karnataka, India
6
Computer Science and Engineering Sai Vidya Institute of Technology Bengaluru, Karnataka, India

Publication Date: 2025/03/17

Abstract: This study explores the integration of Convolutional Neural Networks (CNNs) and Long Short-Term Memory
(LSTM) networks for the real-time recognition of human activities in video data. By harnessing the advantages of these
two approaches, the system achieves high accuracy in detecting complex human actions. Specifically, CNNs address the
spatial aspects of the task, while LSTMs handle the temporal sequences. A notable feature of the system is its
categorization module, which enables users to select an action and identify similar actions, thereby enhancing productivity
and usability.

Existing models often face challenges related to real-time inter- action capabilities and resilience to environmental
disturbances. This study tackles these shortcomings by refining the CNN-LSTM framework to support real-time
functionality and incorporating preprocessing techniques, such as frame extraction and normal- ization, to improve input
data quality. The system’s effectiveness is measured using indicators like accuracy, recall, and latency, demonstrating its
advantages over traditional rule-based and basic deep learning approaches. The early findings are optimistic,
demonstrating significant improvements in performance.

Nevertheless, challenges remain, particularly in tracking per- formance under occlusion or in cluttered environments.
Future research should explore the integration of multi-modal data and advanced architectures, such as spatio-
temporal graph con- volutional networks (STGCN), to further enhance recognition accuracy and system robustness.

In conclusion, the proposed CNN-LSTM hybrid architecture for activity recognition demonstrates potential for
applications in video surveillance and beyond, including fields like healthcare and sports analytics. The system offers
improved automated monitoring capabilities through enhanced accuracy, scalable human action detection, and user-
friendly design.

How to Cite: Shashikumar D R; Tejashwini N; K N Pushpalatha; Anurag Kumar; Om Chavan; Atharva Mishra (2025). Smart
Video Monitoring: Advanced Deep Learning for Activity and Object Recognition. International Journal of
Innovative Science and Research Technology, 10(3), 168-172.
https://doi.org/10.38124/ijisrt/25mar088

I. INTRODUCTION video data, underscoring the need for automated systems

capable of efficiently and accurately recognizing human
 Context and Motivation activities. Recent developments in deep learning offer
Video surveillanceserves an essential function in powerful tools that could revolutionize surveillance
ensuring public safety and protecting private property. practices, facilitating smarter and more efficient monitoring.
These systems serve as a strong deterrent to crime and
provide essential situational awareness. However, traditional  Problem Statement
surveillance ap- proaches rely heavily on human operators While progress has been achieved in activity
to monitor video streams, making them both labor- recognition, existing surveillance systems still face
intensive and susceptible to errors due to operator fatigue. significant obstacles. Challenges such as varying lighting
This challenge is exacer- bated by the growing volume of conditions, occlusions, and dense environments impact the

IJISRT25MAR088 www.ijisrt.com 168

Volume 10, Issue 3, March – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25mar088
accuracy of human activity detection models. Furthermore, innovations, in- cluding 3D CNNs and Spatiotemporal
many current solutions are resource-intensive, which hinders Graph Convolutional Networks (STGCNs), further
their ability to achieving the real-time performance needed advanced HAR by modeling spatiotemporal features and
for rapid decision-making. These challenges hinder the handling interactions in crowded settings. However, the high
adaptation of automated systems to dynamic environments. computational demands of these models posed challenges
In addition, the lack of mechanisms to categorize and for real-time applications in large- scale surveillance
retrieve similar actions makes analyzing large volumes of environments.
footage cumbersome, resulting in delays in response times
and resource allocation.  Limitations and Gaps in Current Research
Despite these advancements, several persistent
 Objectives challenges remain. Attaining real-time performance without
The main goals of this study are to: compromis- ing accuracy is still a major issue due to the
computational complexity of deep learning models. Real-
 Design a hybrid CNN-LSTM model that combines time video surveil- lance requires low-latency solutions, a
CNN’s ability to extract spatial features with LSTM’s demand that many current systems fail to meet effectively.
strength in analyzing temporal sequences, enabling accu-
rate recognition of complex human actions. Environmental factors, including occlusion, lighting
 Enhance the system to ensure real-time performance, with varia- tions, and crowded environments, further complicate
low latency and quick responsiveness for timely decision- activity recognition, often leading to a decrease in accuracy.
making in surveillance contexts. Addi- tionally, while technical improvements in activity
 Implement an action categorization function to facilitate detection have been prioritized, many existing models
the efficient grouping and retrieval of similar activities, overlook user experience. Features for efficiently organizing
optimizing the process of reviewing extensive video data. and retrieving detected actions are often missing, making it
 Address practical challenges, such as lighting fluctua- difficult for users to process large volumes of video data.
tions, occlusions, and crowded settings, to improve the These gaps highlight the demand for systems that combine
robustness and adaptability of the system. technical robustness with user-centric features for better
 Assess the model’s effectiveness using metrics such as usability.
accuracy, recall, and latency to ensure it aligns with the
demands of contemporary surveillance applications.  Contributions of the Proposed System
The proposed system addresses these shortcomings by
II. RELATED WORK intro- ducing a hybrid CNN-LSTM model specifically
optimized for real-time human activity recognition. By
 Overview of Existing Detection Approaches combining CNNs for extracting spatial features and LSTMs
Human activity recognition (HAR) in video for temporal sequence analysis, the system ensures effective
surveillance has made significant strides, largely driven by recognition of complex actions in dynamic environments.
developments in deep learning techniques. Early systems
were based on rule-based or heuristic models with manually A key contribution of this system is its action
defined criteria for detecting actions in video streams. While categorization feature, which groups similar activities and
these initial methods performed adequately for simple tasks, simplifies the review process for users. This feature
they lacked the flexibility to handle complex or overlapping enhances the retrieval and analysis of specific events,
activities. improving the overall usability of the surveillance system.
Additionally, the system is de- signed to be resilient to
With the introduction of machine learning, HAR common environmental challenges, such as occlusion and
systems became more automated, allowing models to learn fluctuating lighting conditions, while maintaining low
patterns directly from data instead of relying on pre- latency to ensure real-time performance.
programmed rules. However, manual feature extraction
remained a bottleneck, introducing variability and limiting In conclusion, the proposed system presents a balanced
the overall performance of these models. The introduction of solution that integrates cutting-edge technical capabilities
Convolutional Neural Net- works (CNNs) marked a pivotal with practical user-friendly features, advancing the potential
shift in HAR by automating spatial feature extraction from of HAR applications in video surveillance.
video frames. CNNs proved highly accurate at recognizing
objects and basic actions, but struggled with the temporal III. PROPOSED SYSTEM
dependencies necessary for detecting sequential actions,
such as running or jumping. The proposed system enhances real-time human action
detection in video surveillance by integrating Convolutional
To address these limitations, Long Short-Term Neural Networks (CNNs) with Long Short-Term Memory
Memory (LSTM) networks, a subtype of Recurrent Neural (LSTM) networks. This hybrid approach leverages CNNs
Networks (RNNs), were integrated with CNNs. LSTMs are for spatial feature extraction and LSTMs for temporal
particu- larly adept at handling sequential data, allowing sequence analysis, enabling robust, context-aware activity
CNN-LSTM hybrid models to simultaneously capture both recognition. Additionally, an action categorization unit
spatial and temporal characteristics of activities. Other improves acces- sibility by organizing detected actions,

IJISRT25MAR088 www.ijisrt.com 169

Volume 10, Issue 3, March – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25mar088
facilitating efficient retrieval and review. Each component to detect relevant actions without omission. Latency
contributes meaningfully to achieving accurate, real-time measures the processing speed, which confirms suitability
surveillance. for real-time applications. Initial results indicate that the
integration of CNN and LSTM substantially enhances
At the core of this system is the challenge of accurately accuracy over traditional approaches, allowing efficient,
de- tecting human activities within complex environments. high-performance video frame analysis in real time.
CNNs are adept at spatial feature extraction, identifying
static ob- jects and visual cues essential for surveillance To support real-time processing, the system employs
tasks. By complementing this with LSTMs, which process opti- mizations such as GPU acceleration and model tuning,
time-series data, the system captures relationships across which reduce latency and improve processing speed. These
time, allowing it to understand how activities evolve over optimiza- tions enable the model to meet the high demands
multiple frames. Together, these models enable the of large-scale surveillance networks, where timely response
system to identify not only momentary actions but also is crucial.
more complex activities unfolding over time. The CNN
module initiates the process by analyzing each frame of Despite the effectiveness of CNN-LSTM integration,
input video data for spatial information. By utilizing chal- lenges persist, particularly with occlusion and crowded
convolutional layers, pooling, and activation functions, the scenes. Existing design strategies incorporate techniques like
CNN produces feature maps that emphasize crucial elements data aug- mentation and advanced noise reduction during
of each frame, including objects, shapes, and movement preprocessing. However, future improvements may include
indicators. These feature maps are subsequently input for the integration of multimodal data, such as audio or RFID
further temporal analysis by the LSTM module. This input, to enhance the system’s robustness.
architecture enables robust feature extraction and enhances
scalability, allowing the module to handle various input While primarily designed for security surveillance, the
resolutions and adapt to different video surveillance settings. system also holds promise for applications in other fields,
such as healthcare monitoring, sports analytics, and
Following spatial feature extraction, the LSTM module smart city infrastructure. This adaptability highlights the
per- forms temporal sequence analysis by processing these system’s potential for use across a variety of industries and
features sequentially. The LSTM’s architecture, including settings.
cell states and gates that control information flow, allows it
to retain and leverage past inputs over extended periods. In summary, the proposed system offers a holistic
This capability is crucial for distinguishing time-based solution for enhancing human activity recognition in video
activities, such as walk- ing, running, or more complex surveillance by combining CNN and LSTM
scenarios involving interactions between individuals. One of technologies. By combining spatial and temporal analysis
the key benefits is the ability to retain temporal context and with a user-friendly action categorization feature, the system
adapt to varying sequences, as LSTM networks can handle enhances both technical performance and usability. This
sequences of different lengths, making them ideal for a research provides a foundation for future advancements,
range of video scenarios. including multimodal integration and further optimization
for varied operational environments.
A notable innovation of the system is the action
catego- rization unit, which organizes detected activities into IV. METHODOLOGY
prede- fined categories. This functionality enhances user
accessibility, allowing users to search and review similar  System Architecture and Design
actions in one place. For example, security personnel can The proposed architecture for real-time human activity
filter all instances of ”suspicious activity” across multiple recognition (HAR) in video surveillance integrates convolu-
video feeds, expediting threat assessment and decision- tional neural networks (CNN) with long-short-term memory
making. The categorization feature significantly improves (LSTM) networks. This design effectively handles both
usability and supports rapid analysis. spatial and temporal data analysis. The process begins by
processing video feeds from live streams or pre-recorded
Data preprocessing plays an essential role in footage.Then videos are segmented into frames, which
optimizing system performance. Input video data undergoes undergo pre processing steps such as resizing to consistent
frame extrac- tion, resizing for data set consistency, and dimensions and normalization. Noise reduction methods are
pixel normalization. These preprocessing steps standardize used to improve the quality and uniformity of the input data.
the input data, facilitat- ing more effective model training The CNN module is responsible for extracting spatial
and enhancing performance. The system is trained on features from each frame. It consists of convolutional layers
labeled datasets, such as UCF50, within the CNN-LSTM with activation functions like ReLU, followed by pooling
pipeline. Hyperparameters, such as learning rate, batch layers to reduce dimensionality while preserving essential
size, and layer count, are adjusted to opti- mize features. The feature maps pro- duced by the CNN capture
performance. Evaluation metrics such as accuracy, recall, visual cues andobject relationships within each frame. These
and latency measure the effectiveness of the system. are subsequently transferred to the LSTM module, which
Accu- racy captures the frequency of correct activity processes the feature maps in sequence, transitioning from
identifications, while recall assesses the model’s ability analyzing individual frames to recognizing dynamic actions

IJISRT25MAR088 www.ijisrt.com 170

Volume 10, Issue 3, March – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25mar088
over time. previous frames and discards irrelevant information, making
it ideal for recognizing activities that progress over time,
The LSTM module is adept at temporal analysis, such as walking, running, or group interactions. Together,
utilizing its cell states and gating controls to manage the CNN and LSTM form a powerful pipeline that captures both
flow of infor- mation. It identifies important data from spatial and temporal contexts for accurate activity detection.

Fig 1 Flowchart of the System Workflow Illustrating the CNN-LSTM Pipeline.

 Training and Evaluation Moreover, techniques like batch normalization and dropout
The training of the CNN-LSTM model involves using are incorporated to prevent overfitting, ensuring the model
labeled datasets like UCF50 to fine-tune the architecture. generalizes well across diverse video datasets.
Essential hyperparameters such as learning rate, batch size,
and the number of layers are optimized to enhance model  Action Categorization and Usability
performance. The training process utilizes Backpropagation A key feature of the system is the action categorization
Through Time (BPTT) and optimization algorithms such as component, which groups detected activities into predefined
Adam or RMSProp to minimize loss and improve accuracy. categories. This enhances usability by enabling operators to
efficiently retrieve similar activities, such as ’suspicious
Model evaluation is based on metrics such as accuracy, behav- ior,’ across different video feeds. The categorization
re- call, and latency. Accuracy and recall are crucial for process optimizes the review workflow, allowing for quicker
assessing the reliability of activity detection, while latency decision- making and more efficient responses in high-
measures the system’s suitability for real-time use. stress situations.

IJISRT25MAR088 www.ijisrt.com 171

Volume 10, Issue 3, March – 2025 International Journal of Innovative Science and Research Technology
ISSN No:-2456-2165 https://doi.org/10.38124/ijisrt/25mar088
V. RESULTS AND DISCUSSION could focus on integrating multimodal data sources, such as
audio and RFID signals, to provide added context and
 Results increase robustness. Exploring advanced architectures like
Spatiotemporal Graph Convolutional Networks (STGCNs)
 Model Performance: The CNN-LSTM model demon- and implementing edge computing could further enhance the
strated an average accuracy of 92% for activity recog- system’s real-time perfor- mance and adaptability in
nition, with strong detection in sequences containing complex surveillance environments. These advancements
distinct motions. would lay the groundwork for building more comprehensive
 Efficiency Metrics: The latency measurements revealed and efficient automated surveillance sys- tems, with
that the system processes each video frame in approxi- potential applications that extend to fields such as
mately Y milliseconds, demonstrating its suitability for healthcare, sports analytics, and smart city infrastructure.
real-time surveillance applications.
 Comparative Analysis: The proposed system outper- REFERENCES
formed traditional standalone CNNs by achieving higher
accuracy in complex activity recognition scenarios. [1]. H. Park, Y. Chung and J. -H. Kim, ”Deep Neural
 Error Analysis: Misclassification rates were observed in Networks-based Classi- fication Methodologies of
actions with subtle or overlapping gestures, indicating the Speech, Audio and Music, and its Integration for
need for enhanced feature extraction. Audio Metadata Tagging,” in Journal of Web
Engineering, vol. 22, no. 1, pp. 1-26, January 2023,
 Discussion doi: 10.13052/jwe1540-9589.2211.
[2]. W. Huang, Y. Liu, S. Zhu, S. Wang and Y. Zhang,
 Comparison with Related Work: This approach im- ”TSCNN: A 3D Convolutional Activity Recognition
proved upon existing models by integrating a more ro- Network Based on RFID RSSI,” 2020 International
bust sequence analysis, resulting in higher accuracy than Joint Conference on Neural Networks (IJCNN),
reported in similar studies. Glasgow, UK, 2020, pp. 1-8, doi:
 Challenges Faced: The main challenge was the process- 10.1109/IJCNN48605.2020.9207590.
ing of activities involving minimal motion or occlusions, [3]. A. M. F and S. Singh, ”Computer Vision-based
where the model’s performance dropped slightly. Survey on Human Activity Recognition System,
Challenges and Applications,” 2021 3rd
 Future Improvements: Integrating advanced noise re-
International Conference on Signal Processing and
duction techniques or hybrid architectures, like ConvL-
STM, could address the identified issues and improve Communication (ICPSC), Coimbatore, India, 2021,
pp. 110-114, doi: 10.1109/IC-
system robustness.
SPC51351.2021.9451736.
 Potential Applications: The system’s reliable real-time
[4]. S. Aarthi and S. Juliet, ”A Comprehensive Study on
processing makes it applicable not only in surveillance
Human Activity Recognition,” 2021 3rd
but also in areas like sports analytics, automated video
International Conference on Signal Processing and
editing, and behavior analysis.
Communication (ICPSC), Coimbatore, India, 2021,
pp. 59-63, doi: 10.1109/ICSPC51351.2021.9451759.
VI. CONCLUSION
[5]. C. Zhao, L. Wang, F. Xiong, S. Chen, J. Su and H. Xu,
”RFID-Based Hu- man Action Recognition Through
 Summary of Findings
Spatiotemporal Graph Convolutional Neural
The combination of Convolutional Neural Networks
Network,” in IEEE Internet of Things Journal, vol.
(CNNs) and Long Short-Term Memory (LSTM) networks
10, no. 22, pp. 19898-19912, 15 Nov.15, 2023, doi:
has proven to be a highly effective approach for improving
10.1109/JIOT.2023.3282680.
human activity recognition in video surveillance systems.
This integrated model utilizes CNNs for the extraction of
detailed spatial fea- tures and LSTMs for processing
temporal sequences, leading to significant improvements in
both detection accuracy and real-time performance. The
evaluation results highlight that the hybrid model
outperforms traditional methods in terms of both precision
and response time, making it well-suited for large-scale
applications such as public safety monitoring and industrial
surveillance. Furthermore, the inclusion of an action
categorization feature enhances system usability,
streamlining the review process and boosting operational
efficiency.

 Future Work
While the current model shows strong potential,
challenges remain, particularly in handling occlusions and
variable en- vironmental conditions. Future developments

IJISRT25MAR088 www.ijisrt.com 172