IEEE International Conference On Recent Trends In Electronics Information Communication Technology, May 20-21, 2016, India

Detection and Identification of

Unattended/Removed Objects in Video
Surveillance
Lakhan H. Jadhav, Bashirahamad F. Momin,
Abstract—This paper presents an unattended or removed Object application automatic detection of abandoned object requires
detection method for video surveillance data captured by single high accuracy and computation complexity should be low
static camera. Two backgrounds are modelled with different enough which will assure real time performance.
learning rates; one for transient and another for permanent.
These both backgrounds are defined as a mixture of Gaussian
model, it uses online Bayesian for updating. Two binary
foregrounds are extracted by subtracting those backgrounds from
current frame. Then shadow removing algorithms are applied to
extract the real shape of the foreground objects. Then blob level
likelihood image model is used to detect temporary static objects.
Then we classify the extracted object using features like size,
height, width, colour of that object. Input to this is video and the
output will be detection and classification of abandoned objects.
Finally, we trigger an alarm on detection of abandoned object.
We provide good results and efficient model which can be used in
real time surveillance systems.

Keywords—video surveillance, background modeling, Fig.1: Surveillance system for unattended object detection.
abandoned object.
Consumer surveillance system takes video as an input
I. INTRODUCTION captured by surveillance cameras. It processes those videos
Consumer surveillance cameras are available in low cost and provides the required results. Fig:1 shows this structure, it
and everywhere. Advancement in smart surveillance cameras takes video (I. e. The sequence of frames) as an input from
and high processing capabilities of computer has made it camera, process that video and finally gives the output.
possible to design video surveillance system which can
contribute for people safety in public places. Now a days there A. System requirements of unattended/removed object
are various cases of explosive attacks with luggage left detection
unattended or suicide bombing in case of still person. In consumer surveillance system, detection of left or
Automatically detection of left baggage unattended or forgotten baggage and detection of removed or stolen objects
removed objects is important in criminal investigation. is vital feature of any surveillance system. For this some
Unattended object detection in consumer video surveillance surveillance system requirements are as follows:
becomes more difficult due to high volume of data, 1) Detection of static and moving objects should have
illumination, different weather conditions, occlusion and sufficient accuracy for end user acceptance.
moving backgrounds. In these situations a traditional approach 2) Low computational cost that will gives the real time
like object tracking fails. This paper proposes a method for results.
detection of unattended/removed objects which works well 3) The conversion of visual data to semantic level data
under these conditions and helps to develop end-to-end system like still person existence, object stolen, baggage left.
for automatic video surveillance. We detect temporarily static The rest of the paper is organized as follows: section II
objects by maintaining multiple backgrounds with different covers the related work. Methodology is described in section
learning rate. Attributes selected for classification and III. Section IV shows the Experimental results. Finally,
identification of static object are height, width, size, colour, concluding remarks are presented in section V.
time. The consumer requirements from unattended object II. RELATED WORK
analyzers are: identify suspicious, unattended objects within
time of its abandonment and allow fast recognition of In literature, many approaches of object detection using
existence of threat. As a surveillance background subtraction were proposed [1-4]. These methods
mainly differ in the type of background model and procedure
Lakhan H .Jadhav, Department of Computer Science and Engineering, used to update background. Gaussian distributions have been
Walchand College of Engineering, Sangli, Maharashtra, India utilized for updating the pixel intensities [4]. [4] shows good
([email protected]) results in illumination and different lightening conditions. For
Bashirahamad F.Momin, Department of Computer Science and Engineering, motion based background subtraction and the detection of
Walchand College of Engineering, Sangli, Maharashtra, India moving objects in complex backgrounds Adaptive Kernel
([email protected]) Density estimation is used in [1]. [2] presents Background
Modelling Approach for Moving Object Detection in

978-1-5090-0774-5/16/$31.00 © 2016 IEEE

1770
Authorized licensed use limited to: ULAKBIM UASL - Uluslararasi Kibris Universitesi. Downloaded on March 22,2023 at 04:55:29 UTC from IEEE Xplore. Restrictions apply.
 IEEE International Conference On Recent Trends In Electronics Information Communication Technology, May 20-21, 2016, India

illumination sensitive environment, to sudden illumination III. METHODOLOGY

changes. A good review of Background subtraction techniques
This section presents how we are going to implement this
is given in [2], it introduces various background subtraction
system. The block diagram of the proposed system is shown in
techniques and discuss the pros and cons of each.
fig: 4. It consist of three main components we will see each in
Some papers are in literature which focuses on foreground
brief.
analysis [5,6]. In [7] author uses spatial, temporal and spectral
A. Foreground Blob Extraction
features to discriminate the background presence at each pixel.
The timely foreground blob extraction is done for that we
A model-based framework to segment static foreground
model two timely updating backgrounds; one for Short Length
objects against a moving foreground objects method used in
and another for Long Length by simply varying learning rates
[6]. Block matching has been used to calculate motion vector
of background updating. The main reason behind this is to
in [8].
differentiate between static regions that was changed and
original background regions. After background modelling,
background subtraction is performed to extract foreground
objects. To get accurate foreground object shadow removal is
applied, shadow removing technique eliminates the shadow
pixels that are detected as foreground at time of background
subtraction. Shadows changes the original shape of the object
which will affect feature values so it need to be eliminated.
The intensity and texture information is used to identify
shadow pixels.
After background subtraction and shadow removing, the
resulting detected moving objects may have multiple regions
Fig.2: Long-term and short-term backgrounds are learned by processing video that correspond to a single object, this is called as split
at different frame rates. Ref [11]
detections. Also, it may have multiple objects, which is called
as combined detection. These both events split and combined
Stationary or static object detection using two
detection, make the object detection and identification critical.
backgrounds is discussed in [9][10][11]. In this two
To overcome this system should ensure one-to-one
backgrounds are maintained for short and long length. Long-
correspondence between detected regions and the actual object
term and short-term backgrounds by processing video at
present.
different frame rates has been given in [11], It also presents
Then features of each blobs are extracted. These features
Hypotheses on the long-term and short-term foregrounds. This
to be used in object identification. Features extracted are:
hypothesis differentiate the moving, stationary , removed
1) Location: It is the coordinates of centre of the blob.
objects and scene background.
2) Height: Height of blob in pixel.
Abandoned/unattended object detection is discussed in
3) Width: Width of blob in pixel.
[9][10][12]. [9] propose an intelligent vision based analyzer
for semantic analysis of objects left unattended but it fails to
B. Object Classification
handle the relation of unattended objects with human. A hybrid
Extracted object needs to be classified as moving,
version of two level, timeliness background and edge features
stationery and removed objects. Following algorithm describes
used in [10] for object detection and classification. [12] uses
this classification process:
boundary matching to classify objects as abandoned or
removed. In many surveillance systems, the initial background ALGORITHM 1: Object Classification
contains objects that are later removed from the frame or left Input: Two arrays of m and n elements respectively; Element
into the frame. Classifying whether a foreground blob contains Location, height and width of object
corresponds to unattended or removed objects or still person is
an essential problem with background model, but current 1: //a1[] : Array having m elements
systems ignore it. 2: //a2[] : Array having n elements
3:for i = 1 to m do
4: for j = 1 to n do
5: if (a1[i].location ≈ a2[j].location and a1[i].height ≈
a2[j].height and a1[i].width ≈ a2[j].width )
then
6: a1[i] is moving object;
7: Discard a2[j] element from array a2;
8: Continue i;
9: end if
10: end for
11: //a1[i] doesn’t match with any element in a2[].
12: a1[i] is removed object;
Fig. 3:Hypotheses on the long- and short-term foregrounds. Ref [11]
13: Pass a1[i] to rule based classifier;
14: end for
15: for each a2[k] do

1771
Authorized licensed use limited to: ULAKBIM UASL - Uluslararasi Kibris Universitesi. Downloaded on March 22,2023 at 04:55:29 UTC from IEEE Xplore. Restrictions apply.
 IEEE International Conference On Recent Trends In Electronics Information Communication Technology, May 20-21, 2016, India

16: //a2[k] is stationary object. TABLE II: PERFORMANCE OF PROPOSED SYSTEM

17: Pass a2[i] to rule based classifier;
18: end f Detectio Classification Identification False
Data Set
n Rate Rate Rate Alarms
S1-T1-C 83% 89% 82% 8%
S2-T3-C 87% 90% 86% 10%
S3-T7-A 85% 87% 79% 17%
S4-T5-A 84% 93% 86% 13%
S5-T1-G 83% 92% 84% 9%
S6-T3-H 84% 88% 81% 9%
S7-T6-B 86% 91% 83% 12%

IV. EXPERIMENTAL RESULTS

Set of experiments carried out on surveillance data

obtained from college campus using CA-IP600P camera and
standard dataset PETS2006 to evaluate the performance of
proposed system.

Fig. 4:Block diagram of proposed system

Objects are classified on the basis of its presence in Short Fig.5: Long-term and short-term backgrounds , and identification of still
Length and Long Length foreground. Below table clears this person
concept very well.
TABLE I: OBJECT CLASSIFICATION USING TWO FOREGROUNDS
Long Length Short Length
Object Type
Foreground Foreground
1 0 Stationary Object
0 1 Removed Object
1 1 Moving Object

C. Object Identification and Responsive Actions

After classification of objects as stationary or removed,

they are identified and proper action is taken. To identify Fig.6: Long-term and short-term backgrounds and abandoned object
object rule based classifier is used. If static object identified as
a person then it’s considered as still person; but abandoned Modelling of two backgrounds, learning rates used are
still person cannot be recognized automatically as he may 0.005 and 0.0001 for short length and long length backgrounds
waiting for someone so system leave that decision manually. If respectively. Proposed system gives 84.57 percent accurate
the object identified as baggage that is unattended baggage so detection of objects, error occurs in crowded scenes and
system triggers alarm. occluded regions. Classification of detected objects is
If object is removed one and identified as a person then it performed 90 percent accurately using above algorithm.
is the event of still person leaving the place so system discards In the experiment, for rule based classifier the training
it and is identified as baggage then it's the case of stolen samples were randomly selected, and the remaining was for
objects, the system triggers alarm testing. It gives 83 percent average accuracy. Average
detection of false alarms is 11.14 percent, false alarm occurs
due to inability of recognize human behaviour. It can be
improved by using human behaviour analysis; we put it for
future work.
Following graph shows the performance of this system.

1772
Authorized licensed use limited to: ULAKBIM UASL - Uluslararasi Kibris Universitesi. Downloaded on March 22,2023 at 04:55:29 UTC from IEEE Xplore. Restrictions apply.
 IEEE International Conference On Recent Trends In Electronics Information Communication Technology, May 20-21, 2016, India

[7] Liyuan Li, Weimin Huang, Irene Yu-Hua Gu and Qi Tian,

“Statistical Modelling of Complex Backgrounds for Foreground
Object Detection”, Vol-13. IEEE Trans on Image Proc., pp 1459-
1472, Nov2004.

[8] CHI Jian-nan ZHANG Chuang ZHANG Han LIU Yang YAN
Yantao, “Approach of Moving Objects Detection in Active Video
surveillance”, DOI: 10.1109 CDC.2009.5399604 IEEE Conference
Publications, pp 3130-3136, 2009.

[9] Thi Thi Zin , Pyke Tin, Hiromitsu Hama and Takashi Toriu,
“Unattended Object Intelligent Analyzer for Consumer Video
Surveillance”, Vol-57. IEEE Trans on Consumer electronics, pp 549-
557, 2011.

[10] Tai Yu Lai1, Jong Yih Kuo, Chien-Hung Liu, Yue Wei Wu1, Yong
Yi Fanjiang, Shang-Pin Ma, “Intelligent Detection of Missing and
Fig.7: Performance of proposed system Unattended Objects in Complex Scene of Surveillance Videos”,
DOI: 10.1109 IS3C.2012.172 IEEE Conference Publications, pp
662-665, 2012.
V. CONCLUSION
[11] Fatih Porikli, “Detection of Temporarily Static Regions by
This paper presents a method for detection and Processing Video at Different Frame Rates”, IEEE Conference
Publications, pp 236-241, 2007.
identification of unattended/removed objects in the consumer
video surveillance system. For stationary object detection two [12] P. Spagnolo; A. Caroppo; M. Leo; T. Martiriggiano; T. D’Orazio,
foregrounds extracted using a dual background model. A “An Abandoned/Removed Objects Detection Algorithm and Its
hybrid model is used to detect stationary objects. Once Evaluation on PETS Datasets”, IEEE Conference Publications,DOI:
10.1109 AVSS.2006.18, 2006.
stationary object is detected it’s attributes are extracted in
order to classify the object. Attributes extracted are height,
width, size, colour and time. It is reliable and efficient in terms
of computational complexity. For more accuracy more features
like human baggage relation, human postures can be used.

ACKNOWLDGEMENT

I would like to express my sincere thanks to all who

helped me directly or indirectly in this esteemed work. This is
part of my Master’s Dissertation work and sponsored under
Research Promotion Scheme (RPS), AICTE, New Delhi, India
(2013-2016).

REFERENCES

[1] Anurag Mittal, Nikos Paragios, “Motion- based background

subtraction using adaptive kernel density estimation”, Vol-2. IEEE
Conference Publications, DOI: 10.1109 R.2004.1315179,
2004
.
[2] Massimo Piccardi, “Background subtraction techniques: a review”,
Vol- 4. IEEE Conference Publications, pp 3099-3104, 2004.

[3] Fan-Chieh Cheng, Shih-Chia Huang, and Shanq Jang Ruan,

“Illumination-Sensitive Background Modelling Approach forAccurate
Moving Object Detection", Vol-57. IEEE Journals and Magazines, pp
794-801, 2011.

[4] Lionel Carminati, Jenny Benois-Pineau, “Gaussian Mizture

Classification for moving object Detection in video surveillance
Environment”, Vol-3. IEEE Conference Publications, pp 113-
116, 2005.

[5] Rita Cucchiara, Costantino Grana, Massimo Piccardi, and Andrea

Prati, “Detecting Moving Objects, Ghosts, and Shadows in Video
Streams”, Vol-25. IEEE Trans on Pattern anlys and machine Intl., pp
1337-1342, Oct2003.

[6] Lucia Maddalena and Alfredo Petrosino,“Stopped Object Detection by

Learning Foreground Model in Videos”, Vol-24. IEEE Journals and
Magazines, pp 723-735, 2013.

1773
Authorized licensed use limited to: ULAKBIM UASL - Uluslararasi Kibris Universitesi. Downloaded on March 22,2023 at 04:55:29 UTC from IEEE Xplore. Restrictions apply.