Fall Detection for Elderly People in Homes using

Kinect Sensor
Disha M. Pathak Dr. R.M. Autee V.K. Bhosale
DIEMS Aurangabad, India DIEMS Aurangabad, India DIEMS Aurangabad, India
[email protected] [email protected] [email protected]

Abstract— All over the globe, the population of elderly people major attention in research. Many approaches and algorithms
has been steadily increasing. Fall events are a foremost reason of have been developed by researchers all over the world for
fatal injury for them. The immediate detection of fall events can reporting or detecting falls among elderly people. These
decrease the death risk. Thus it is very important to detect fall consist of wearable devices that allow an individual to
events as soon as possible, so that immediate assistance may be
manually press a button in the event of fall, as well as
provided. Computer vision-based systems offer low-cost and
satisfactory results for fall detection. In this paper, a method for wearable devices such as wristwatches and necklaces that
detecting falls for elderly people in homes is proposed, by automatically detect fall events with the help of sensors such
analyzing the tracked key joints of the human body using the as accelerometers. However, studies have indicated elderly
Microsoft Kinect. In contrast to other fall detection methods that people prefer nonwearable sensors. A number of researchers
rely on the RGB inputs, the proposed system is independent of have also looked at the use of environmentally mounted
illumination of the lighting conditions and can work even in a sensors for fall detection, such as floor vibration sensors,
complete darkroom. Using depth images obtained by Kinect passive infrared sensors, acoustic sensors, and video-based
sensor could bring innovative ways to build smart fall detection sensors. Considering all the above aspects proposed system is
systems that could use to observe the elderly people at homes,
designed that uses vision-based approach and which utilizes
and notify the caretakers by raising an alarm in case of falling
events. In this paper, an algorithmic method of fall detection the skeleton tracking of a human being. This paper presents a
system is proposed, that runs in a real time scenario based on method for detecting fall events using a depth imaging sensor
skeleton extraction of a human figure with the help of Kinect i.e Microsoft Kinect. As soon as a fall is detected a SMS
depth sensor. Once a fall event is encountered an alert is sent to message is sent on a predefined person’s number using GSM
the particular elderly person’s caretaker using SIM900A GSM modem. Other benefits of the proposed method, with respect
modem. Based on a data set of 50 people, experimental results to traditional video-based ones are as follows:
validates that the proposed method accomplish near 94.65%
frame accuracy in a real home environment.  Does not depends on lighting conditions
 Need of a single camera to obtain 3D information
Keywords—Fall detection, Microsoft Kinect, Depth image,  Possible to hide the face of the elderly people
Skeleton extraction, GSM SIM900A captured by the depth camera.

I. INTRODUCTION II. RELATED WORK

Fall of frail people is both a medical and a social issue. Various methods have been proposed for detecting fall
Falls and their consequences, such as bone fractures, tissue events and are mainly divided into two categories: Non -
injuries, lacerations, joint dislocations, bone fractures and computer vision based and computer vision-based methods.
head trauma are a major risk for elderly people especially who Non-computer vision based methods
live alone where instant help is needed. According to [1], falls There are a variety of non-computer vision-based methods
are the sixth most common reason of fatality for people above for fall detection. In these methods, various sensors (acoustic,
the age of 65. Moreover, it is the second most common cause acceleration and floor vibration sensors) are used to acquire
of fatality for people between the age of 65 and 75, and the the sound, vibration, and human body movement data and the
most common cause for people over 75. As per World Health collected information is then analyzed to determine a fall. Non
Organization [2] approximately 28-35% of people aged 65 and -computer vision based methods can be performed either of
over fall each year increasing to 32-42% for those over 70 the two ways :
years of age. The occurrence of fall increases with age and
frailty level and often results in severe physical and  Wearable sensor methods
psychosomatic consequences. Immediate fall detection can  Ambient/Fusion based method
thus decrease the mortality risk and raises the possibility to Various fall detection techniques use accelerometers,
survive the incident and return back to independent living [3]. gyroscopes, or both. However, these techniques have a major
Therefore, automatically detecting falls at home has become a
disadvantage that the person has to wear these devices camera, digital compass and gyroscope. However, major
throughout the day. Also, the usage of button is ineffective in drawback of this approach is the low battery life of smart
case of devices requiring actions on the part of wearer due to phones and handling smartphones precisely.
loss of consciousness of the wearer following a fall event[4-7].
Computer vision based methods
These systems have a foremost benefit over non-vision
Fall Detection system. In these methods, it is not mandatory to wear any
devices. Here one or more cameras is used to track people.
Wearable sensor Ambient/Fusion They can be performed either of the three ways :
 Methods using a single RGB camera
Vision
Accelerometer  3D- based methods using multiple cameras
 3D- based methods that use depth cameras
Various algorithms for fall detection have been developed
Audio & so far. Kępski [12], detected fall events by tracking of
Inactivity Posture Vibration Video person’s head in depth maps. However, this method is not
robust since there is a large contrast between the fall like event
and an actual fall. Using 3D data a fuzzy logic based fall
Body Posture/ 3D Head Spatio detection algorithm is proposed in [13]. An another approach
Shape Inactivity change temporal in detecting the fall events is proposed by Feng [14], which
gives detection by extracting foreground human silhouette via
Fig. 1. Classification of Fall detection methods background modeling. Sumiya [15] used well defined skeleton
tracking feature to track the person. In this work, the Kinect
Using MEMS accelerometers a fall detection method is device is used along with a mobile robot to accomplish fall
presented in [8]. Acoustic sensor based systems make use of detection. An improvement of 80% in fall detection rate in
sound volume from the microphone to detect fall events. Khan comparison to conventional monitoring technique using
[9] proposed an unsupervised fall detection system that uses
position fixed sensors is obtained. Stone [16] proposed a two-
acoustic signals (footstep sound signals) and then used them to
stage approach using Microsoft Kinect sensor for detecting
differentiate falls from non-falls events. An additional
approach is the floor vibration detectors that are based on floor fall events. Here depth map of the person is used for
dynamics. In 2014, Yahar [10] presented a method for segmenting the spectral intensities of varied body depths. In
detecting falls using vibration and passive infrared (PIR) [17], depth map estimation of 3D bounding boxes are formed
sensors. Smartphone based fall detection system are another for the relevant targets and fall detection is done. Bevilacqua
unobtrusive method widely used these days. Suryadevara [11] [18] proposed a system based on assessing the contraction and
proposed an android based fall detection system. These the expansion speed of the width, height and depth of the 3D
smartphones has an inbuilt GPS, accelerometers , microphone human bounding box. In [19] another Kinect based method
using the raspberry pi and Open Cv is used.

TABLE I : COMPARISON OF FEW DIFFERENT APPROACHES

Reference Detection technique Method Type of sensor Remarks

[8] Signals are acquired from the extremities. Wavelet Wearable Accelerometers Provide discomforts and false
transform and thresholding technique is used device based alarms

[10] Vibration waveforms are extracted to get feature vector with Ambience Vibration and Passive Performance gets poor during noisy
the help of complex wavelet transform (CWT) and then sensor based infrared (PIR) sensors and reverberant environments
classified using support vector machine (SVM)

[11] Application vector magnitude of 3-axis accelerometer, Smartphone Smartphone built-in Need to carry the Smartphone.
absolute vector magnitude and the orientation of the sensor based accelerometers Low battery life
are used
[15] Kinect sensor is used with mobile robot system to follow a Vision based Microsoft Kinect Need for the mobile robot system
person and detect fall events

[16] 1st Stage - Vertical state characterization Vision based Microsoft Kinect Decreased resolution of faller
2nd Stage - On ground event features, ensemble for fall makes foreground segmentation
confidence difficult

[19] Canny Algorithm, Contour Approximation Method Vision based Microsoft Kinect The Raspberry pi and Open CV 1.0
are used
 Besides providing depth images, an additional feature of
III. SYSTEM DEVELOPMENT the Kinect sensor is a skeletal tracker. The Kinect for
In the proposed method Kinect studio version 1.8.0 is Windows SDK provides a skeleton-tracking feature which
installed to grab and access the depth data from Kinect sensor. allows developers to recognize people and track their actions.
Entire image processing task is accomplished on PC. For this With the help of depth sensors, Kinect can recognize up to six
Matlab Software is adopted. Here MATLAB R2013 version is users standing from 0.8 to 4.0 meters (i.e 2.6 to 13.1 feet).
used. With the aid of image processing toolbox, detection of Furthermore, two of the detected skeleton can also be tracked
fall events is made. Fig.2 shows the proposed fall detection in aspect with twenty joint positions. Each skeleton joint is
system. calculated in a three dimensional (X, Y, Z) plane.

Kinect Serial to GSM B. SIM 900A GSM modem

Sensor USB module SIM900A is an ultra compact and reliable wireless
converter module. It delivers GSM/GPRS 900/1800MHz performance
Computer for voice, SMS, Data, and Fax. SIM900A GSM modem is
system regulated by the AT (ATtention) command and is set to the
with SMS text mode to send an SMS message. Few of SIM900A
Matlab features are as follows :
tool Alert
Live  Dual band GSM/GPRS 900/1800MHz.
video
Finally, message
 Configurable baud rate
stream
 SIM card holder.
Fig. 2. Proposed Fall Detection system  Low power consumption: 1.5mA (sleep mode)

A. Microsoft Kinect
C. Proposed Algorithm
Microsoft Kinect is essentially a set of sensors that was
The proposed algorithm is based on depth map and makes
formerly developed as an input tool for the Xbox 360 and
use of a Microsoft Kinect device. Here system will detect the
Xbox One gaming console. It comprises of the infrared
skeletal data to recognize elderly people and track their
projector, IR camera which is utilized for getting precise depth
actions. In this work, it is assumed that the elderly person is
maps, a RGB color camera , a four-microphone array and a
completely perceptible and do not have any occlusions. The
motorized tilt. Image streams at 30 frames per second (fps) are
Kinect sensor is steady and the movement of the subject along
produced in both cameras. The IR camera can capture 3D
the sensor is restrained to a range. An overview of the
video data under any ambient light conditions with the help of
proposed fall detection method is as shown.
structured light technology. The depth values depict the
distance not to the focal point but to the imaginary image The logical steps on which the proposed system is built are
plane. The IR projector shoots an irregular pattern of dots with as follows :
a wavelength comprising of 700nm to 1mm.Thus this pattern
of dots is invisible to humans. Further the infrared camera
captures this infrared light. This camera capture the image
from its viewpoint and it all comprise of infrared data.

Initialize Kinect Grab frame Find Human Find number of

Depth Sensor and metadata Skeleton skeleton tracked

Find joint Track all joint points Find the index

indices matrix from skeleton value of skeleton

Track Hip and Head Compare Y-axis Display person position

joint and access joint Pixel values (Standing/Fall)
indices matrix with threshold
Send Alert
Message

Fig. 3. Data Flow of proposed fall detection system

1. Initialize Depth Sensor The distance of the Kinect sensor has been varied from
In the proposed method, this block helps to acquire the live 0.8m to 4.5m, which is technically its sensing range. The
depth image from Kinect sensor using Image Acquisition results are summarized in Figure 4. Here the y-axis represents
Toolbox. the fall detection rate (%), whereas the x-axis denotes the
distance between the sensor and subject in meters.
2. Grab Frames and Metadata
Following initialization of the depth sensor, the next step is
to trigger the depth sensor to start grabbing the frames and the
related metadata.

3. Human Skeletons
Extricate skeleton diagram of the human body which is in
the vicinity of the Kinect sensor irrespective to other objects in
the frame. Thereafter the number of skeletons tracked and its
index value is calculated.

4. Joint Indices Matrix

For each tracked skeleton 20 joint points are obtained. This Fig. 4. Fall detection rate (Different distance)
is a 20 x 2 double matrix of x-and y-coordinates for 20 joints
in pixels relative to the depth image.

5. Access Hip and Head Joint Points

After obtaining the joint indices matrix (20X2), the indices
values related to the hip joint and head joint is found and
accessed. Store the y- axis pixel values of the hip and head
joint in predefined variables.

6. Compare Pixel Values with Threshold

Compare the y-axis pixel values of the hip and head joint
with a threshold and compute the person’s position (Standing/
Fall).

7. Alert SMS Message

As soon as a fall is detected, the system starts its timer and Fig. 5. Standing Person in color image
waits for 10sec to send an alert SMS message to a predefined
mobile number. If the person is still in fallen condition after
the given delay of 10sec then the system sends the text
message. Otherwise, no alert message is sent.
IV. RESULTS
The proposed method is implemented in MATLAB R2013
using the Open Kinect libraries and has been tested at different
time and lighting conditions on a 50 person dataset of different
height, weight and age.

TABLE II : PERCENTAGE SUCCESS OF DIFFERENT POSITIONS

Category No. of Fall Fall rejection

activities acceptance rate (FRR)
rate (FAR) Fig. 6. Fallen Person in color image
Falls 150 95.33% 4.66% Above figures (5-6) shows the color image of captured data
Walking 80 100% 0% with Skeleton tracking. The live image is captured and featured
around by the fall detection algorithm, the y-axis pixel values of the
Lying on 60 90% 10% hip and the head joint are extracted and compared with their
sofa respective thresholds. The image is taken for testing and the
Sitting on 60 93.33% 6.67 following result is displayed.
chair/sofa
 [2] Igual, Raul, Carlos Medrano, and Inmaculada Plaza. "Challenges, issues
and trends in fall detection systems." Biomedical engineering
online 12.1 (2013): 1.
[3] Yang, Lei, Yanyun Ren, and Wenqiang Zhang. "3D depth image
analysis for indoor fall detection of elderly people." Digital
Communications and Networks2.1 (2016): 24-34.
[4] Jagtap, Arti, et al. "Fall Detection and Alerting System using Kinect
Sensor and GSM." International Journal of Engineering Science 7069
(2016).
[5] Garripoli, Carmine, et al. "Embedded DSP-based telehealth radar system
for remote in-door fall detection." IEEE journal of biomedical and
health informatics 19.1 (2015): 92-101.
[6] Delahoz, Yueng Santiago, and Miguel Angel Labrador. "Survey on fall
detection and fall prevention using wearable and external
sensors." Sensors14.10 (2014): 19806-19842.
[7] Mubashir, Muhammad, Ling Shao, and Luke Seed. "A survey on fall
detection: Principles and approaches." Neurocomputing 100 (2013):
144-152.
[8] Prince PGK, Hemamalini R, Rajkumar RI. LabVIEW based abnormal
muscular movement and fall detection using MEMS Accelerometer
Fig. 7. Alert SMS message during the occurrence of seizure. Indian Journal of Science and
Technology. 2014; 7(10):1625–31.
V. CONCLUSION AND FUTURE WORK [9] Khan, Muhammad Salman, et al. "An unsupervised acoustic fall
detection system using source separation for sound interference
In this paper, a fall detection method for elderly people suppression." Signal processing 110 (2015): 199-210.
based on the skeleton extraction using Kinect sensor is [10] Yazar, Ahmet, Fatih Erden, and A. Enis Cetin. "Multi-sensor ambient
presented. The system is highly reliable and vigorous since it assisted living system for fall detection." Proceedings of the IEEE
works on the real-time data instead of a database. The International Conference on Acoustics, Speech, and Signal Processing
proposed fall detection approach uses the infrared based depth (ICASSP’14). 2014.
camera, so the system can operate even in the dark condition. [11] Suryadevara, Vinay Kumar, and Maher Rizkalla. "Smartphone Based
The system can easily be applied in a smart home environment Fall Detection and Logic Testing Application Using Android
SDK." Journal of Biomedical Science and Engineering 8.09 (2015):
and supports independent living of elderly people. The 616.
proposed fall detection system can be used in homes, hospitals
[12] Kępski, Michał, and Bogdan Kwolek. "Person detection and head
and recreation centres for the elderly. A significant innovation tracking to detect falls in depth maps." International Conference on
of the proposed method is its independence from the direction Computer Vision and Graphics. Springer International Publishing,
of the fall, in contrast to other approaches in which fall is [13] Planinc, Rainer, and Martin Kampel. "Robust fall detection by
usually identified only as a rapid vertical motion. combining 3D data and fuzzy logic." Asian Conference on Computer
Vision. Springer Berlin Heidelberg, 2012.
The experimental results based on 50 people demonstrates [14] Feng, Weiguo, Rui Liu, and Ming Zhu. "Fall detection for elderly person
that proposed fall detection system attains a high fall detection care in a vision-based home surveillance environment using a monocular
rate near 94.65%. The use of skeleton tracking for fall camera." Signal, Image and Video Processing 8.6 (2014): 1129-1138.
detection was quite satisfactory, but future work requires an [15] Sumiya, Takuma, et al. "A mobile robot for fall detection for elderly-
additional upgrading in the fall detection methodology. Also, care." Procedia Computer Science 60 (2015): 870-880.
future work includes to enlarge the coverage area, advance the [16] Stone, Erik E., and Marjorie Skubic. "Fall detection in homes of older
minor movement tracking of elderly people and update the adults using the Microsoft Kinect." IEEE journal of biomedical and
health informatics 19.1 (2015): 290-301.
status to the caretakers using cloud services, and to develop an
IOT application of the same. Using GPS the latitude and [17] Mastorakis, Georgios, and Dimitrios Makris. "Fall detection system
using Kinect’s infrared sensor." Journal of Real-Time Image
longitude of the fallen person can be extracted, this Processing 9.4 (2014): 635-646
information then can be directly sent to the nearby hospitals. [18] Bevilacqua, Vitoantonio, et al. "Fall detection in indoor environment
with kinect sensor." Innovations in Intelligent Systems and Applications
(INISTA) Proceedings, 2014 IEEE International Symposium on. IEEE,
REFERENCES 2014.
[1] Mundher, Zaid A., and Jiaofei Zhong. "A real-time fall detection system [19] Madhubala, J. Sree, and A. Umamakeswari. "A vision based fall
in elderly care using mobile robot and kinect sensor." International detection system for elderly people." Indian Journal of Science and
Journal of Materials, Mechanics and Manufacturing 2.2 (2014): 133- Technology 8 (2015): 167.
138.