See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/344409407

Human Activity Recognition Using Smartphone and Smartwatch

Article in International Journal of Computer Engineering in Research Trends · October 2016

DOI: 10.22362/ijcert/2016/v3/i10/48906

CITATIONS READS
9 1,863

3 authors, including:

Ali M. Muslim Hamid M. Ali Alshaibani

Universiti Putra Malaysia University of Baghdad
6 PUBLICATIONS 17 CITATIONS 14 PUBLICATIONS 123 CITATIONS

SEE PROFILE SEE PROFILE

Some of the authors of this publication are also working on these related projects:

Smartphone Applications View project

Human emotion recognition using smartphone and smartwatch View project

All content following this page was uploaded by Hamid M. Ali Alshaibani on 28 September 2020.

The user has requested enhancement of the downloaded file.

 Volume 3, Issue 10, October-2016, pp. 568-576 ISSN (O): 2349-7084

International Journal of Computer Engineering In Research Trends

Available online at: www.ijcert.org

Human Activity Recognition Using Smartphone

and Smartwatch
Hamid M. Ali1, Ali M. Muslim2
1
Computer Engineering Dept., Baghdad University, Iraq, [email protected]
2
Computer Engineering Dept., Baghdad University, Iraq, [email protected]

Abstract— Human activity recognition is influential subject in different fields of human daily life especially in the mobile
health. As the smartphone becomes an integrated part of human daily life which has the ability of complex computation,
internet connection and also contains a large number of hardware sensors, encourage implementation of the human activity
recognition system. Most of the works done in this field imposed the restriction of firmly fixing the smartphone in a certain
position on the human body, together with machine learning mechanism, to facilitate the process of classifying human
activities from the smartphone sensors raw data. To overcome this restriction, the proposed approach incorporated a
smartwatch, fixed on the human ankle, together with smartphone freely carried by the user. The use of smartwatch assisted
in providing distinctly separable signal variation from the smartwatch accelerometer and gyroscope sensors raw data which in
turn facilitated the use of a threshold-based mechanism to classify 20 various human activities. Furthermore, this work
provides a service for remotely real-time monitoring of the user human activities the system is tested with different subjects
and achieved an accuracy of 97.5%.
.
Keywords— Real-time monitoring, human activity recognition, threshold-based mechanism, mHealth, smartphone, and
smartwatch.
——————————  ——————————
quadratic classifier, k-nearest neighbor algorithm,
1. INTRODUCTION support vector machine, and artificial neural networks
Recently, the smartphone becomes an integrated part of as classifiers to classify 5 activities. Actually the lower
human daily life and is turned into more and more number of activities, 5 only, not enough to predict user
sophisticated device with growing the computation health. The accuracy achieved by this research is 84.4%.
power, network capability and sensing powers. Also, it In [3] the authors used fixed smartphone in the front
contains a large number of hardware sensors such as pocket of the user trouser using both Clustered KNN
accelerometer, gyroscope, compasses, barometer, and Naïve Bayes as classifiers to recognize 4 activities.
temperature, humidity, light sensor and GPS receiver. Actually the lower number of activities, 4 only, not
All these features encourage implementation of human enough to predict user health. The best performance
activity recognition system and make the smartphone to achieved by this research is 92%. In [4] the author's used
become a rich environment for many systems like smartphone fixed on the user's chest using SVM as a
healthcare system called Mobile Health (mHealth) classifier to recognize 16-activities. The total recognition
system. Mobile Health system is the intersection rate of 95.03% is achieved by this work. In [5] the
between Electronic Health (eHealth) and smartphone author's used smartphone fixed to one of the four
technology [1]. Smartphone sensors have become a positions on the waist, wrist, chest and thighs using
source data to measure various human activities, for SVM as classier to classify six activities.
example, physical states such as running, walking,
walking upstairs and downstairs. Monitoring of these Actually the lower number of activities, 6 only, not
activities can be used to predict user health and provide enough to predict user health. The accuracy achieved by
health care services at the right time. In the last few this research is 96%. In [6] the authors used smartphone
years, several works have been established using a fixed in one of the user trouser pockets in addition to a
smartphone as a tool for building human activity fixed wearable sensor in the user's chest (Zephyr
recognition system. The authors in [2] used smartphone BioHarness) using Weka machine learning package to
fixed at the position around waists such as jacket pocket recognize 12- activities. The best performance achieved
and pants pocket, with arbitrary orientation using a by this research is 95%. The Zephyr BioHarness sensor is

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 568

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

not easy to use for the patient people because it is Figure 1 Overall system structure
relatively big and it hinders breathing by tightening it
around the chest. In [7] the authors used a smartphone The main restriction that is imposed by the existing
fixed in one of the front-right and the back-left user systems is: Fixing firmly the smartphone on the human
trouser pockets using a neural network as a classifier to body; actually this is not a realistic solution, as the user
recognize 5 activities, the overall accuracy is 93.311%. sometimes wants to serve in the smartphone which
The training data sets for this work is difficult and takes implies holding the smartphone in different positions
a long time because the training of the neural network and is then subjected to drastic movements. This
needs to be performed by using MATLAB on a desktop maneuver affects the smartphone inertial sensors
workstation. In [8] the author's used smartphone fixed coordinate’s readings which could then produce
on one of five body positions, right jeans pocket, left inaccurate activity recognition results. As a consequence,
jeans pocket, belt position, right upper arm and right the proposed system integrated the use of smartwatch
wrist using SVM as a classifier to recognize 7-activities. fixed on user's ankle together with smartphone carried,
The highest classification accuracy achieved by this by the user, in any position and direction.
research is 82%.
Figure 2 illustrates the position and direction of placing
To obtain best results from the smartphone sensors raw and fixing the smartwatch on the right human ankle-
data, all the related work imposed the restriction of foot. The main technical reason behind using the
fixing the smartphone on the human body. Actually, this smartwatch is to place the smartwatch sensors
restriction is cumbersome and prevents the user from (accelerometer and gyroscope coordinates) in a position
using the smartphone for the purpose of calling, serving that become steady and precisely sensitive to human
the internet, social media etc. The rest of the paper is
body movements. In other words, the mentioned
organized as follows: Section 2 describes system
smartwatch sensors can provide more generalized and
approach mechanism, section 3 presents the results and
separable features to be exploited for more accurate
performance achieved from real test, section 4 discusses
results in the human activity recognition system.
the obstacles and challenges encountered in human
activities recognition, section 5 lists and justifies the
smartwatch role in achieving more accurate results,
section 6 concludes the paper.

2. SYSTEM APPROACH
To the best of the researchers’ knowledge that the most
up-to-date approaches, conducted in this field, depend
on the smartphone inertial sensors to accomplish human
activity recognition. The main contribution of this work
is the incorporation of the android smartwatch as a part
of the system components which in turn is used in
conjunction with a smartphone for better accurate
human activity recognition results. Figure 1 shows the Figure 2: Smartwatch position and orientation
main system components of the overall system structure.
Figure 3 shows the overall system design mechanism
and it's interoperated components that are working
simultaneously to achieve the intended human activity
recognition system. Furthermore, this work provides a
service for real-time monitoring of user human activities
which is accomplished by continuously sending user
identified activities to a database server to be accessed
by web browser by authorized people. The system is
implemented under Android Platform.

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 569

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

to inspect and measure the properties of the window

raw data pattern relative to the required activity
recognition. In another word, to find the best
approximate distribution of the raw data to which
activity it belongs. In time domain, the total features that
are analyzed and extracted for this work, concerning
each human activity, are shown in table 1. Also, the
feature extraction stage is executed in the smartwatch.

Table 1: Total extracted feature values for a window

No Feature
1 Mean of the smartwatch accelerometer x-axis
2 Mean of the smartwatch accelerometer y-axis
3 Mean of the smartwatch accelerometer z-axis
Number of the samplings that are greater than 10.5
4
m/s2 of the smartwatch accelerometer x-axis
Number of the samplings that are less than 9.1
5
m/s2 of the smartwatch accelerometer x-axis
Mean of the smartwatch accelerometer x-axis after
6 ignoring the data that are laying between 9.1 and
10.5 m/s2
7 Mean of the smartwatch gyroscope x-axis
Figure 3: Proposed System Design
8 Mean of the smartwatch gyroscope y-axis
2.1 Data collection 9 Mean of the smartwatch gyroscope z-axis
The data collection component integrated into the 10 Mean of the smartwatch barometer sensor
smartwatch and the smartphone, in the smartwatch to
collect data from smartwatch sensors while in the 2.4 Classification
smartphone to collect GPS data. The smartwatch collects
As a result of the use of smartwatch fixed on human
data from their sensors at the rate of every 20ms, the
body ankle to obtain a distinguishable and separable
sensor sampling rates are chosen carefully after
raw data from the smartwatch sensors, the classifier of
conducting many experimental tests with different
this work adopted a threshold-based mechanism to
situations to obtain the best clear features that are
recognize human activity. Hence, the feature extraction
suitable for all activities to achieve minimum processing stage separates essential features of each window
load and separable/generalized features. according to the adopted classification scheme of each
2.2 Windowing human activity. The extracted features of each window
size (on the smartwatch) are sent to the smartphone, via
The windowing process is used to split the continuously
Bluetooth, to be classified to the intended activity. the
collected data, of each sensor, into small time segments
human activities that are recognized by this work are:
called windows for the purpose of feature extraction. In
Walking (slow, normal and fast), Jog, Run (slow, fast
this work, a window size of 3-seconds is adapted to be
and run), Walking upstairs and downstairs, Elevator up
adequate for feature extraction. The windows size is and down, Sitting (completely or partially static), Sitting
chosen after having thoroughly analyzed the intended or Standing (completely or partially static) and Lying on
human activities to satisfy reliable activity recognition his (left, right, stomach and back). Actually, due to the
with high accuracy. For example, increasing the window space limit of the paper, samples of the human activity
size could miss detecting short activity like up and down classifications are discussed and analyzed in details in
stairs especially in case there is a stair with a half- the following section.
landing between floors. On the other hand, a short
window size increases the misclassification because 2.4.1 Walk, Jog and Run
some features would not be clear to recognize. The The smartwatch accelerometer and Gyroscope sensors
windowing process is executed on the smartwatch. are used to classify the walk, jog and run activities from
other activities, such as walking upstairs/downstairs
2.3 Feature Extraction and static, in the following manner:
Each smartwatch sensors window size exemplifies 150 1-Figure 4 shows the raw data of the smartwatch
vectors of raw data; some sensors have 3-axes accelerometer three axes signal variability for a walking
coordinates. The purpose of feature extraction process is activity for two window sizes. It is shown that as long as

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 570

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

the accelerometer x-axis is perpendicular to the earth activities, such as walking upstairs/ downstairs and
and the user's foot is on the ground, hence the x-axis represents a period when the user is putting her/his
value of the accelerometer is approximately 9.8 m/s2 foot on the ground. This period is different from
(±0.7 greater or less than 9.8 m/s2) which is first depicted user to another and it is also depending on the level
in the figure from the period 1 to 10. From period 11 to of the user vitality.
50 the user has lifted his foot above the ground which is b. After inspecting the raw data of the smartwatch
illustrated by the variation of the signal from accelerometer x-axis of the walk activity and
approximately 4 m/s2 to 20 m/s2. In this period the x- walking upstairs/downstairs activities, it is found
axis is no longer perpendicular to the earth, while the that the ratio of the number of samplings that are
user is lifting her/his foot above the ground that above than 10.5 m/s2, in walk activity, to the
produces different values for the x-axis depending on number of samplings that are below than 9.1 m/s2 is
the angle of the x-axis to the earth and the linear equal to or greater than 1.4 and the mean of the
acceleration of the user's foot. In the consecutive periods smartwatch accelerometer x-axis is greater than 10.3
from 51 to 66 and from 67 to 108, nearly the same pattern m/s2. While in walking upstairs/downstairs
mentioned above is repeated when the user's foot is on activities, this ratio is less than 1.4 and the mean of
the ground and when the user is lifting her/his foot the smartwatch accelerometer x-axis is less than 10.3
above the ground, respectively. m/s2. Hence, after ignoring all data that are laying
between 9.1 m/s2 and 10.5 m/s2, as mentioned in
This work exploited this pattern to discriminate between
step (a) above, the system is able to distinguish and
the user's foot is on the ground or is lifted up. The other
separate between the walk, jog and run activities
y-axis and z-axis signal variations do not yield a clear
from one side and the walking upstairs/downstairs
enough systematic pattern to be exploited for the
activities from the other side.
mentioned activity recognition. Therefore, in order to
achieve very high accuracy in discriminating between
Walk, Jog and Run activities and other activities such as
acceleration m/s2

19
Gravity + linear

walking upstairs/downstairs, it is practically found that

the best features that are unique and separable in the X
period are when the user is raising her/his foot above -1 Y
the ground.
1
25
49
73
97
121
145
169
193
217
241
265
289
Z
-21
Sampling rate (20 ms)
acceleration m/s2

19
Gravity + linear

X Figure 5: Raw data of the smartwatch accelerometer

-1
sensor for walking upstairs activity
Y
1
29
57
85
113
141
169
197
225
253
281

Z
-21
acceleration m/s2

19
Gravity + linear

Sampling rate (20 ms)

X
Figure 4: Raw data of the smartwatch accelerometer sensor -1 Y
for walk activity
1
25
49
73
97
121
145
169
193
217
241
265
289

Z
2-Figures 5 and 6 show the raw data of the smartwatch -21
Sampling rate (20 ms)
accelerometer three axes signal variability of walking
upstairs and downstairs activities, respectively, for two
Figure 6: Raw data of the smartwatch accelerometer
window sizes. Similar to the walking activity, the x-axis
sensor for walking down stair activity
is used to discriminate whether the user foot is on the
3-Figure 7 shows the statistics relationship procedure,
ground or is lifted up. These similarities raise the
mentioned in step 2 above, for separating the walk, jog
problem of how to discriminate between the walk, jog,
and run activities from walking upstairs/downstairs
and run activities from one side and the walking
activities. The x-axis in the figure represents the ratio of a
upstairs/downstairs from another side. Actually, after
number of samplings that are above than 10.5 m/s2 to
conducting several tests, the following procedure is used
the number of samplings that are below than 9.1 m/s2.
to discriminate between the above-mentioned activities:
While the y-axis represents the mean value of the x-axis
a. Ignoring all x-axis readings of the smartwatch
of the smartwatch accelerometer sensor, in m/s2,
accelerometer sensor when their values laying
excluding the number of samplings that are laying
between 9.1 m/s2 and 10.5 m/s2 (±0.7 greater and
between 10.5 and 9.1 m/s2. Each colored dot represents
less than 9.8 m/s2) because it is common in the other

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 571

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

the ratio, mentioned above, of one window size for each the speed of the user is increased. This phenomenon is
activity walk, walking upstairs and downstairs. This test observed clearly, for example, from the 3-axes signal
is conducted for 50 window sizes for a person doing the variation shown in the figures mentioned above. It is
walk, walking upstairs and downstairs activities. It is also found that the sum of the mean values of the
noticed from the figure that the adopted procedure has absolute three axes in walk activity is less than that of
discriminated the walking activity from walking Jog activity and the latter is less than Run activity.
downstairs/upstairs activities at threshold ratio of 1.4
and mean of 10.3 m/s2. 10
14 5
accelerometer x-axis after ignoring

Rad/sec
X
12
the samplings that are laying
The mean of the smartwatch

0
between 9.1 and 10.5 m/s2

10 y

73
1
25
49

97
121
145
169
193
217
241
265
289
-5
8 Z
6 -10
upstaie Sampling rate (20 ms)
4
downstair
2 Figure 8: Raw data of the smartwatch Gyroscope
walk
0 sensor for walk activity
0 1 2 3
The ratio of number of samplings that are above than 20
10.5 m/s2 to the number of samplings below than 9.1
m/s2 of smartwatch accelerometer x-axis 10 X
Rad/sec
Figure 7: Separation between walk, and walking 0
Y
upstairs/downstairs activities 1
25
49
73
97
121
145
169
193
217
241
265
289
Z
-10
After having separated the Walk, Jog, and Run Sampling rate (20 ms)
activities from other two activities walking upstairs
and downstairs, the walking upstairs and downstairs Figure 9: Raw data of the smartwatch Gyroscope
activities include intervening points as shown in figure sensor for Jog activity
7. The walking upstairs and downstairs activities are
discriminated from each other using the smartwatch
20
barometer sensor. After determining that the current
activity falls in the domain of Walk, Jog, or Run. Now it 10
x
Rad/sec

is the time to separate between the walk (slow, normal,

and fast), jog and run (slow, normal, fast) activities 0
Y
1
25
49
73
97
121
145
169
193
217
241
265
289
themselves which is accomplished by using the -10
smartwatch gyroscope sensor. The gyroscope sensor is Z
a motion sensor used to measures a device's rate of -20
Sampling rate (20 ms)
rotation in rad/s around each of the three physical axes
(x, y, and z). The values of the Gyroscope three axes are
positives and sometimes negatives depending on the Figure 10: Raw data of the smartwatch Gyroscope
direction of the rotation. The procedure that is used to sensor for Run activity
discriminate between these activities is explained as
follows: 2- Hence, the procedure used to discriminate between
1- Figures 8, 9 and 10 show the raw data of the the activities, mentioned in step 1 above, is by
smartwatch gyroscope three axes signal variability of finding the sum of the mean values of each absolute
the Walk, Jog, and Run activities, respectively, for two Gyroscope three axes readings to provide motion
window sizes. The figures depict that the repetition of level for each window size, and then applying a
the Gyroscope three axes signal variations (for the threshold-based value to recognize each activity as
same period) in the Walk activity is less than the Jog follows:
activity which in turn is less than the Run activity. The a. Walk slow: Threshold between [1.5 rad/s – 2.3
increased repetition of the signal variations for the rad/s).
Walk, Jog, and Run activities occurs because the b. Walk normally: Threshold between [2.3 rad/s – 4
movement speeds of these activities are different. The rad/s).
repetition of signal variations is increased whenever c. Walk fast: Threshold between [4 rad/s – 5 rad/s).

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 572

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

d. Jog: Threshold between [5 rad/s – 6 rad/s). The rest of the human activities, presented in this work,
e. Run slow: Threshold between [6 rad/s – 6.5 are classified in a similar manner that a threshold based
rad/s). values are applied concerning the pattern which is
f. Run normal: Threshold between [6.5 rad/s – 7.5 produced from performing the intended activity.
rad/s).
g. Run fast: Threshold (greater than 7 rad/s). 3 RESULTS AND PERFORMANCE
Twenty human activities have been practically tested in
Figure 11 shows the variation of the sum of the mean this work. In order to reduce the amount of the results
values for the absolute Gyroscope three axes readings presented in the table, the test results are concentrated
to provide motion level for 88 window sizes that on the main human activities. For example, the sub-
illustrate the function of the Gyroscope in determining activities of the walk (slow, normal and fast) are
the above seven mentioned activities. The figure shows omitted and regarded as the only walk. In fact, these
the motion level from the first window which sub-activities are integrated into the system to
represents that the user is completely static as long as represent the level of the motion which in turn is used
the mean is zero, to window number 82 which to predict the level of the user vitality. Several
represents the user is running fast. experimental tests are conducted with 5-subjects (2-
male 2-female and 1-boy). The overall tests of each
10 activity for whole users are represented in the
Sum of the mean values for

9 confusion matrix shown in table 2. The type of the

the absolute Gyroscope

8
device used in this work test is Huawei smartwatch
three axes readings

7
6 watch and HTC one smartphone. Also, this work has
5 been tested using different types of Android platform
4
3 smartphones like Galaxy 4 and Sony Xperia Z1. The
2 results obtained from the three smartphones are fairly
1
0 similar.
1 7 13 19 25 31 37 43 49 55 61 67 73 79 85
3 seconds window size

Figure 11: Sum of the mean values for the absolute

Gyroscope three axes readings during the Walk, Jog,
and Run activities

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 573

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

sometimes it is very difficult to achieve 100%

4 CHALLENGES classification accuracy even if distinct and separable
During the time of the system analysis and results features obtained from the sensors. In fact, it is
inspection of the human activity recognition systems, impossible to synchronize the start and end of each
it is found that some unpredictable phenomenon activity time with the start and end of each window
occurs during the time of recognition. Furthermore, size. Also, when the user transition time between
there are other factors, such as window size and different activities is increased, then the probability
sampling rate, also affect the accuracy of the human of classification accuracy is decreased.
recognition system. These phenomenon and factors
are explained below: 2- Choosing adequate time for the window size is very
crucial subject because there are many pros and cons
1- The phenomenon that affects the classification of the when increasing or decreasing the time of the
human activities is the completion of the activity window size:
motion inside one window size. It happens that part a- When the activities are changing in a short period
of the activity motion occurs in a certain window of time (less than one window size) then the
and the another part occurs in the next window. In ability to reorganize such activities is decreased
other words, the activity does not necessary start at when increasing the window size, and vice versa.
the beginning of each window. For example, in a b- The accuracy of recognizing human activities will
walking activity, a small part of the walk activity increase when increasing the time of the window
signal variation occurs in the current window, while size because sufficient and clear features will be
the largest signal variation part occurs in the next available, and vice versa.
window in which the user starts the walking c- The amount of the database storage will decrease
upstairs activity. Hence, because of the effect of the when increasing the time of the window size, and
walk signal variation, the latter window could be vice versa.
misclassified as a walking activity. Hence,

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 574

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

d-The computation power of the system will 3- Due to the analyses mentioned in step 2 above,
decrease when the time of the window size is and because the features obtained (from solely
increased and vice versa. using the smartphone sensors) are not
e- Increasing the time of the window size is not generalized and separable, an artificial
suitable for real-time monitoring. intelligent classifier is needed to predict and
discriminate each activity features. The
3- The other factor that affects the accuracy of the burdens of using artificial intelligent classifier
recognition system is the sampling rate which has are:
the following pros and cons:
a- Increasing the sampling rate will increase the c- The recognition system must be trained for each
accuracy of the human recognition system, and user or many users.
vice versa. d-The training data and the classifier must be loaded
b- Increasing the sampling rate will increase the into where the processing will take place,
computation power of the system, and vice versa. smartphone, networked computer.
e- The artificial intelligent classifier does not
guarantee accurate recognition results due to the
5 SMARTWATCH ROLE different situation of the smartphone when it is
Actually, most of the works done in the field, of
fixed on the human body.
human activity recognition, adopted the approaches of
f- Also, extra processing time is required by the
using smartphone alone or smartphone together with
artificial intelligent classifiers to accomplish their
an external sensor. These approaches associated with
duty.
difficulties and obstacles that prevent them from
achieving flexible and highly accurate systems. These e-The signal variation of the accelerometer and
difficulties are analyzed and justified as follows: gyroscope, for the user performing walking upstairs
1- Due to the change in smartphone position and and downstairs activities, are varying relative to the
orientation, the human activity recognition type and structure of the stairs. Hence, in the
system does not work properly when the systems that depend on the training data set, the
smartphone is freely held by the user's hand or accuracy of walking upstairs and downstairs
in a bag as most women do. recognition depends on the type of the satires used
in the training data set. In other words, the system
2- In order to obtain, to a certain degree, distinct
could be prone to error when different types of stairs
signal variation from the smartphone sensors,
are conducted by the user.
all the related works have imposed the
restriction of carrying the smartphone in a 4- The activity recognition system does not work
fixed position on the human body, such as properly and sticks to only one activity, when
inside trousers pocket or on the chest. This inadvertently people leave their smartphones,
restriction has the following drawbacks and most of the time, on a table when they are
shortcomings: working or lying.

a- Sometimes the user wants to use the The main contribution of this work is the
smartphone for the purpose of calling, serving incorporation of the android smartwatch as a part of
the internet, social media, social networks, the system components. From the practical test, it is
knowing the weather, camera etc. Moving the found that the smartwatch, fixed on the human ankle,
smartphone from its fixed position impose a is the best external sensor that can be used in
wrong interpretation of the smartphone sensor conjunction with a smartphone for better accurate
coordinates due to the change in smartphone human activity recognition results. Justifications and
position and orientation. the rational reasons behind the use of the smartwatch
as part of the proposed system components, compared
b- The smartphone in the human pocket is not to the smartphone alone or smartphone with external
settled and to somewhat changing its dedicated sensor, are discussed as follows:
orientation due to the difference in the type of 1- Smartwatch is equipped with some sensors that are
the clothes. Also, most of the pockets are not not available on most smartphones, such sensors like
tight and small enough to prevent the barometer sensor, which is used in this work to
smartphone from changing its position and discriminate between walking upstairs and
orientation during the user movement.

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 575

Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

downstairs, and heart rate sensor that can be used 6- The smartwatch minimizes the amount of data
for heart rate checking. (network load) transferred from the smartwatch to
the smartphone. Actually, only the extracted
Due to the smartwatch specifications like light weight, features of each window size (of 3 seconds) are sent
small size, water resistance and wireless to the smartphone. Hence, the extracted features are
communication, hence it can be placed and fixed in a sent once every 3 seconds rather than sending the
position that generates generalized and separable smartwatch raw data every 20ms.
features from its sensors. Figure 12 shows the
smartwatch fixed on human ankle compared to the 6 CONCLUSION
smartphone fixed on the human trousers pocket in a During the study and development of human activity
walking activity. It is realized and observed from the recognition system, several points observed and
figure that the range of displacement and orientation noticed:
performed by the smartwatch are much wider and 1- It is realized and concluded that for such system to
bigger than in a smartphone. The range of the work properly, then it is necessary and compulsory
smartwatch displacement and orientation makes it obtaining clear distinguishable signal variations
generates notable and distinguishable signal variations from the smartphone sensors such that 3-axes
by its accelerometer and gyroscope sensor coordinates. accelerometer and gyroscope. In response to that,
all the like systems have imposed the restriction of
fixing the smartphone on the human body.
2- Fixing the smartphone on the human body is not
practical and is not the ultimate solution for such
systems to work properly which is also prone to a
partial movement that could affect the signal
variation of the smartphone sensors.
3- Furthermore, the variation in the pattern movement
of each activity from user to user due to the change
in the user gender and age, also within the same
user due to the change in the user vitality and
mood makes it hard and difficult to achieve subject
independent and accurate real-time human activity
Figure 12: Represent the user variation for fixed smartwatch
recognition system.
and fixed smartphone
4- Consequently, integrating the smartwatch (steady
fixed on the human angle) with a smartphone is
2- Unlike fixed smartphone on the human body, the
due to the main reason of obtaining generalized
proposed smartwatch system is continuously
and separable features for a wide number of
working even when the smartphone is in use for
activities with very high accuracy.
other purposes such calling, internet serving, or
5- For future work, to overcome the misclassification
even when the smartphone placed on a table near to
that arises from using fixed window size, a variable
the user.
window size can be studied and incorporated in
3- The use of the external sensor in conjunction with
such system that prevents the intervening of two
the smartphone, like Zephyr Bio Harness, is too
activities in one window size.
expensive and is not Android platform compatible;
also it does not contain all the sensors of the
smartwatch. REFERENCES
4- Fixing the smartphone position and orientation on
the human body is not feasible due to the size of the [1] S. Adibi, "Introduction," in Mobile Health A Technology Road Map,
smartphone in comparison with smartwatch Springer International Publishing Switzerland, 2015, p. 1.
5- Due to the processing capability of the smartwatch, [2] R. Amin, C. Chien-An and L. Yan, "Human Activity Recognition using
it minimizes, to a large degree, the computation Smartphone," CSCE666, 2011.
power load on the smartphone by performing most [3] K. Mustafa, I. Ozlem Durmaz and E. Cem, "Online Human Activity
of the feature extraction process on the smartwatch Recognition on Smart Phones," in 2nd International Workshop on
rather than the smartphone. The processing Mobile Sensing, Beijing, China, 2012.
capability of the smartwatch is not available in the [4] H. Yi and L. Ye, "Physical Activity Recognition Utilizing the Built-In
external wearable sensors. Kinematic Sensors of a Smartphone," International Journal of

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 576

 Hamid M. Ali et al. ," Human Activity Recognition Using Smartphone and Smartwatch”, International Journal of Computer
Engineering In Research Trends, 3(10):568-576,October-2016. DOI:10.22362/ijcert/2016/v3/i10/48906

Distributed Sensor Networks, vol. 9, no. 1550-1477, p. 10, 2013.

[5] A. Davide, G. Alessandro, O. Luca, P. Xavier and L. R.-O. Jorge, "A
Public Domain Dataset for Human Activity Recognition Using
Smartphones," in European Symposium on Artificial Neural
Networks, Computational Intelligence and Machine Learning,
Bruges Belgium, ESANN, 2013, pp. 437-442.
[6] Y. Bingchuan, H. John and E. Yalda, "Smartphone-based Activity
Recognition Using Hybrid Classifier-Utilizing Cloud Infrastructure
for Data Analysis," in Proceedings of the 4th International
Conference on Pervasive and Embedded Computing and
Communication Systems, Lisbon, Portugal, 2014.
[7] T. H. Cesar and A. L. Andres, "Accelerometer-Based Human Activity
Recognition in Smartphones for Healthcare Services," in Mobile
Health A Technology Road Map, Springer International Publishing
Switzerland, 2015, p. 147.
[8] D. Natarajasivan and M. Govindarajan, "Filter Based Sensor Fusion
for Activity Recognition using Smartphone," International Journal
of Computer Science and Telecommunications, vol. 7, no. 5, pp. 26-
31, 2016.

©2016, IJCERT All Rights Reserved DOI:10.22362/ijcert/2016/v3/i10/48906 Page | 577

View publication stats