See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/261427466

Empirical study based on machine learning approach to assess the QoS/QoE

correlation

Conference Paper · June 2012

DOI: 10.1109/NOC.2012.6249939

CITATIONS READS

101 999

3 authors, including:

M.Sajid Mushtaq Abdelhamid Mellouk

University of Paris-Est Université Paris-Est Créteil Val de Marne - Université Paris 12
24 PUBLICATIONS 391 CITATIONS 366 PUBLICATIONS 4,795 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by M.Sajid Mushtaq on 11 April 2017.

The user has requested enhancement of the downloaded file.

 Empirical Study based on Machine Learning
Approach to Assess the QoS/QoE Correlation
M. Sajid Mushtaq, Brice Augustin and Abdelhamid Mellouk

University of Paris-Est Creteil (UPEC) Val de Marne

Image, Signal and Intelligent Systems (LiSSi) Lab, France
Transport Infrastructure and Network Control for E2E Services (TINCS) Group

Ahstract- The appearance of new emerging multimedia analyze video streaming services thoroughly in order to find
services have created new challenges for cloud service providers, out the degree of influence of (technical and non-technical)
which have to react quickly to end-users experience and offer a parameters on user satisfaction. Among these factors, one can
better Quality of Service (QoS). Cloud service providers should
find network parameters, which represent the QoS. Delay,
use such an intelligent system that can classify, analyze, and
adapt to the collected information in an efficient way to satisfy jitter and packet loss are the main parameters of QoS, and they
end-users' experience. This paper investigates how different have a strong influence on user (dis)satisfaction. In addition to
factors contributing the Quality of Experience (QoE), in the network parameters, some other external environmental
context of video streaming delivery over cloud networks. factors have a great impact on user perceived quality, such as
Important parameters which influence the QoE are: network video parameters, terminal types, and psychological factors.
parameters, characteristics of videos, terminal characteristics
To evaluate the quality of multimedia contents, researchers
and types of users' profiles. We describe different methods that
are often used to collect QoE datasets in the form of a Mean dispose of two methods: the subjective and the objective
Opinion Score (MOS). Machine Learning (ML) methods are then method. The subjective method is proposed by the
used to classity a preliminary QoE dataset collected using these International Telecommunication Union (ITU) and the Video
methods. We evaluate six classifiers and determine the most Quality Expert Group (VQEG), and it consists of a group of
suitable one for the task of QoS/QoE correlation. people watching different video sequences under a specific
controlled environment, and rating their quality. The Mean
Index Terms-QoE, QoS, Machine Learning, Data
Opinion Score (MOS) is an example of a subjective
classification models.
measurement method in which users rate the video quality by
giving five different point score from 5 to 1, where '5' is the
I. INTRODUCTION best and ' l' is the worst quality. On the other hand, objective
method use different models of human expectations and try to

Msuch a way that they have become dominant in the

ULTIMEDIA services over the Internet are growing in
estimate the performance of video streaming service in an
automated manner, without involving human. For instance, a
global Internet traffic. Many new multimedia services (like
number of methods depend on video signal distortions due to
High Definition (HD) video, interactive video gaming) require
the encoding process and packet delivery delays.
more processing power, so the concept of Cloud Computing
This paper analyses the influence of QoS and other
improves end users' experience by managing these services at
parameters on the QoE of video streaming services. We
remote data centers. Because of this trend, a large number of
propose a testbed for subjective measurements, in order to
remote data centers have emerged, which is made possible by
collect QoE datasets in the form of a MOS score and evaluate
the availability of fast and reliable internet networks. In Cloud
the impact of different parameters (delay, jitter, packet loss,
Computing, many applications and services are available to
video types, users' profile and terminal characteristics) on
users remotely. As a consequence, users expect better network
perceived quality. Based on these datasets, we evaluate how
Quality of Service (QoS) with a high quality standard [1].
Machine Learning (ML) methods can help in building an
The concept of Quality of Experience (QoE) has recently
accurate and objective QoE model that correlates low-level
gained greater attention in Cloud Computing networks. Its
parameters with (high-level) quality. ML methods successfully
main objective is not only to consider and evaluate the
apply on this type of problem, as the main application of ML
network QoS, but also to keep it nearest to end users in order
is data mining. The concept of data classification is significant
to better estimate the perceived quality of services. In fact, the
in ML, and the following methods are widely used in this
aim of network service providers is to provide a good user
field: Naive Bayes (NB), Support Vector Machines (SVM), k­
experience with the usage of minimum network resources. It is
Nearest Neighbours (k-NN), Decision Tree (DT), Random
also essential for network service providers to consider the
Forest (RF) and Neural Networks (NNet). In our study, we use
impact of each network factors on user perception, because
ML methods to classifY original datasets, collected in the form
their businesses are highly dependent on users' satisfaction.
of MOS scores. We analyze datasets on six classifiers, in
Video streaming services has now major shares of internet
search of the most suitable one for the task of QoS/QoE
traffic. To meet the high expectation of users, it is necessary to
correlation. As QoE is purely related to end-users, we can

978-1-4673-0951-6/12/$31.00 ©2012 IEEE

analyze the effect of both core networks and cloud networks it is necessary to precisely identifY the factors that affect QoE,
as a whole. Therefore, in this paper, we use the terms of and then try to define methods to measure these factors. We
"cloud service provider" and "network service provider" categorize these factors in four types, as follows.
interchangeably.
A. Network Parameters
The remaining of the paper is structured as follows: we
discuss related works in Section II. Section III contains QoE is influenced by QoS parameters, which highly depend
discussion about metrics affecting the QoE. Section IV gives a on network elements. Key factors are packets loss, jitter and
brief detail of ML data mining techniques and discusses six delay. The impact of each individual or combined factors lead
ML classification methods. Section V is dealing with the to blockings, blurriness or even blackouts with different levels
different environmental approaches for assessing video QoE. of quality degradation of video streaming.
Packet losses have a direct effect on the quality of video
Section VI focuses on the testbed experiment setup and
discusses experimental details. We present our results in presented to end users. Packets losses are occurring due to the
Section VII, and conclude the paper in Section VIII. congestion in the networks and late arrival of packets at
application buffers. If packet loss is occurring, then it becomes
difficult for the video decoder to properly decode the video
II. RELATED WORK streaming. This results in the degradation of video quality.
Jitter is another important QoS parameter which has a great
A large number of research works have been achieved to
impact on video quality. It is defined as the variance of packet
correlate QoS with QoE in search of capturing the degree of
arrival times at the end-user buffer. It occurs when packets
user entertainment. Some other techniques are also developed
travel on different network paths to reach the same
to evaluate and predict the users' QoE, in order to deliver a
destination. It causes jerkiness and frozen video screens.
better quality of service to end-users. To do so, many testbed
However, the effects of jitter can be nullified or reduced to
studies have been undertaken, involving different tools,
some extent, by adding a large receiving buffer at the end user
equipments and methods. In [5], a testbed experiment is
and delay the playout time of the video. By adding a larger
proposed, to explore how network QoS affects the QoE of
buffer, the tolerance level of network jitter will be high but
HTTP video streaming. In [6], a testbed is implemented to
there is still a playout limitation to tolerate its effects. When
collect data with the help of ten participants, correlating
packets arrive out of order, after the expiration of a buffering
stream state data with video quality ratings. These datasets
time this packet is discarded by the application. In this
were used to develop self-healing networks, i.e., having the
context, jitter has the same influence as packet loss [2].
ability to detect the degradation of video streaming QoE, react
Delay is defined as the amount of time taken by the packet
and troubleshoot network issues. The correlation of QoE-QoS
to travel from its source until its reception at the final
is studied in [7] by controlling QoS parameters (packet loss,
destination. Delay has a direct influence on user perception
jitter, delay) of networks.
while watching the video. If the delay exceeds a certain
Because subjective campaigns are, by nature, quite limited
threshold, then its effect is a freeze and lost blocks of video.
in size and number of participants, it is impossible to cover all
The threshold of delay values varies according to the nature of
possible configurations and parameter values. However, a
the multimedia service.
QoE prediction model is proposed in [8], for the unseen cases
based on primarily limited subjective tests. This model B. Video Characteristics
reduces the need of cumbersome subjective tests, to the price The characteristics of video have a direct influence on QoE.
of a reduced accuracy. To overcome the weakness of [8], a The characteristics of video are defined in terms of frame and
Learning-based prediction model is proposed in [9]. In [10], a resolution rate, codec and types of content. Network service
machine learning technique is proposed using a subjective providers reduce the bit rate of video streaming services
quality feedback. This technique is used to model according to the available bandwidth, which strongly
dependencies of different QoS parameters related to network influences perceived quality. The impact of these two factors
and application layer to the QoE of the network services and is presented in [3]. This work shows that these two parameters
summarized as an accurate QoE prediction model. also have high impact on the users' satisfaction while using
the video streaming services.
The video content types can also influence users' opinions.
III. METRICS AFFECTING THE QOE In case of "interesting" video contents, a user will be more
QoE is very subjective by nature, because of its relationship tolerant, and low quality will not influence user's experience
with user's point of view and its own concept of a "good as much as in case of a boring content. In [4], authors found
quality". However, it is very important to devise an automated that if users show enough interests in the video content, then
strategy to measure it as realistically as possible. The ability to they can accept even an extremely low frame rate. In this
measure QoE would give network operators some sense of the study, a group of participants interested in soccer were
contribution of the network's performance to the overall selected. Participants gave a very high acceptable rate (80%),
customer satisfaction, in terms of reliability, availability, although they watched a video with only 6 frames per second.
scalability, speed, accuracy and efficiency. As a starting point, This result clearly shows that if there is a sufficient interest in
the topics, then the human visual system can tolerate the assumption that all terms are conditionally independent of
relatively gross interruptions and users can tolerate a very low each other in a given category. Because of this independence
quality video streaming. assumption, the parameters for each term can be learned
Uncompressed video requires a large amount of storage and separately, and as a result this simplifies and speeds up the
bandwidth, to be streamed over a network. Therefore, a large computation operations [12].
number of video codecs were developed (H.262, H.263,
2 Support Vector Machines
H.264, WVID, WMV3, etc) to compress the video in an
Support Vector Machines (SVM) are a very powerful
effective and efficient way, so that acceptable quality of
classification method, used to solve the two-class-pattern
videos can be maintained. Each codec has its own standard
recognition problem. It analyzes the data and tries to identifY
way to compress the video contents, providing various video
patterns so that a classification can be done. The idea here is to
quality levels. The quality levels of video codecs explain the
find the optimal separating hyperplane between two classes,
important impact of codecs on users' perceptions. by maximizing the margin between the closest points of these
C. Terminal Types two classes.
The SVM classifies data which have the possibility to be
Consumers' electronic devices expand largely with the
linearly separable in their origin domain or not. The simple
rapid growth of new advancement in telecommunication
linear SVM can be used if the data is linearly separable. When
industries, and they offer a large number of products available
the data is non-separable in their original domain through the
for modern multimedia services. These new-generation hyperplane, then it can be projected in an higher order
devices are available in different sizes, processing powers, dimensional Hibert space. By using a kernel function, it is
advanced functionalities, usage and so many other aspects. We possible to linearly separate the data in a higher dimensional
can classifY these devices into three categories: Personal space [14].
Computers, Mobile devices, and Television (TV). All these
terminal devices influence user satisfaction while using video 3 K-Nearest Neighbors
streaming services. For example, it is pointless to send HD The k-Nearest Neighbours (k-NN) method is an instance­
video streaming on a low processing terminal equipped with a based ML method and it is considered a very simple method
small screen. as compared to all other ML classification methods.
In supervised statistical pattern recognition, the k-NN
IV. MACHINE LEARNING CLASSIFICATION METHODS method often performs better than other methods. There is no
need of prior supposition of distribution, when the training
Machine Learning (ML) is concerned with the design and
sample is drawn. It works in a very simple and straightforward
development of programs and algorithms which have the
way: to classifY any new test sample, it compares the new test
capability to automatically improve their performance either
sample with all other samples in the training set. The category
on the basis of their own experience over time, or earlier data labels of these neighbours are used to estimate the category of
provided by other programs. The general functions provided
the test sample. In other words, it calculates the distance of the
by ML are training, recognition, generalisation, adaptation,
new test sample with the nearest training sample, and then at
improvement and intelligibility. There are two types of ML,
this point finds out the classification of the sample [15].
i.e. unsupervised and supervised learning. Unsupervised refers
to find the hidden structure in unlabelled data in order to 4 Decision Tree
classifY it into meaningful categories, while Supervised Decision Tree (DT) is a method used to create a model to
Learning assumes that the category structure or hierarchy of predict the value of a target variable based on several input
the database is already known. They require a set of labelled variables. The structure of DT consists of the following
classes and return a function that maps the database to the pre­ elements: (1) internal nodes, that tests an attribute; (2)
defined class labels. In other words, it is the search for branches, corresponding to attribute values, and (3) leaf nodes,
algorithms that reason from externally supplied instances to which assign a classification.
produce general hypotheses. It makes predictions about future Instances are classified by starting at the root node, and
instances in order to build a concise model that represents the based on the feature values, the tree is sorted down to some
data distribution. In our case we are considering Supervised leaf node. It is a simple classifier which can efficiently classifY
Learning, and we are interested in classification methods new data and compactly store them. It has the capability of
because of the discrete nature of our datasets. We have applied reducing complexity and automatically features selection. The
six ML data classification methods on our datasets, which are information about the prediction of classification can be easily
Naive Bayes (NB), Support Vector Machines (SVM), k­ interpreted, thanks to its tree structure. Finally, the accuracy of
Nearest Neighbours (k-NN), Decision Tree (DT), Random DT is less affected by user-defined factors as compares to the
Forest (RF) and Neural Networks (NNet). k-NN classifier [16].
1 Naive Bayes 5 Random Forest
The Naive Bayes (NB) classifier is a probabilistic model that Random Forest (RF) is an ensemble classifier, that uses
uses the joint probabilities of terms and categories to estimate multiple models of several DTs to obtain a better prediction
the probabilities of categories given in a test document. The performance. It builds on many classification trees and a
naive part of the classifier comes from the simplifYing bootstrapped sample technique is used to train each tree on the
set of training data. This method only searches for a random
subset of variables in order to find out a split at each node. For
the classification, the input vector is submitted to each tree in
the RF, and each tree votes for a class. Finally, RF chooses the
class which with the highest number of votes. It has the ability
to handle larger input data sets than other methods [17].

6 Neural Networks
A Neural Network (NN) is a structure of a large number of
units (neurons) linked together in a pattern of connections.
The interconnections are used to send signals from one neuron
to the other. The calculation by neural networks is based on
the spread of information between basic units of computation. Database

The possibilities of each one are small, but their

interconnection allows a complex overall calculation. Figure 1 Framework
The behaviour of a neural network is determined by its
architecture: number of cells, how they are connected and the The framework setup contains the following items:
weights assigned to each connection. Each connection a. A Firefox plug-in is developed and installed on end
between two neurons is characterized by its weight, that users' devices to run the real-time experiment. In
measures the degree of influence of the first neuron on the particular, the plug-in is able to detect the presence of a
second one. The weight is updated during a training period.
video in a Web page, and automatically adds a button,
This method has the ability to solve multivariate non-linear
on which the user can click whenever user is unhappy
problems. Its performance is degraded when it is applied on a
of the video quality.
large number of training datasets [17].
b. A large number of remote volunteers are invited to
watch video sequences online, on their machines.
c. Each video can have different characteristics and
v. EXPERIMENTAL ENVIRONMENT FOR QoE ASSESSMENT
experience various, realistic QoS parameters.
QoE assessment is subjective by nature, because it tries to d. During the video, and in the end of it, users rate the
match the real perception of users while using a service. This quality of video (MaS) according to their perception.
section discusses two different approaches to collect QoE e. All feedback information are stored in the database for
datasets: a crowd-sourced, and a controlled environment the analysis of QoE parameters.
approach.
B. Controlled Environment Approach
A. Crowd-sourcing Approach
In parallel to the crowd-source approach, we also take an
In this part, we consider the crowd-sourcing approach,
orthogonal approach in which the environment is totally
which helps in gathering a large amount of QoE tests. This
controlled. The ITU has provided guidelines to conduct such
approach allows a large number of users to participate
subjective tests in a controlled environment, including the
remotely. In crowd-sourcing, one assigns the video testing
selection of participants that represent the users of a service
task to a large number of anonymous users who can
[11].
participate from different regions of the world. For this
In order to analyze the impact of different parameters on
purpose, we can use the wide variety of video streaming
user's perceived quality in video streaming, a subjective test is
services currently offered on the internet. In this study, we
carried out with the participation of 45 persons. The
select YouTube, because it is considered as one of the most
participants watch the video streaming and rate the quality of
prominent video streaming website. According to [13], in May
the different videos. In the next sections, the relationship
2010, 14.6 billion videos were served per day.
between MaS and the different low-level parameters will be
In this context, we propose a framework for recording the
analyzed.
degree of users' satisfaction, in the form of feedback while
In this testbed experiment, the QoS parameters (packet loss,
using the video services on the Internet. The framework
jitter and delay) are varied in a fully controlled manner.
detects the presence of a YouTube video on a Web page, and
Further, their influence on user perception is recorded in the
automatically adds a button on which the user is asked to
form of a MOS. In addition, another parameter is taken under
click, whenever she is unhappy of the video she is viewing.
observation, the conditional loss. Conditional loss reflects the
The plugin also stores the QoE values, which, in terms, are
loss probability of the next packet, given that the current
used to build a large dataset of heterogeneous users, devices
packet has already been lost. As most real-time applications
and situations. In future work, this dataset will be used for
exhibit a certain tolerance against occasional packet losses,
enhancing a QoS/QoE correlation model for video service.
this metric helps in concentrating losses on a single part of the
Fig. 1 shows the framework structure in which remote users
sequence, which makes the losses occasional.
participate via the IP network (Internet).
For our experiment, the relevant parameters and their
selected values are given in Table 1.
 (NetEm) are configured on a Linux as. The experimental
Table 1 QoS metrics setup is shown in Fig. 2.

Parameters Values

Delay Oms, 30ms, 60ms,100ms 120ms PHP r::'\

,0; �T :=tHEll -.!J
Jitter Oms, 4ms, 8ms, 16ms, 32ms 3
CI'-nt ...... r SQLOolltablil.
HTMLlCSS
Loss 0% to 5% with a step of 0.5%
Figure 2 Experiment Setup
Conditional Loss 0%, 30%, 60%, 90%

We have stored 25 videos at the server side and the client

In this experiment, we consider the users participation
can reach them through a private Web site configured
according to ITU-R Rec. BT.500-1 I. Indeed, to obtain a
specially to stream those videos. During the session, when the
subjective notation according to this recommendation
participants should be non-experts, in the sense that the ; client changes the video, QoS parameters settings are also
randomly changed according to pre-defined values. Finally, a
should not be directly concerned with image or video quality
total of 25 videos were streamed at the client end, each time
as part of their normal work. User characteristics are also
using a different combination of parameters.
stored for analysis purposes, which include user's participant
At the client side, the user connects to the Web site to read
profile like age, gender, familiarity with video streaming, and
the description of the experiment. Before the beginning of the
interest on video content. End-user devices are Dell desktops
video streaming procedure, the user also fills a form with her
,,;ith Intel core duo processor, 2 OB of RAM, and a display personal information (age, gender etc). Users are unaware of
sIze set to 1024 x 740. Mozilla Firefox is used as the Web
the QoS parameters settings on the videos, and they are asked
navigator.
to rate the perceived quality after watching each video. The
rating is done by picking out one of the five quality levels of
Table 2 User Characteristics
MaS, possibly adding different remarks: problems
encountered during the visualisation, user's personal tolerance
User Profile Values
to the quality, and her personal interest on the video's topic.
Age 18 to 30 years In this experiment, a total of 45 users are participating in
which 20 are female and 25 are male participants. Most of
Gender Men, women them belong to the age group ranging from 18 to 30 years old.
Familiarity with the We collected 25 * 45 = 1125 samples in our database, which
Rarely, weekly, daily means that we have 1125 different combinations of all settled
video streaming
parameters, associated with a MaS value to each combination.
Interest on the content Interested, not interested However, we reduced this number after a deeper look over on
the dataset, to average repeated lines and try to eliminate
parasite ones. This cleaning was necessary for the next phase,
25 HD and Non-HD video streams are selected for this
which consists on training the learning models and evaluated
experiment, with different motion complexities (high,
their accuracy.
alternating, and low). These videos have the same frame rate
(25 frames per second) and video codec (H.264), and they are
related to different fields of interests (e.g. politics, sports,
VII. RESULTS
news, social life, commercial ads, and animated cartoons). In
our experimental analysis, we used NetEm as a network Initially, datasets resulting from the controlled experiment
emulator to control QoS parameters. This tool has the ability were processed and cleaned from any parasite information.
to emulate the properties of wide area networks. Therefore, we have a dataset that is ready to apply for data
analysis. As an input to our ML tool, we are considering all
nine parameters, which are gender, frequency of viewing,
VI. EXPERIMENT SETUP interest, delay, jitter, loss, conditional loss, motion complexity
and resolution. In order to minimize biases, we perform 4-
The experimental setup consists of three important
fo d-cross-validation to estimate the error rate efficiently,
�
elements: a video streaming server, a video client, and the
USIng the following procedure: a single sub-sample is chosen
Network Emulator (NetEm), that emulates a core and cloud
as testing data, and the remaining 3 sub-samples are used as
network. The traffic flows between the server and the client is
training data. This procedure is repeated 4 times, in which
forwarded via the network emulator. The emulator introduces
each of the 4 sub-samples is used exactly once as the testing
artificial delay, jitter and packet loss within a dedicated
data. All results are averaged and a single estimation is
connection. The client side is built on a Windows
obtained. For the modelling process, we use the six classifying
environment, while the streaming server and the shaper
models for determining which one is best and offers the best Table 3 Average weighted for RF and DT models
model. Recall that these six classifYing models are: Naives
Bayes (NB), 4-Nearest Neighbour (4-NN), Support Vector Model TP FP Precision Recall F-Measure
Machine (SMV), Decision Tree (DT), Random Forest (RF) RF 0.753 0.078 0.752 0.753 0.752
and Neural Network (NNT). DT 0.743 0.084 0.748 0.743 0.745
We use the WEKA tool to run those different algorithms on
the dataset. This tool gives information about the classification We consider five statistical metrics to compare the
model that was generated, along with its performance and performance of DT and RF models, which are: True Positive
imperfection with detailed averaged statistics. We consider the (TP), False Positive (FP), Precision, Recall and F-measure.
mean absolute error rate to compare the error rate between the
different models. The results are illustrated in Fig. 3. In terms 1) TP (True Positive) occurs when a statistical test rejects a
of classification, this figure shows that DT has the minimum true hypothesis. The best value for this measure is I.
absolute error rate, with a value of 0.126, followed by the RF 2) FP (False Positive): a false value means rejecting
model with 0.136. SVM has the highest error rate with 0.26. the hypothesis. Its value should be close to 0, which
The results clearly depicts that the DT model and RF model means the model works well.
are the most reliable models on the current datasets. 3) Precision is the probability when a (randomly selected)
retrieved result is relevant:
0,3 ,------, Precision = TP/ (TP+FP)
0,25+-----
o 4) Recall is the probability when a (randomly selected)
� 02 '
relevant document is retrieved in a search:
]s
.

0,15
Recall = TP/ (TP+FN)
i 0,1
5) F-measure is a measure of a test accuracy, where an
E 0,05
F1 score reaches its best value at 1 and in worst case its
NB SVM 4-NN DT RF NNT value is O.
classifier

F-measure 2 * (Precision * Recall) / (Precision + Recall)

Figure 3 Mean absolute error rate for six classifiers
=

To choose the best model, we also perform an instance The results of a classification can be negative or positive. If
classification test on the six algorithms, in terms of the number the results of the test correspond to reality, then one considers
of correctly classified instances. Fig. 4 shows that two that a correct decision has been made. However, if the result
methods correspond to the best classification: RF with 74.8% of the test does not correspond to reality, then an error has
of correctly classified instances, followed by the DT model occurred. According to these metrics, we conclude in Table 3
with 74% of correctly classified data. The worst model is 4- that RF is slightly more suitable than the DT model for
NN model with 49% of correctly classified instances. These QoS/QoE correlation.
results again clearly demonstrate that the DT and RF models
are the best models, according to our datasets.
VIII. CONCLUSION
80 ,-------, In this paper, we have investigated the correlation between
70 +--------���.r---�
QoS and QoE in the perspective of video streaming services.
60 +---�----���.r-,.��
50 -1-1___-__-- ML classifiers are used to classifY the collected dataset. In
� 40 case of mean absolute error rate, it is observed that DT has a
30 good performance as compared to all other algorithms. An
20
10 instance classification test is also performed to select the best
model, and results clearly show that performance of RF and
NB SVM 4-NN DT RF NNT
classifier DT are approximately at the same level. Finally, to evaluate
the efficiency of DT and RF, a statistical analysis of
Figure 4 Instances classification classification is done, and results show that RF performs
slightly better than DT.
To find more details about the models and their
classification errors, we compare the efficiency of DT and RF
models. The efficiency of these models is evaluated by ACKNOWLEDGMENTS
measuring the statistics analysis data about classification.
This work was funded by the CELTIC European Project
Results are presented in Table 3.
IPNQSIS. We specially thank our Master's Students, Emna
Rekik and Mayssa Iemel, from SupCom School (Tunisia) for
their great participation in this work.
 REFERENCES International Conference on Advances in Mobile Computing and
Multimedia (MoMM '09). ACM.
[I] M. Jarschel, D. Schlosser, S.Scheuring, T. Hossfeld, "An Evaluation of
[10] V.Menkovski, G.Exarchakos, ALiotta" "Machine Learning Approach
QoE in Cloud Gaming Based on Subjective Test",
for Quality of Experience Aware Networks," Intelligent Networking and
Innovative Mobile and Internet Services in Ubiquitous Computing
Collaborative Systems (lNCOS), 2010 2nd International Conference on ,
(lMIS), pp.330-335, June 30 201I-July 2 2011.
vo!., no , pp.461-466, 24-26 Nov. 2010
[2] M.Venkataraman, M.Chatterjee, "Inferring video QoE in real
[II] R.I. ITU-T,910, "Subjective video quality assessment methods for
time," Network, IEEE , vol.25, no. I, pp.4-13, January-February 20II.
multimedia applications, 1999
[3] Lucjan Janowski and Piotr Romaniak, "QoE as a function of frame rate
[12] Book: Antoine Cornuejols-Laurent Miclet. "Apprentissage artificiel:
and resolution changes", In Proceedings of the Third international concepts et algorithms" EYROLLES, 2010.
conference on Future Multimedia Networking (FMN'IO), ACM 2010.
[13] D.K.Krishnappa, S.Khemmarat, M.Zink, "Planet YouTube Global,
[4] John D. McCarthy, M. Angela Sasse, and D.Miras. Sharp or smooth?:
measurement-based performance analysis of viewer;'s experience
comparing the effects of quantization vs. frame rate for streamed video.
watching user generated videos," Local Computer Networks, Annual
In Proceedings of the SIGCHI conference on Human factors in
IEEE Conference on, pp. 948-956, 2011 IEEE 36th Conference on Local
computing systems (CHI '04). ACM.
Computer Networks, 20II.
[5] K.P. Mok, Ricky, EW.W. Chan, R.K.C. Chang, "Measuring the quality
[14] G.Zhang, W. Jin, L. Hu, "Radar emitter signal recognition based
of experience of HTTP video streaming," Integrated Network
on support vector machines," Control, Automation, Robotics and Vision
Management (1M), May 20II.
Conference, (ICARCV), vol.2, no., pp. 826- 831 Vol. 2, 6-9 Dec. 2004.
[6] B French, JLin, T. Phan, AC. Dalal, "Real time video QoE analysis of MJ. Islam, Q.M.l Wu, M. Ahmadi, M.A Sid-Ahmed, "Investigating
[15]
RTMP streams," Perfonnance Computing and Communications
the Performance of Naive- Bayes Classifiers and K- Nearest Neighbor
Conference (IPCCC), 2011 IEEE 30th International , vol., no , pp.I-2,
Classifiers," Convergence Information Technology, International
17-19 Nov. 2011.
Conference, pp.1541-1546, 21-23 Nov. 2007.
[7] T.H Truong, T.H Nguyen, H.T Nguyen, "On Relationship between
[16] M. Pal, P.M. Mather, "A comparison of decision tree and
Quality of Experience and Quality of Service Metrics for IMS-Based
backpropagation neural network classifiers for land use
IPTV Networks," Computing and CommunicationTechnologies,
classification " IEEE International Geoscience and Remote Sensing
Research, Innovation, and Vision for the Future (RIVF), 2012 IEEE
Symposium GARSS), vo!.l, no., pp. 503- 505 vo!.l, 2002.
c'I
[8] F.Agboma and A.Liotta, "QoE-aware QoS management",
[17] W.T. Aung, K..H.M Saw Hla, "Random forest classifier for multi­
In Proceedings of the 6th International Coriference on Advances in
category classification of web pages," IEEE Asia-Pacific Services
Mobile Computing and Multimedia (MoMM '08), ACM 2008.
Computing Conference (APSCC), pp.372-376, 7-11 Dec. 2009.
[9] V. Menkovski, A.Oredope, A.Liotta, and A.Cuadra. "Predicting quality of
experience in multimedia streaming", In Proceedings of the 7th

View publication stats