Wang et al.

EURASIP Journal on Wireless Communications and

Networking (2018) 2018:173
https://doi.org/10.1186/s13638-018-1180-8

RES EARCH Open Access

Data analysis on video streaming QoE

over mobile networks
Qingyong Wang1 , Hong-Ning Dai1* , Di Wu2,3 and Hong Xiao4

Abstract
One of recent proposals on standardizing quality of user experience (QoE) of video streaming over mobile network is
video Mean Opinion Score (vMOS), which can model QoE of video streaming in 5 discrete grades. However, there are
few studies on quantifying vMOS and investigating the relationship between vMOS and other quality of service (QoS)
parameters. In this paper, we address this concern by proposing a novel data analytical framework based on video
streaming QoE data. In particular, our analytical model consists of K-means clustering and logistic regression. This
model integrates the benefits of both these two models. Moreover, we conduct extensive experiments on realistic
dataset and verify the accuracy of our proposed model. The results show that our proposed framework outperforms
other existing methods in terms of prediction accuracy. Moreover, our results also show that vMOS is essentially
affected by many QoS parameters such as initial buffering latency, stalling ratio, and stalling times. Our results offer a
number of insights in improving QoE of video streaming over mobile networks.
Keywords: Quality of service (QoS), Quality of experience (QoE), K-means, Logistics regression, Video streaming,
Mobile networks

1 Introduction expenditure, consequently decreasing the profit of service

Video streaming is becoming one of the most popu- providers [6].
lar services over mobile networks. It is predicted in [1] Therefore, QoE improvement of video streaming over
that the traffic caused by video streaming will occupy mobile networks has received extensive attention recently.
more than 77% of all consumer Internet traffic by 2021, In particular, the work of [7] investigates QoE-driven
among which mobile video traffic will be more than cross-layer optimization for video transmission in wire-
55% of all video traffic. The growing demands on video less networks. Ramamurthi et al. [8] propose a resource
streaming over mobile networks inevitably lead to the management scheme at network core in wireless net-
challenges in optimizing network resource in order to works to improve video QoE. The work of [9] presents
improve the user perceptual experience. Many previ- a large-scale measurement-based study on the effects of
ous studies mainly focus on improving quality of service Internet path selection in video QoE and offers several
(QoS) of video streaming over mobile networks. Typi- QoE enhancement schemes.
cal QoS measures include throughput, bandwidth, outage, However, the prerequisite of QoE improvement of video
jitter, and delay [2]. However, most of these QoS met- streaming is to quantify QoE appropriately. Video QoE
rics fail to characterize user perceptual experience, which assessment schemes can be generally categorized into sub-
is also called quality of experience (QoE). It is more jective tests, objective assessments, and data-driven anal-
crucial to conduct video quality assessment from QoE ysis [3]. Compared with subjective tests and objective
than that from QoS [3, 4] because (i) enhancing QoS assessments, data-driven analysis is more promising due
does not directly improve QoE [5] and (ii) only improv- to the availability of massive datasets and the accuracy
ing QoS sometimes significantly increases operating of characterizing user perception while overcoming the
drawbacks of subjective tests and objective assessments
*Correspondence: [email protected]
1
(such as high cost and insufficient human visual sys-
Faculty of Information Technology, Macau University of Science and
Technology, Wai Long Avenida, Macau SAR, China tem knowledge). In particular, the work of [6] proposes a
Full list of author information is available at the end of the article data-driven model to quantify the metrics affecting video

© The Author(s). 2018 Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the
Creative Commons license, and indicate if changes were made.
Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 2 of 10

QoE. Jiang et al. [10] improve video QoE by exploiting To address the above concerns, we propose a data-
data-driven QoE prediction. The work of [11] improves driven analysis framework to analyze the relationship
the video bitrate adaptation based on data-driven QoE between vMOS and other QoS parameters. Although
prediction. Huang et al. [12] propose a dynamic adap- our previous work [20] presented preliminary results
tive streaming via HTTP to optimize user QoE. Ref. on quantifying vMOS and other QoS parameters, this
[13] presents a Multi-Constraint Quality-of-Experience study is significantly different from our previous work
(QoE) centric Routing (MCQR) scheme to improve user in the following aspects: (1) we conduct a data pre-
video streaming QoE over mobile networks. The work liminary analysis on vMOS data; this analysis has
of [14] presents a queue-based model to analyze the been ignored in our previous work; (2) we propose
video buffer using discrete-time analysis. Marai et al. [15] a novel analytical framework in this paper, which is
propose a client-server cooperation-based approach to significantly different from the previous work; and
achieve efficiency, fairness, and stability of video adaptive (3) experiment results have shown that our proposed
streaming. model can improve the predication accuracy than our
In addition to the above efforts, there are also previous work.
other solutions on standardizing QoE. One of recent In addition, this paper has the following research con-
video QoE measurement standards is U-vMOS tributions in contrast to other existing studies: (i) our
(User/Unified/Ubiquitous video Mean Opinion Score)1 , model consists of K-means clustering approach and logis-
which was proposed by Huawei in 2016 [16]. The score tic regression; the combination of these two approaches
of vMOS is essentially established according to Mean can greatly improve the predication accuracy; (ii) we
Opinion Score (MOS) standardized by International have conducted extensive experiments on three train-
Telecommunication Union (ITU) [17] as shown in ing datasets and one testing dataset and the experiment
Table 1, where discrete grades from 1 to 5 represent bad, results show that our proposed model outperforms other
poor, fair, good, and excellent, respectively. It is shown existing methods in terms of predication accuracy; and
in [16] that vMOS at video playback startup is mainly (iii) moreover, our results also imply that a small set of
determined by three key factors: video quality, initial QoS parameters play an important role in determining
buffering delay, and video freezing duration, each of which vMOS.
is also affected by multiple QoS variables. Recently, Pan The remainder of this paper is organized as follows.
et al. [18] investigate machine learning-based bitrate esti- Section 2 describes the data used in this paper and iden-
mation on YouTube video streaming based on Huawei’s tifies the challenges. We then present the overview of our
vMOS assessment model. However, they just give a math- method in Section 3. Section 4 presents the experimental
ematical expression of vMOS based on their subjective results. Finally, we conclude this paper in Section 6.
estimations. The work of [19] presents using vMOS to
investigate the impacts of several system parameters on 2 Data description
long-term evolution (LTE) networks based on simula- We obtained the realistic datasets from SpeedVideo
tions. To the best of our knowledge, there is no data-driven Global Operating Platform (SVGOP) established by
QoE analysis on vMOS. Huawei2 ; SVGOP is a specific application of vMOS in
Therefore, this paper aims to conduct data-driven QoE mobile networks throughout the world. In particular,
analysis on vMOS. In particular, we obtain a realistic three datasets contain totally 89,266 samples with 11 fea-
dataset on video QoE based on SpeedVideo Global Oper- tures (i.e., QoS parameters) and 1 scoring factor vMOS.
ating Platform (SVGOP) established by Huawei. This Table 2 summarizes the features. As shown in [16], vMOS
dataset has the following unique characteristics: (1) het- is a function of the above QoS parameters. However, to the
erogeneous data types, (2) positive/negative correlations, best of our knowledge, there is no data-driven analysis on
and (3) dependence of features; these characteristics result the relation between vMOS and other QoS parameters.
in the difficulties in analyzing video QoE data. We first conduct a preliminary statistic analysis on the
dataset. In particular, Fig. 1 displays the histogram of
Table 1 Mean Opinion Score (MOS) evaluation scale vMOS in different scales (ranging from 1 to 5), where
the vertical axis represents the frequency of vMOS and
Score Quality Distortion Class
the horizontal axis represents the evaluation scale of
1 Bad Very annoying but objectionable 0
vMOS. We observe from Fig. 1 that most of vMOS val-
2 Poor Annoying, but not objectionable ues concentrate on the range from 3.5 to 4. When plot-
3 Fair Perceptible and slightly annoying 1 ting the normal distribution curve (i.e., the red curve
4 Good Just perceptible, but not annoying
in Fig. 1), we can find that the median is about 4.0,
implying that most of vMOS values are quite close
5 Excellent Imperceptible
to “good”.
Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 3 of 10

Table 2 Description of dataset some small values (i.e., less than 0.3), we also plot a sub-
Types Features graph in Fig. 4 to better present the results. We observe
QoS parameters Average rate of playing phase (kbps) from Fig. 4 that there is a negative correlation between
vMOS and stalling ratio. In other words, the higher the
Video total download (DL) rate (kbps)
stalling ratio is, the lower the vMOS is.
Video bitrate (kbps)
Due to the space limitation, we do not show other cor-
Initial max DL rate (kbps) relation analytical results on other QoS parameters in this
End-to-End (E2E) round-trip time (RTT) (ms) section. In summary, we find that the dataset has the
Initial buffering latency (ms) following characteristics:
Video Initial buffer downloaded (byte) • Heterogeneous data types. The preliminary results
Playing time (ms) show that QoS parameters are in different types and
Playing total duration different ranges. For example, the initial buffering
Stalling times latency is ranging from 500 ms (millisecond) to 30,000
Stalling ratio ms while the average rate of play phase is ranging
from 300 kbps (kilobit per second) to 16,000 kbps.
Scoring factors vMOS
• Positive/negative correlations of QoS parameters. As
shown in the statistics results, we observe that there
are positive or negative correlations between vMOS
and other QoS parameters. For example, there is a
We then investigate the correlations between vMOS and positive correlation between vMOS and video total
other QoS parameters. In particular, Fig. 2 shows a his- DL rate while there is a negative correlation between
togram of vMOS versus video total DL rate. It is shown in vMOS and initial buffering latency.
Fig. 2 that there is a positive correlation between vMOS • Dependence on QoS parameters. The preliminary
and video total DL rate, implying that the higher the video statistics results also show that vMOS is essentially
total DL rate is, the higher the vMOS is. affected by multiple factors, such as average rate of
In order to measure the correlation between vMOS playing phase, video total DL rate, end-to-end (E2E)
and initial buffering latency, we plot vMOS against initial round-trip time, and initial buffering latency.
buffering latency in Fig. 3 where there is a negative cor-
relation between vMOS and initial buffering latency. In The above characteristics result in the difficulties in
other words, vMOS decreases with the increment of initial analyzing video QoE data. To address the above chal-
buffering latency, implying the worse QoE to users. lenges, we propose a novel data-driven QoE analy-
Figure 4 plots vMOS against stalling ratio. Essen- sis framework (which will be described in details in
tially, stalling ratio is defined as Stalling ratio = Section 3).
Stalling duration
Playing total duration according to the white paper of Huawei
[16]. Since most of Stalling ratio values concentrate on 3 QoE analysis framework
In order to address the above concerns, we propose a
novel data-driven QoE analysis method, which consists of
the following procedures: (1) data preprocessing and (2)
10000 data analysis with integration of K-means clustering and
logistic regression. Figure 5 shows the flow chart of our
8000 proposed method.

3.1 Data preprocessing

6000
We need to normalize the features since they are in dif-
Frequency

ferent units. In particular, we make a conversion from

4000 the absolute value to the relative value. In particular, we
choose the MAX-MIN scaling method to normalize the
2000
positive and negative values. More specifically, we have
• Positive values:
0
! "
1 1.5 2 2.5 3 3.5 4 4.5 5 xij − min xij
Score uij = ! " ! ", (1)
max xij − min xij
Fig. 1 vMOS histogram of frequency distribution
• Negative values:
Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 4 of 10

vMOS

0-1 1-2 2-3 3-4 4-5 5-6 6-7 7-8 8-10

103 kbps
Video Total DL Rate
Fig. 2 Histogram of vMOS versus video total DL rate

! "
max xij − xij (binary). Therefore, we first exploit K-means to categorize
uij = ! " ! ", (2)
max xij − min xij the sample datasets into two groups according to “good”
or “bad”. We then use logistic regression to predict the
where xij represents the original value, uij represents the QoE of video streaming.
value after normalization, min(·) is the minimum value,
and max(·) is the maximum value. 3.2.1 K-means clustering
The main idea of K-means algorithm [21] is to find a par-
3.2 K-means + LR method tition such that squared error between the empirical mean
In this paper, we use logistic regression (LR) mainly to pre- of a cluster and the points in the cluster is minimized. In
dict whether a user’s QoE of video streaming is “good” or particular, given a dataset X = {xi }, i = 1, 2, ..., m, we
“bad” since it is hard to determine it based on a contin- partition them into K disjoint clusters so that the sum of
uous vMOS value within [1, 5]. However, logistic regres- the intra-cluster variances is minimized. We denote the K
sion requires that the dependent variable is dichotomou disjoint clusters by C = C1 , C2 , ..., Ck .

4.5

3.5

3
vMOS

2.5

1.5

1
0 0.5 1 1.5 2 2.5 3 3.5
Initial Buffering Latency x 10
4

Fig. 3 vMOS versus initial buffering latency

Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 5 of 10

4.5
Stalling Ratio subgraph
4.5
4
4

3.5 3.5

vMOS
3
vMOS

2.5

2.5 2

1.5
2

1
0 0.05 0.1 0.15 0.2 0.25 0.3
1.5
Stalling Ratio

1
0 5 10 15 20 25 30 35 40 45
Stalling Ratio
Fig. 4 vMOS versus stalling ratio

The goal of K-means is to minimize the within-cluster represent “good” and “0” to represent “bad”. Without loss
sum of variance over all clusters. In other words, we need of generality, we classify a data sample as “bad” when its
to find vMOS score is within [1, 2] and “good” when its vMOS
score is within [3, 5], as shown in Table 1.
K #
# Recall that a dataset contains m samples, D =
argmin ∥xi − µk ∥2 , (3)
C (X1 , y1 ), (X2 , y2 ), ..., (Xm , ym ), where Xi = (xi1 , xi2 , ..., xij ) is
k=1 x∈Ci
the i − th input pattern with dimensionality j and yi is a
where µk is the mean of cluster Ci . corresponding variable that takes a value of 0 or 1. The
This clustering process can be completed by alternating term yi = 0 indicates the i − th sample is bad and yi = 1
between assigning instances to their closest centers and indicates the i − th sample is good. The vector Xi contains
recomputing the centers until a local minimum reaches. j influence features (for all n QoE features) for the i − th
sample and xij denotes the value of feature j for the i − th
3.2.2 Logistic regression sample. We denote the probability of vMOS being “good”
In this paper, we concern with a binary classification prob- by p and by 1 − p of vMOS being “bad”. Then, the logit
lem of categorizing QoE of a video streaming into two transform of probability p is as follow,
cases of “bad” and “good”. To solve this problem, we exploit $ %
logistic regression (LR). In particular, we denote the QoE p
logit(p) = ln = β0 + β1 x1 + β2 x2 + ... + βn xn ,
by a binary-dependent variable. This variable only takes 1−p
two values, either “0” or “1”. Specifically, we use “1” to (4)

Fig. 5 Our method used in this paper

Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 6 of 10

where β0 is the offset and βi (i = 1, ..., n) is the correspond- Table 4 Summary of features
ing regression coefficient for each QoS parameter. Types Features Variables
It is shown in Eq. (4) that the probability of occurrence QoS parameters Average rate of playing phase (kbps) x1
denoted by p(x) can be expressed as a non-linear function
Video total download (DL) rate (kbps) x2
of features,
Video bitrate (kbps) x3
1 1 Initial max DL rate (kbps) x4
p(x) = = ,
1 + e−logit(p(x)) 1 + e−(β0 +β1 x1 +β2 x2 +...+βn xn ) End-to-End (E2E) round-trip time (RTT) (ms) x5
(5) Initial buffering latency (ms) x6
Video Initial buffer downloaded (byte) x7
where p ∈ [0, 1]. Since the logistic regression model is Playing time(ms) x8
non-linear, the maximum likelihood estimation method
Playing total duration x9
can be used to estimate the regression coefficient
βi (i = 0, ..., n). Stalling times x10
Stalling ratio x11
4 Empirical study
As we summarize in Section 2, the dataset has the char-
acteristics such as different types, positive and negative to search the cut-off values. Specifically, we obtain classi-
correlations, and dependence. Therefore, we need to use fication standard values, 3.9, 3.9, and 3.93 for datasets 1, 2,
the proposed method to address these concerns. In par- and 3, respectively. Table 5 shows the K-means clustering
ticular, we describe the experiment settings in Section 4.1. results.
We then show the intermediate results of K-means clus-
tering and logistic regression in Section 4.2. We next 4.2.2 Logistics regression analysis
compare our proposed method with other existing meth- We next conduct logistics regression analysis. In partic-
ods in terms of predication accuracy in Section 4.3. Finally, ular, we give the regression expression on 11 features as
we conduct performance analysis of our proposed method follows,
in Section 4.4. logit(p) = β0 + β1 x1 +β2 x2 +β3 x3 + β4 x4 + β5 x5 + β6 x6
4.1 Experiment settings +β7 x7 + β8 x8 + β9 x9 + β10 x10 + β11 x11 , (6)
We obtain three sample datasets from SpeedVideo Global where xi (i = 1 to 11) corresponds to each of 11 features
Operating Platform of Huawei. Table 3 presents the meta as given in Table 4 and βi is the regression coefficient.
data information of these datasets. In particular, dataset 1, Table 6 lists regression results on the three datasets,
dataset 2, and dataset 3 contain 30,000, 30,000, and 26,984 where “Value” denotes the resulting value of each coeffi-
samples, respectively, each of which has 11 QoS parame- cient, “Wal” denotes the value of the Wald test under the
ters. Table 4 summarizes the 11 features where we denote significant level of “Sig” [22]. We observe from Table 6
each of these 11 features by variable xi (i = 1 to 11). that the lower “Sig” value (or the higher “Wal” value) of a
coefficient indicates the higher impact of that coefficient
4.2 K-means LR method on vMOS. In general, Sig> 0.05, the coefficient is not sta-
4.2.1 K-means Analysis tistically significant, implying that no correlation can be
As shown in Section 3, we exploit the proposed K-means found between the QoS parameter and vMOS. In prac-
logistic regression method to predict QoE. In particular, tice, there is no correlation between a QoS parameter and
we classify each dataset into two classes: one class con- vMOS when Sig < 0.05 according to the Wald test [23].
tains samples with QoE being “good” and another class
contains samples with QoE being “bad”. The experimental
results show that these two classes are fairly unbalanced Table 5 K-means clustering analysis
in terms of class size. Thus, we choose K-means scheme Dataset Classification standard QoE No. of samples
Dataset 1 3.90 Bad 10,306
Good 19,694
Table 3 Meta data information
Description No. of samples No. of features Dataset 2 3.90 Bad 19,833

Dataset 1 30,000 11 Good 10,167

Dataset 2 30,000 11 Dataset 3 3.93 Bad 9,199

Dataset 3 26,984 11 Good 17,785
Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 7 of 10

Table 6 Regression coefficients

Coefficient Dataset 1 Dataset 2 Dataset 3
Value Wal Sig Value Wal Sig Value Wal Sig
β1 34.43 0.005 0.943 − 15.51 0.272 0.602 − 11.094 0.214 0.643
β2 − 12.65 0.001 0.98 9.88 0.103 0.748 12.21 0.245 0.621
β3 105.18 7.374 0.007 1.721 38.976 0 2.263 75.108 0

β4 − 18.403 0.27 0.599 − 0.632 0.129 0.72 − 0.736 0.152 0.697

β5 15.89 0.41 0.522 0.313 0.074 0.785 0.943 0.585 0.444
β6 112529 7.47 0.006 2156.26 404.857 0 2598.501 408.67 0
β7 − 14.49 0.013 0.909 − 4.778 0.312 0.576 4.1 0.454 0.501

β8 211762 2.37 0.123 3612.752 6.299 0.012 4204.75 5.961 0.015

β9 − 3252.45 2.359 0.125 − 56.298 6.409 0.011 − 64.055 5.813 0.016
β10 − 1911.44 0.265 0.607 19.609 0.062 0.803 − 45.572 4.02 0.045
β11 − 207352 2.369 0.124 − 3553.314 6.368 0.012 − 4158.129 4.132 0.042

It is shown in Table 6 that coefficient β6 (corresponding 4.4.2 Accuracy analysis

to initial buffering latency) has the most significant impact In order to assess the accuracy of prediction, we use
on vMOS. Similarly, we can find that coefficient β3 (cor- Accuracy, which is defined as follows,
responding to video bitrate) also has a strong influence on
vMOS because of high Wal value (i.e., 7.374). (TP + TN)
Accuracy = , (7)
(TP + FP + TN + FN)
4.3 Performance comparison
We next present the experimental results over the given where TP is true positive, FP is false positive, FN is false
datasets with performance comparison with conven- negative, and TN is true negative.
tional traditional methods including multivariate linear We obtain another dataset from SVGOP. This dataset
regression, logistic regression, support vector machines, contains 2517 samples. This new dataset serves as the
K-nearest-neighbor and Naive Bayes [24]. testing dataset. We next evaluate the accuracy of our
We then evaluate the performance of our proposed proposed model over two datasets: training dataset (i.e.,
method and other traditional methods. Table 7 presents dataset 1) and testing dataset (i.e., the dataset contain-
performance comparison of our proposed K-means + LR ing 2517 samples). Figure 7 gives the histogram of the
method with other methods, where we choose the pre- accuracy values of 11 QoS parameters over two afore-
cision as the performance metrics (the higher precision mentioned datasets. We observe from Fig. 7 that training
implying the better performance). Among all the meth- dataset has the accuracy value close to that of testing
ods, our proposed K-means + LR method has the high- dataset in most of QoS parameters while there are some
est performance with precision 96.94, 97.13, and 97.54% gaps between several QoS parameters (e.g., playing total
on dataset 1, dataset 2, and dataset 3, respectively. This duration); the performance gaps may owe to the fact of
performance improvement may lie in the integration of the small sample size of testing dataset (2517 samples ver-
K-means method and logistic regression method. sus 30,000 samples in training dataset). Note that we have
similar findings in dataset 2 and dataset 3. Without repe-
4.4 Performance analysis of our method tition, we do not show the results on dataset 2 and dataset
4.4.1 Residual plots 3 here.
We first use the residual error plots to evaluate the accu- We next use the deviance to evaluate the measure of the
racy of the proposed model. In particular, we define fitness of the data. In particular, the deviance denoted by
Error = Prediction − Original, where Prediction is the D value is calculated by comparing a given model with the
prediction value and Original is the given value. Figure 6 saturated model [24]. We define D value = |prediction −
shows the residual plots over the three datasets, in which original|, where prediction and original denote the predic-
blue points denote the “good” QoE values and blue points tion value and the original value, respectively. Table 8 gives
denote the “bad” QoE values. It is shown in Fig. 6 that the deviance results over the three datasets and the cor-
there are more red points than blue points, implying that responding rankings (sorting D value in ascending order).
our model has a bias of regarding “good” QoE values We observe from Table 8 that initial buffering latency,
as “’bad”. stalling times, and stalling ratio have the smaller D values
Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 8 of 10

Table 7 Performance comparison way toward improving vMOS in mobile networks. We

Method Precision summarize several possible future directions as follows.
Dataset 1 Dataset 2 Dataset 3
• Concentrating on optimizing several dominating
Multivariate linear regression 72.55% 72.51% 71.51%
QoS parameters in vMOS. For example, we may
Logistics regression 74.93% 77.55% 83.55%
focus on optimizing the network resource to reduce
SVM 82.56% 87.17% 85.42% initial buffering latency, stalling times, and stalling
K-nearest-neighbor 52.48% 61.34% 56.77% ratio so that we can significantly improve the video
Naive Bayes 88.12% 83.47% 91.09% QoE while maintaining relatively low operating
K-means + LR (proposed) 96.94% 97.13% 97.54% expenditure. However, it is not an easy task to
Data in bold is the proposed method that outperforms other existing methods
achieve this goal because the enhancement of these
QoS parameters is also involved with many other
than other QoS parameters, implying that they essen- technologies, such as cross-layer optimization and
tially have the dominant influence on vMOS. This result distributed resource allocation [10, 25, 26].
is consistent with the previous finding in [20]. • Identifying QoS bottlenecks. Determining QoS
In summary, we can see from Table 8 that initial buffer- bottlenecks can help to enhance system performance
ing latency, stalling times, and stalling ratio can signifi- and consequently improving vMOS. However, it is
cantly affect vMOS. In particular, initial buffering latency, also difficult to identify the QoS bottlenecks since
stalling times, and stalling ratio are negatively correlated they are often affected by many factors. For example,
with vMOS. We have the following major findings: (1) video stalling is essentially caused by many factors,
vMOS is affected by multiple QoS parameters together. such as network congestion, network failure, device
Essentially, vMOS is affected by 11 QoS parameters as mobility, and radio spectrum scarcity. There is a
given in Table 4. (2) Small set of QoS parameters domi- challenge in identifying the causality of stalling. In the
nates the performance of vMOS. Interestingly, we observe future, we may apply data-driven approach to identify
that a small set of QoS parameters has the stronger influ- the reason behind video stalling according to
ence on vMOS than other QoS parameters. different scenarios.
• Distributing the videos appropriately to improve
5 Discussion and future work vMOS. For example, we can distribute the most
The analytical results essentially offer us some useful popular videos at the servers close to users so that we
insights in improving video QoE, which may pave the can significantly reduce the initial buffering latency.

a
1
Error

−1
500 1000 1500 2000 2500
Testing Sample(Dataset 1)
b
1
Error

−1
500 1000 1500 2000 2500
Testing Sample(Dataset 2)
c
1
Error

−1
500 1000 1500 2000 2500
Testing Sample(Dataset 3)
Fig. 6 Residual error plot. a Dataset 1. b Dataset 2. c Dataset 3
Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 9 of 10

Fig. 7 Training samples versus testing sample

However, to determine the popularity of video outperforms other existing methods in terms of predic-
streaming is challenging since it requires the extensive tion accuracy. For example, our proposed method has
efforts in analyzing the massive video data [27]. the precision of 96.94, 97.13, and 97.54% on dataset 1,
dataset 2, and dataset 3. Our results also show that a small
6 Conclusions set of QoS parameters play an important role in deter-
In this paper, we propose a novel data analysis model mining vMOS; this implies that we can concentrate on
to analyze video Mean Opinion Score (vMOS), which enhancing these key QoS parameters. It can be achieved
is an important measure of user quality of experience by integrating cross-layer optimization and distributed
of video streaming. In particular, our proposed model is resource allocation schemes together and mitigating QoS
a combination of K-mean clustering method and logis- bottlenecks.
tic regression method, which can essentially improve the Our model has a broad range of applications. For exam-
prediction accuracy than other existing methods. We con- ple, it can be used to enhance the QoE of video service
duct experiments over several realistic datasets. Exten- providers (such as Netflix and YouTube), video-centric
sive experiment results show that our proposed method mobile applications (including Facebook LIVE, Instagram

Table 8 D value and ranking

Features Dataset 1 Dataset 2 Dataset 3
D value Ranking D value Ranking D value Ranking
Average rate of playing phase 0.0605 7 0.0662 7 0.0667 6
Video total DL rate 0.0596 8 0.0651 8 0.0653 8
Video bitrate 0.1848 3 0.1620 3 0.2160 3
Initial max DL rate 0.3008 2 0.2997 2 0.3155 2
E2E RTT 0.5409 1 0.5186 1 0.5602 1
Initial buffering latency 0.0222 11 0.0222 11 0.0199 11
Video initial buffer downloaded 0.1020 4 0.1025 4 0.1033 4
Playing time − 0.0668 6 − 0.0667 6 − 0.0657 7
Playing total duration − 0.0814 5 − 0.0741 5 − 0.0695 5
Stalling times − 0.0231 10 − 0.0229 10 − 0.0279 9
Stalling ratio − 0.0232 9 − 0.0231 9 − 0.0224 10
Wang et al. EURASIP Journal on Wireless Communications and Networking (2018) 2018:173 Page 10 of 10

LIVE, Snapchat, etc.), video game live streaming services 4. H Nam, KH Kim, H Schulzrinne, in IEEE INFOCOM. QoE matters more than
(such as Twitch, Hitbox and NetEase Game Lives). More- QoS: Why people stop watching cat videos (IEEE, New Jersey, 2016),
pp. 1–9
over, it can be used to improve the usability of video 5. V Joseph, G de Veciana, in IEEE INFOCOM. NOVA: QoE-driven optimization
surveillance systems. For example, the quality of video of DASH-based video delivery in networks (IEEE, New Jersey, 2014),
streaming of video surveillance systems can be helpful in pp. 82–90
6. A Balachandran, V Sekar, A Akella, S Seshan, I Stoica, H Zhang, in
detecting dangers in advance. Proceedings of the ACM SIGCOMM Conference. Developing a Predictive
Model of Quality of Experience for Internet Video (ACM, New York, 2013)
Endnotes 7. S Thakolsri, W Kellerer, E Steinbach, in Proceedings of IEEE International
Conference on Communications (ICC). QoE-Based Cross-Layer Optimization
1
For simplicity, we use vMOS to represent U-vMOS of Wireless Video with Unperceivable Temporal Video Quality Fluctuation
throughout this paper. (IEEE, New Jersey, 2011)
8. V Ramamurthi, O Oyman, J Foerster, in IEEE Global Communications
2
http://speedvideo.huawei.com/ Conference (GLOBECOM). Video-QoE aware resource management at
network core (IEEE, New Jersey, 2014), pp. 1418–1423
Abbreviations 9. M Venkataraman, M Chatterjee, Effects of internet path selection on
DL: Download; E2E: End-to-end; LR: Logistic regression; QoS: Quality of service; video-QoE: analysis and improvements. IEEE/ACM Trans. Networking
QoE: Quality of experience; Sig: Significance; SVGOP: SpeedVideo global (TON). 22(3), 689–702 (2014)
operating platform; SVM: Support vector machine; vMOS: video mean opinion 10. J Jiang, V Sekar, H Milner, D Shepherd, I Stoica, H Zhang, in 13th USENIX
score; Wal: Wald test value Symposium on Networked Systems Design and Implementation (NSDI). CFA:
A Practical Prediction System for Video QoE Optimization, (Berkeley,
Acknowledgements 2016), pp. 137–150
The authors would like to express their appreciation for Gordon K.-T. Hon for 11. Y Sun, X Yin, J Jiang, V Sekar, F Lin, N Wang, et al, in Proceedings of ACM
his thoughtful discussions. SIGCOMM. CS2P: Improving video bitrate selection and adaptation with
data-driven throughput prediction (ACM, New York, 2016), pp. 272–285
12. W Huang, Y Zhou, Xie X, D Wu, M Chen, E Ngai, Buffer State is Enough:
Funding Simplifying the Design of QoE-Aware HTTP Adaptive Video Streaming.
The work described in this paper was partially supported by Macao Science IEEE Trans. Broadcast. 64(2), 1–12 (2018)
and Technology Development Fund under Grant No. 0026/2018/A1, the 13. C Lal, V Laxmi, MS Gaur, M Conti, Enhancing QoE for video streaming in
National Key R&D Program of China under Grant No. 2016YFB0201900, the MANETs via multi-constraint routing. Wirel. Netw. 24(1), 235–256 (2018)
National Natural Science Foundation of China (NSFC) under Grant No. 14. V Burger, T Zinner, L Dinh-Xuan, F Wamser, P Tran-Gia, A Generic
61572538 and Grant No. 61672170, the Fundamental Research Funds for the Approach to Video Buffer Modeling Using Discrete-Time Analysis. ACM
Central Universities under Grant No. 17LGJC23, the NSFC-Guangdong Joint Trans Multimedia Comput Commun Appl. 14(2s), 33:1–33:23 (2018)
Fund under Grant No. U1401251 and Guangdong Science and Technology 15. OE Marai, T Taleb, M Menacer, M Koudil, On Improving Video Streaming
Plan with Grant No. 2015B090923004. All the funding bodies are involved with Efficiency, Fairness, Stability, and Convergence Time Through
the design of the study and collection, analysis, and interpretation of data and Client-Server Cooperation. IEEE Trans. Broadcast. 64(1), 11–25 (2018)
in writing the manuscript. 16. Huawei, “Requirements of Mobile Bearer Network for Mobile Video
Service”. White Paper, 1–8 (2016)
Authors’ contributions 17. International Telecommunication Union (ITU), “Subjective video quality
QW proposed the idea, derived the results, and wrote the paper. H-ND assessment methods for multimedia applications”. ITU Recommendation:
supervised the work and revised versions. DW gave valuable suggestions on 1–37
the motivation of conducting data analysis on video streaming over mobile 18. W Pan, G Cheng, H Wu, Y Tang, in IEEE/ACM 24th International Symposium
networks. HX contributed to motivating, revising and proofreading of the on Quality of Service (IWQoS). Towards QoE assessment of encrypted
article. All authors read and approved the final manuscript. YouTube adaptive video streaming in mobile networks (IEEE, New Jersey,
2016)
Competing interests 19. CM Lentisco, L Bellido, QJCCE Pastor, JLA H, QoE-Based Analysis of DASH
The authors declare that they have no competing interests. Streaming Parameters Over Mobile Broadcast Networks. IEEE Access. 5,
20684–20694 (2017)
Publisher’s Note 20. Q Wang, HN Dai, H Wang, D Wu, in The 15th IEEE International Symposium
Springer Nature remains neutral with regard to jurisdictional claims in on Parallel and Distributed Processing with Applications. Data-Driven QoE
published maps and institutional affiliations. Analysis on Video Streaming in Mobile Networks (IEEE, New Jersey, 2017)
21. X Wu, V Kumar, JR Quinlan, J Ghosh, Q Yang, H Motoda, et al, Top 10
Author details algorithms in data mining. Knowl. Inf. Syst. 14(1), 1–37 (2008)
1 Faculty of Information Technology, Macau University of Science and
22. J Friedman, T Hastie, R Tibshirani, The elements of statistical learning. vol.
Technology, Wai Long Avenida, Macau SAR, China. 2 School of Data and 1. Springer series in statistics New York (2001)
Computer Science, Sun Yat-sen University, Guangzhou 510006, China. 23. RR Wilcox, Introduction to robust estimation and hypothesis testing,
3 Guangdong Province Key Laboratory of Big Data Analysis and Processing,
Academic press. (Elsevier, MA, USA, 2011)
Guangzhou 510006, China. 4 Faculty of Computer, Guangdong University of 24. E Alpaydin, Introduction to machine learning. (MIT press, MA, USA, 2014)
Technology, No. 100 Waihuan Xi Road, Guangzhou 510006, China. 25. Y Wang, X Lin, User-provided networking for QoE provisioning in mobile
networks. IEEE Wirel. Commun. 22(4), 26–33 (2015)
Received: 6 November 2017 Accepted: 18 June 2018 26. Z Su, Q Xu, Q Qi, Big data in mobile social networks: a QoE-oriented
framework. IEEE Netw. 30(1), 52–57 (2016)
27. Z Su, Q Xu, F Hou, Q Yang, Q Qi, Edge Caching for Layered Video Contents
in Mobile Social Networks. IEEE Trans. Multimed. 19(10), 2210–2221 (2017)
References
1. Cisco, Visual Network Index. “forecast and methodology, 2015-2020”.
White Paper, 1–41 (2016)
2. M Venkataraman, M Chatterjee, Inferring video QoE in real time. IEEE
Netw. 25(1), 4–13 (2011)
3. Y Chen, K Wu, Q Zhang, From QoS to QoE: A tutorial on video quality
assessment. IEEE Commun. Surv. Tutorials. 17(2), 1126–1165 (2015)