Quality of Experience Engineering for Customer Added Value Services: From Evaluation to Monitoring

Preface

It will be fascinating to look back in the years ahead and note the convergence of two formerly separate technologies, namely telecom technology and information technology. The former one originates from the telephone world and is based on dedicated architectures with circuit-switched point-to-point connections designed for real-time services. The latter comes from computer communication with flexible architectures and packet-based communication. Due to the emergence of different kinds of communication and networking technologies and the foreseen proliferation of different and specific types of services supported by these technologies, both are merged in a mixture of dedicated and flexible architectures, clearly targeting the use of the Internet Protocol suite as the basis communication protocol. Nevertheless, it appears recently that traditional network control strategies developed in the last decade are not sufficient to handle the complexity and diversity of information in use that we see today. Over the years, the continuous technological evolution and the development of new applications and services have steered networking research toward new problems, which have emerged as the network evolves with new features toward what is usually referred to as the future internet, which has become one of the basic infrastructures that supports the world economy nowadays.

In fact, there is a strong need to build a new network scenario, where networked computer devices would proliferate rapidly, supporting new types of services, usages and applications: from wireless sensor networks and new optical network technologies to cloud computing, high-end mobile devices supporting highdefinition media, high-performance computers, peer-to-peer networks and various platforms and applications.

The overall challenge here is to find scalable and sustainable solutions for the ever-growing smart communications field, which supports different kinds of services, for a wide variety of future next-generation network applications.

To address this challenge, new and cross-disciplinary approaches are required to optimize the whole treatment chain of network application. The building of new mechanisms should be based on both estimates of expected demand and the final consumer demands on perceived quality. Also, statistical methods for online estimation of consumer demands will be crucial. Many of these issues are complex and can, in the short term, hardly be solved by any single approach. Researchers need to find solutions to deliver network services in the most efficient way to provide users with the best perception while taking into consideration scarce network resources.

Over the years, network operators have been assessing network performance based only on the traditional Quality of Service (QoS) parameters such as throughput, delay, jitter and loss rate. These QoS metrics take into account the network components' characteristics. Nevertheless, these measurements are not sufficient now and must be completed by user perception, such as emotional, intensity or satisfaction, which is a natural way to interact with the real world, and will serve as a powerful metaphor to interact with online services and perform affective computing paradigm.

After a pioneering book published in Autumn 2013 dedicated to this new paradigm applied to content delivery network, entitled Quality of Experience for multimedia: application to content delivery network architecture, this second book, edited with my colleague Antonio Cuadra-Sanchez, follows the same direction. It focuses on the current state-of-the-art research results and experience reports in the area of quality monitoring for customer experience management, addressing, among others, currently important topics such as Service-aware Future Internet architecture for Quality of Experience (QoE) management on multimedia applications. In particular, it addresses the QoE paradigm for improving customer perception when using Added Value Services offered by service providers, from evaluation to monitoring and other management processes.

This book shows that QoE is a very dynamic area in terms of theory and application. The continuous emergence of new services along with the increasing competition is forcing network operators and service providers to focus all their effort on customer satisfaction, although determining the QoE is not a trivial task. The field of QoE has been growing rapidly, producing a wide variety of mechanisms for different applications.

I am certain that Research and Development investment in QoE will result in added value for network operators and service providers to anticipate future user needs and to adapt continuously to the new global challenges they face.

This book is a start, but also leaves many questions unanswered. I hope that it will inspire a new generation of investigators and investigations.

Abdelhamid MELLOUK

May 2014

1

Challenges for Quality of Experience Engineering for Added Value Services

Abdelhamid MELLOUK and Antonio CUADRA-SANCHEZ.

1.1. Introduction and challenges

In recent years, multimedia applications and services have experienced a sudden growth. Today, the video display is not limited to the traditional areas of movies and television on TV sets, but accesses these applications in different environments, devices and under different conditions.

In addition, the continuous emergence of new services along with increasing competition is forcing network operators and service providers to focus all their effort onto customer satisfaction, although determining the Quality of Experience (QoE) is not a trivial task.

Due to the emergence of different kinds of communication and networking technologies (core networks with intra-domain and inter-domain challenges, access networks, aggregation networks, spontaneous networks, Internet of Things, etc.) and the current and envisaged proliferation of different and specific types of services supported by these technologies (real-time services, IPTV, VoD, social networking, E-Health, multimedia, gaming, smart cities, etc.), traditional network control strategies are not sufficient to handle the complexity and diversity of information in use that we see today. There is a strong need to develop a new paradigm to obtain the continuity of network services based on the new concept of smart communications and user interaction. The user’s perception, such as emotions, intensity or satisfaction, is a natural way to interact with the real world, and will serve as a powerful metaphor to interact with online services, and performs affective computing paradigm. On the other hand, the use of artificial intelligence tools together with biologically inspired techniques is needed to control network behavior in real-time so as to provide users with the quality of service that they request, and to improve network robustness and resilience based on continuous user feedback.

The key idea of this book is to present a new paradigm driven by user perception and based on control theory and machine learning techniques in order to support smart communications to avoid any complete interruption in the whole chain of service treatment. The main goal is to present state-of-the-art research results and experience reports in the area of Quality Monitoring for customer experience management, addressing, amongst others, currently important topics such as Service-aware Future Internet architecture for Quality of Experience (QoE) management on multimedia applications with respect to the following steps:

– to develop, at the theoretical level, an appropriate unified formal model based on bio-inspired modeling and knowledge distribution in order to construct a scalable and robust environment with low complexity for large-scale dynamic networks;

– to study, at the empirical level, why and how user perception can be quantified, analyzed and influenced;

– to create, at the conceptual level, a general framework for protocol stacks dedicated to smart communications featuring multiple technologies, devices and users,

– finally, to build, at the engineering level, the appropriate model of programming abstractions and software architecture to develop a full-scale framework.

This book addresses the QoE for improving customer perception when using Added Value Services offered by service providers, from evaluation to monitoring and other management processes.

1.2. Contents

This section summarizes the content of the chapters that are gathered in this book.

1.2.1 Chapter 2: An ecosystem for customer experience management

In this chapter, the authors describe an ecosystem that allows us to manage customer experience in order to guarantee the quality levels delivered to end-users, which has been defined into the Eureka Celtic internet protocol network for quality of service intelligent support (IPNQSIS) project and is being adapted for over-the- top (OTT) services inside the Eureka Celtic next generation over-the-top multimedia service (NOTTS) project. The QoE ecosystem lies on a customer experience architecture formed by data acquisition level, monitoring level and control level. The work proposed in this chapter will settle the basis of next generation Customer Experience Management Systems (CEMS).

The authors present an overview of the CEMS under development within the IPNQSIS and NOTTS projects. On the one hand, a generic overall CEMS architecture is introduced and, on the other hand, it has been specialized for the IPNQSIS and NOTTS scopes, reinforcing specific areas such as network monitoring, as well as having IP television (IPTV) as a main application use case.

The IPNQSIS project, in which 18 companies and institutions from Spain, France, Sweden and Finland have collaborated, developed next generation management architectures to improve the QoE-driven network management. This project ended in April 2013 and its main objectives were accomplished, from the definition of a general Customer Experience Management (CEM) architecture to IPNQSIS prototypes focused on IPTV multimedia services. The results of the project consisted of Quality of Service (QoS) measuring tools, mechanisms to quantify the QoE, its correlation with the QoS parameters, and their influence on QoE. The outcome of the analysis can be applied to the integrated management of network resources to improve the user’s experience. This technology is also going to make it possible to develop tools to enable greater correlation between the quality of the service and the actual experience of the user, thereby ensuring greater customer satisfaction.

Future research in this area will extend its scope to next generation services such as OTT services, specifically the NOTTS project, which is continuing the activities regarding QoE management on a task dedicated to this purpose.

1.2.2 Chapter 3: Measuring MPEG frame loss rate to evaluate the QoE in IPTV services

The chapter describes a model to predict the QoE that is a function of the loss of the different types of moving pictures experts group (MPEG) frames, providing a mean opinion score of the delivered service. The authors have implemented this model in a network monitoring tool, which has been validated in both Intel and ARM platforms.

An empirical evaluation of the computational cost of both MPEG frame loss and packet loss ratio (PLR) measurement algorithms has been done with a desktop personal computer (PC) and a low-cost device, providing interesting results to decide which one is better to be used as a network probe. The system running on a PC can be used to measure at the core or access network, whereas the low-cost device can be used at the user’s premises.

The final results show that this model is able to better predict the QoE of such video services than just using the packet loss rate. Based on these results, they have defined a method to measure the QoE by capturing the live video channels, inspecting the packets to detect losses and applying the measured parameters with the obtained model. The authors have implemented a prototype and have also measured its performance, testing its feasibility in both personal computer (PC) and low-cost probes.

As future work to improve the QoE estimation model, the authors will investigate how the amount of movement also influences the perceived QoE. Also, it is interesting to consider how well the obtained model fits with an experiment using a panel of users to get a subjective evaluation of the videos watched.

1.2.3 Chapter 4: Estimating the effect of context on the QoE of audiovisual services

In order to estimate the effect of context on QoE of audiovisual services, in this work, the authors compared the results of formal subjective audiovisual assessment with more informal assessments performed in actual usage contexts (in this case two public exhibition halls). They observed significant differences in the results, both in terms of the mean opinion score (MOS) values and on the impact of the different quality-affecting factors. Interestingly the results show that the subjects in public places were less tolerant to the quality degradations than the subjects in the laboratory. Specifically, tests separating the effects of contextual factors on the basis of 1) voting behavior and 2) actual experience should be conducted.

In this way, in this chapter, the authors compared the results of a laboratory- based audiovisual assessment campaign with that of two separate (and smaller scale) campaigns carried out in public places, in a completely different context. Besides the explicit goal of comparing the results of subjective assessments in a lab versus non-lab environment, this work provides a first step into developing context-specific bias functions to easily and cheaply adapt quality models, typically trained on laboratory-based data, to new contexts of use. These experiments are the first in a series of experiments with the purpose of understanding the effect the context of use has on QoE.

The authors also demonstrated the viability and limitations of an audiovisual model trained on the laboratory-obtained data, when used in a different context, namely in crowded public places. The performance of the model in the exhibition context was inferior to the performance in the laboratory context. However, the estimations could still provide usable values for quality monitoring purposes, e.g. in public displays.

In addition, the authors are currently working on a model calibration method that uses information derived from lightweight user tests performed in specific context. The idea is to test and model the effects of the dominating influence factors in order to formulate a context specific correction function. To this end, and in order to understand different contexts of use and devices generally, user tests outside the laboratory shall be continued.

1.2.4 Chapter 5: IPTV multiservice QoE management system

In this chapter, the authors justify the claim that suitable solutions to determine the quality of a video sequence based on multiservice probes are needed. This chapter deals with an IPTV multiservice QoE management system developed as part of the SAVAGE project.

The SAVAGE project aims to design and develop an advanced system for IPTV multiservice quality management. This system consists of a multiservice advanced probe embedded in a monitoring platform that can operate automatically and remotely, allowing information on the QoE perceived by the end users. The SAVAGE project started in July 2011 and ended in December 2013; they are now implementing the integration of QoE algorithms inside the multiservice probes.

This chapter describes the concepts and proposals related to the objective assessment of the quality of audiovisual services, which consist of the automatic estimation of the QoE perceived by users. First, the state-of-the-art in relation to multimedia quality metrics and, second, the authors describe the multiservice IPTV probe to be developed during the project. In addition, they present the global QoE management system.

The proposed system is intended to deal with the most common problems on these networks, which are, especially, packet losses and network delays that can cause important degradations of video quality, such as blocking effects (i.e. macroblocking), freezing the video and audio losses. In addition, other impairments could affect the quality perceived by the end users, as coding artifacts or capture distortions. However, in real video delivery systems these degradations are less dramatic since an acceptable quality of service under normal conditions should usually be guaranteed.

Furthermore, the assessment of the video quality would facilitate the work of planning and designing distribution networks, and could allow video distributors to implement user fees based on the final quality that they can enjoy in their homes. In addition, an interesting application is real-time monitoring of the video quality perceived by the end user of delivery networks.

Further activities of this project will both consolidate the management architecture and implement the multiservice probe in terms of a prototype to measure the QoS in IPTV platforms.

1.2.5 Chapter 6: High speed multimedia flow classification

This chapter presents a system that unifies the entire process involved in flow classification at high speed. It captures the traffic, builds flows from the received packets and, finally, classifies them inside a graphics processing unit (GPU), at 10 Gbps using commodity hardware.

The authors propose a technique to speed up Deep Packet Inspection (DPI) processing for multimedia protocols using GPUs, and the methodology to integrate it inside a network probe. This shows that DPI with deterministic finite automata (DFA) can be used at very high speed with practically no system overhead. However, other problems apart from high speed traffic classification arise: it is difficult to obtain real high-speed traffic (10 Gbps and over) and build the flows on the fly, which in other context (e.g. below 1 Gbps) could be seen as an obvious thing. The GPU modules process up to 29.7 Gbps, which means about 14.5 mega flows per second.

The tests show how important signatures are when using DPI for flow classification. Signatures define the accuracy of protocol classification. The accuracy of each signature and how it influences false positives and false negatives should be studied during the process of signature creation. An example is the Real-time Transport Protocol (RTP).

Finally, the authors point out that the proposed system has a wide variety of possibilities and configurations, allowing its use in other types of classification, such as hyper text transfer protocol (HTTP), peer-to-peer (P2P) or other non-multimedia protocols. In addition, the high configurability allows us to vary the latency and throughput according to the needs of a given network.

The results show that the achieved performance is very much influenced by the number of protocols to find, and it is limited by the number of network flows. In any case, the system reaches up to 29.7 Gbps (about 14.5 mega flows per second).

1.2.6 Chapter 7: User driven server selection algorithm for CDN architecture

This chapter presents a new routing algorithm based on QoE for Content Distribution Network (CDN) architecture. Theoretically, CDN architecture has two main layers: the routing layer and the metarouting layer. The latter is composed of several modules such as server placement, cache organization and server selection. The first two modules belong to the preparation related phase. More precisely, the server placement module tries to place the replica servers in an optimized way to minimize the delivery delay and the bandwidth consumption. Providers use the cache organization module to organize the content stored in replica servers in order to guarantee the availability, freshness and reliability of content.

Besides these two preparation-related modules, authors focus on the server selection module, which plays an important role in launching the operation phase of a CDN. The fundamental objective of the server’s selection is obviously to offer better performance than the origin server. Another added value of this selection process is lowering the costs of network resources. It is not easy to choose an appropriate server to provide service to users. The appropriate server may be neither the closest one in terms of hop-count or end-to-end delay, nor the least loaded server. The best server is the one that makes end user satisfied when using the provided service. So, the server selection process plays a key role in the decisive success of a CDN.

The chapter includes some related research on server selection methods in the context of CDN. Subsequently, the authors explain their motivation to develop a server selection scheme based on machine learning approaches called multi-armed bandits.

1.2.7 Chapter 8: QoE approaches for adaptive transport of video streaming media

In this chapter, authors discuss the different transport approaches for adaptive video streaming media, and how they influence the QoE. These approaches are solely based on the HTTP protocol, and are specially designed for video transportation over the Internet to support the wide range of devices and maximize end user’s perceived quality. The leading groups and companies, e.g. Microsoft, Apple, Adobe and MPEG/3GPP, have introduced their own standard approaches to facilitate the on demand or live adaptive video streaming transport over HTTP.

The main goals of adaptive video streaming are to improve and optimize user’s QoE by changing the video quality according to network parameters, end user’s device properties and other characteristics. There are five main quality metrics of video streaming that affect the user’s engagement during video watching, and influence user’s QoE. The adaptive video streaming approaches use transmission control protocol (TCP) as a transport protocol. Based on network conditions, TCP parameters provide the client with vital information, and streaming is managed by the end user.

1.2.8 Chapter 9: QoS and QoE effects of packet losses in multimedia video streaming

This chapter analyzes the effect in a multimedia video streaming, i.e. peer-to- peer Television (P2PTV), of a common traffic metric, i.e. packet losses, on the quality parameters, i.e. QoS and QoE. Traditionally, QoS has been used to asses and guarantee the compliance of the deployed Service Level Agreements (SLAs). However, most of the network performance metrics used to estimate the QoS, are only limited to certain aspects of traffic without considering the end user’s subjective perception.

In this context, with the increasing presence of multimedia traffic, the user’s perception (QoE) of networked (multimedia) services has become a major concern for content providers and network operators. While a plethora of works propose solutions for QoS and QoE, authors put the focus, in this chapter, on the relationship between a usual traffic metric and the QoS and QoE assessment.

8 Quality of Experience Engineering for Customer Added Value Services

1.2.9 Chapter 10: A model for QoE estimation based on QoS parameter monitoring for multimedia convergent services (3-play)

In this chapter, the authors present a model for the estimation of user perceived quality, or QoE, from network and/or service performance and/or quality parameters (QoS) in multimedia services, and specifically in triple-play (3P) services: television (TV), telephony and data services, managed and offered by a single operator as a single package. In particular, it focuses on 3P convergent services (deployed over a common, IP-based transport network), and the relationship between the quality perceived by the users of such services and the performance parameters of the underlying network. Specifically, it contributes to the on-line estimation of such quality (i.e. during service delivery, in real or near-real time).

This way the chapter presents a model for the estimation of quality as perceived by the users (i.e. the user QoE) in 3P services. The model is based on a matrix framework defined in terms of user types, service components and user perceptions on the user side, and agents, agent capabilities, and performance indicators on the network side. A quality evaluation process, based on several layers of evaluation functions, has been described, that allows us to estimate the overall quality of a set of convergent services, as perceived by the users, from a set of performance and/or QoS parameters of the convergent IP transport network.

The full sets of services, user perceptions, valuation factors, agents, agent capabilities and performance indicators, have been provided. The full matrix of matching points between agent capabilities and user perceptions has been developed for the particular case of residential (domestic) users with a specific information flow (contents server external to the Internet service provider (ISP), no contents caching outside the contents provider). Valuation and parameterization functions for all services have been provided. For global service quality evaluation, weights for the final services, derived from service usage statistics, have been provided, as well as an example of the use of the analytic hierarchy process (AHP) method for deriving the weights of the elementary services of a final service (Internet access) as well as the weights of the perceptions of an elementary service (digital video broadcast in IPTV). Statistical results for the quality model of a representative service (video quality in IPTV) are presented.

In the summary, the chapter shows the applicability of the proposed model to the estimation of perceived quality (QoE) in convergent 3P services.

1.2.10 Chapter 11: Quality of experience estimators in networks

The Celtic Plus Quality of Experience Estimators in Networks (QuEEN) project was conceived to create a suitable conceptual framework for QoE, and make it operational by means of a suitable software architecture and quality models for different services, covering the full stack from the infrastructure on which a service runs, to the user who experiences it. In this chapter, we will present some of the conceptual results produced so far within QuEEN (and other related activities, such as COST Action IC1003 Qualinet), and the proposed mechanisms from making these concepts operational; that is, a way to theoretically model QoE for any type of (online) service, and a way to go from these theoretical models to concrete implementations. Furthermore, we will introduce some applications of this approach and of QoE in general, such as SLA management, and QoE-driven network management. The rest of the chapter is organized as follows.