CS/EE 5590 / ENG 401 Special Topics

(Class Ids: 16924, 16925,16935)

Multimedia Communication

Zhu Li
http://l.web.umkc.edu/lizhu

Z. Li: Multimedia Communication, 2017 Spring p.1

 Outline
 Background
 Objective of the class
 Prerequisite
 Lecture Plan
 Course Project
 Q&A

Z. Li: Multimedia Communication, 2017 Spring p.2

 Video Growth
 Better and Bigger Video:
 Moving to UHD 4K resolution
 HDR – High Dynamic Range
 Much higher data rate
 Mobile Video:
 25X growth predicted over the next 5 years
(source Cisco Visual Networking Index)
 Low power encoding/decoding on devices
 Mobile network capacity gap
 Internet Video:
 Already accounts for more than half the
internet traffic
 Netflix alone 70% of ISP traffic at peak
time.
 Need better coding efficiency

Z. Li: Multimedia Communication, 2017 Spring p.3

 Video Signal

Time

Z. Li: Multimedia Communication, 2017 Spring p.4

 Video Compression

[Source: ISCAS 2014 tutorial, V.Sze and M. Budagavi]

Z. Li: Multimedia Communication, 2017 Spring p.5
 Compression Approaches
 Basics of Video Compression
 Remove redundancy from the signal – brain to eye connection is
estimated to have only several hundred bits/sec capacity.
 Video Signal Redundancy:
 Spatial Redundancy
 Perceptual Redundancy
 Temporal Redundancy
 Symbol Statistical Redundancy

Z. Li: Multimedia Communication, 2017 Spring p.6

 Spatial Redundancy Removal
 Intra Prediction – predict within the frame

Z. Li: Multimedia Communication, 2017 Spring p.7

 Spatial Redundancy Removal
 Block Transform:

Z. Li: Multimedia Communication, 2017 Spring p.8

 Perceptual Redundancy
 Human Visual System (HSV) is more sensitive to changes in
low freq. areas than high freq.
 Capitalize on this by allow for coarser quantization at different
texture areas.

Z. Li: Multimedia Communication, 2017 Spring p.9

 Perceptual Redundancy Removal
 Quantization

Quant table

Quantized DCT coeff

Z. Li: Multimedia Communication, 2017 Spring p.10

 Temporal Redundancy
 Exploit redundancy among successive frames

Naïve !

Z. Li: Multimedia Communication, 2017 Spring p.11

 Motion Compensation
 Block based motion compensation for Temporal Redundancy
removal

Z. Li: Multimedia Communication, 2017 Spring p.12

 Video Motion Prediction Structure
 I, P, B, b Frames…

Z. Li: Multimedia Communication, 2017 Spring p.13

 Symbol Stats Redundancy
 Not all pixel values have the same frequency
 Say if 128 appears 90% of time, we can use short bits to represent, and
longer bits to represent the rest of symbols
 Entropy coding !

Z. Li: Multimedia Communication, 2017 Spring p.14

 Video Compression in Summary

Z. Li: Multimedia Communication, 2017 Spring p.15

 Video Coding Standard
 Why Standard ?

Z. Li: Multimedia Communication, 2017 Spring p.16

 Video Coding Standard History
 Pre-HEVC

Z. Li: Multimedia Communication, 2017 Spring p.17

 Rate-Distortion Performance
 Pre-HEVC

Z. Li: Multimedia Communication, 2017 Spring p.18

 HEVC – High Efficiency Video Coding
 Objective of HEVC

Z. Li: Multimedia Communication, 2017 Spring p.19

 HEVC History
 Very brutal…

Z. Li: Multimedia Communication, 2017 Spring p.20

 HEVC Performance

Z. Li: Multimedia Communication, 2017 Spring p.21

 HEVC Resources
 Overview Paper:
 G. J. Sullivan, et al. "Overview of the High Efficiency
Video Coding (HEVC) standard,” IEEE Transactions
on Circuits and Systems for Video Technology, 2012
 Spec: H265 High Efficiency Video Coding Specification
 http://sist.sysu.edu.cn/~isscwli/ref/h265.pdf
 Reference Software:
 – https://hevc.hhi.fraunhofer.de/svn/svn_HEVCSovware/
 Python Tool: HARP
 http://lms.lnt.de/HARP/

Z. Li: Multimedia Communication, 2017 Spring p.22

 New Media – 3D Image/Video
 Point Cloud Capture & Compression

Gis: Lidar: http://www.cadalyst.com/cadalyst/gis-tech-news-99-13394 Cultural heritage (images): Culture3D Cloud m336523

Robotics: lidar, kinect: www.pointclouds.org

Z. Li: Multimedia Communication, 2017 Spring p.23

 Point Cloud Compression
 Octa Tree Decomposition & Coding

recursive sub-divsions: complexity O(2^(N))

Limit level N and differentially code of points in larger leafs for real-time
coding [Kammerl12]
Prediction of subdivisions based on the previous level [Schnabel06, Huang06]
Context Adaptive Entropy encoding [Schnabel06, Huang06]
range coding [Kammerl12]

Z. Li: Multimedia Communication, 2017 Spring p.24

 Video Communication
 The Dimension of the Video Communication Problem:
 Content: managed vs UGC
 Transport: managed IP networks vs Over The Top (OTT) networks
 Devices: managed settop boxes, vs unmanaged devices

[Christian Timmerer & Ali Begen, ICME 2015 Tutorial]

Z. Li: Multimedia Communication, 2017 Spring p.25
 Current Landscape
 Managed vs Unmanaged:

[Christian Timmerer & Ali Begen, ICME 2015 Tutorial]

Z. Li: Multimedia Communication, 2017 Spring p.26

 Traditional TV vs OTT Video
 Services are converging…

Z. Li: Multimedia Communication, 2017 Spring p.27

 Netflix Phenomena
 Purely OTT Solution, very successful in user penetration
 Accounted for a large portion of ISP traffic between 8-10pm.

Z. Li: Multimedia Communication, 2017 Spring p.28

 HBO

Z. Li: Multimedia Communication, 2017 Spring p.29

 Internet is dominated by video traffic
Top Down/ Bandaid approach:
 CDN (Content Delivery Network) Solutions, mostly based on HTTP
transport
 New Trend/Clean Slate Approach:
 ICN (Info Centric Networking), design from layer 3 up to support
video

Z. Li: Multimedia Communication, 2017 Spring p.30

 PSS over managed IP networks
 Managed mobile core IP networks

Z. Li: Multimedia Communication, 2017 Spring p.31

 MPEG DASH – OTT
 HTTP Adaptive Streaming of Video

Z. Li: Multimedia Communication, 2017 Spring p.32

 Quality of Experience (QoE)

QoE
 Spatial quality
 Temporal quality
 Viewing condition
 Packets received

Z. Li: Multimedia Communication, 2017 Spring p.33

 Prerequisite & Text book
 Prerequisite
 Good C and Python programming skills
 Taken Digital Signal Processing and/or Digital
Image Processing, or consent of the instructor
 Will have different expectation for MS and
undergrad students
Textbook:
 None required (saving $$) , will distribute relevant
chapters, papers, and notes.
Key References:
 Y. Wang, J. Ostermann, and Y.Q.Zhang, "Video Processing and
Communications," 1st ed., Prentice Hall, 2002. ISBN: 0130175471.
 K. Sayood, Introduction to Data Compression, 3rd Edition

Z. Li: Multimedia Communication, 2017 Spring p.34

 Tentative Plan
 Video Coding
1) Entropy and Info Theory Background
2) Lossless Coding/Entropy Coding: Hoffman, Arithmetic
3) Lossy Image Coding: Transforms, Quantization, Wavelet/JPEG2000
4) Video Signal Processing: Sampling, Motion Compensation
5) Scalability and Super Resolution
6) Video Coding Standards: HEVC
 Video Networking
1) QoE metrics,Source-Channel Coding and Error Resilience
2) Media Transport: RTP/RTSP/RVSP/RTCP, HTTP/Websocket, WebRTC
3) Congestion Measurement and Control: TCP, SCTP, SPDY, NADA
4) CDN, HTTP Cache, Cache Deduplication and delivery acceleration
5) OTT Video Streaming Standards: DASH, MMT
6) Overlay networks and P2P video streaming

Z. Li: Multimedia Communication, 2017 Spring p.35

 Course Outcome
 Upon completion of the course you will be able to:
 Understand the basic compression theory and algorithms for media
compression.
 Familiar with the latest media compression and communication
solutions and have hands on experiences.
 Can apply the knowledge an algorithms to solve real world media
communication problems
 Have good job and internship prospect with both Semi Conductor
Makers like QCOM, MediaTEK, Samsung, Intel, Broadcom, and
content provider and CDN/ISP operators like AKAMAI, Netflix,
Youtube, COMCAST, HBO, Sprint, ATT.

Z. Li: Multimedia Communication, 2017 Spring p.36

 Grading
 Homeworks (30%)
 Lossless coding of symbols
 Lossy image coding
 Motion Compensation
 Video codec excercise
 Network simulation of video streaming
 Quiz (30%) : relax, quiz is actually on me, to see where you
guys stand
 Quiz-1: On Video Coding
 Quiz-2: On Video Communication
 Project (40%)
 Original work leads to publication (1.25x factor), discuss with me by
Spring break
 Regular project: assign papers to read, implement certain aspect, and
do a presentation

Z. Li: Multimedia Communication, 2017 Spring p.37

 Logistics
 Office Hour:
 Mon, Wed: 2:30-4:00pm, 560E FH
 Or by appointment
 TA:
 Dewan Noor
 Lab Sessions are planned to cover certain software tools aspects.
 Office Hour: TBA
 Course Resources:
 Will share a box.com folder with slides, references, data set, and
software
 Volunteer Course Web Master recruiting: count as 10% extra credit


Z. Li: Multimedia Communication, 2017 Spring p.38

 Q&A
 …..

Z. Li: Multimedia Communication, 2017 Spring p.39