Mpeg4 and Iptv
Mpeg4 and Iptv
David A. Smith
Rohde & Schwarz IO GmbH
[email protected]
2 – 20 Mbits/s MPEG2
1 – 6 Mbits/s MPEG4
1920 1920
x1080 x1152 80 high * MP@HL * * HP@HL
x 30 x 25 (100 **)
1440 1440 high- MP SSP HP
x1080 x1152 60 * *
(80 **) 1440 @H14L @H14L @H14L
x 30 x 25
720 720
x480 x576 15 main SP@ML MP@ML SNRP * HP@ML
x 30 x 25 (20 **) @ML
352 352
x240 x288 4 low * MP@LL SNRP * *
@LL
x 30 x 25
scalable
scalable
simple
spatial
main
high
SNR
* not yet defined Profiles
** with high profile
functionality
Coding tools,
functionality
bidirectional
prediction
scalability
scalability
scalability
resolution
4:2:0, no
Main +
total
4:2:0,
SNR
Main +
no
DCT = =
l Spatial Redundancy
l H & V blanking Intervals
l Common colors or features
in local region (“Block”)
l Temporal redundancy
l Motion in picture sequences
l Statistical Redundancy I B P
l Mathematical lossless data
reduction
I
Cb
Y Y Y Line n
Cr
4:2:2 Cb
Y Y Y Line n + 1
Cr
Y Y Y Line n
4:2:0 Cb
Cr
Y Y Y
1
for u, v = 0
C( u), C(v ) = 2
1 otherwise
DCT
IDCT
I B B P B B P
Picture
RLC
173, 6, 2*0, -1, 1*0, 2, 1*0, -2, 6*0, -1, 13*0, -1, 34*0
173, 6, 2*0, -1, 1*0, 2, 1*0, -2, 6*0, -1, 13*0, -1, 34*0
DC
Coefficient
compare with
Huffman code table Morse coding
1920 1920
x1080 x1152 80 high * MP@HL * * HP@HL
x 30 x 25 (100 **)
1440 1440 high- MP SSP HP
x1080 x1152 60 * *
(80 **) 1440 @H14L @H14L @H14L
x 30 x 25
720 720
x480 x576 15 main SP@ML MP@ML SNRP * HP@ML
x 30 x 25 (20 **) @ML
352 352
x240 x288 4 low * MP@LL SNRP * *
@LL
x 30 x 25
scalable
scalable
simple
spatial
main
high
SNR
* not yet defined Profiles
** with high profile
functionality
Coding tools,
functionality
bidirectional
prediction
scalability
scalability
scalability
resolution
4:2:0, no
Main +
total
4:2:0,
SNR
Main +
no
Video Compression
l Part 1: Systems
l Part 2: Visual
l Part 3: Audio
l Part 4: Conformance testing
l Part 5: Reference Software
l Part 6: Delivery Multimedia Integration Framework (DMIF)
l Part 7: Optimized reference software
l Part 8: Carriage on IP Networks
l Part 9: Reference Hardware
l Part 10: Advanced Video Coding (AVC)
l ……..and many more…….
l Complex Scene
Representation
l Virtual Reality Modelling
l BIFS Binary Formats
l Sprites
l Mesh definition of Objects
l Network Transport Model
l Streaming and Backchannel
l Multiple Profiles and levels
l …. Too complicated!
l Standard defines:
l Decoder functionality (but not encoder)
l File and stream structure
Motion
Data
Motion
Estimation
1 QCIF 15 64 kbps 4
1.1 320x240 10 192 kbps 3
QCIF 30 9
1.2 CIF 15 384 kbps 6
1.3 CIF 30 768 kbps 6
2 CIF 30 2 Mbps 6
2.1 HHR 30/25 4 Mbps 6
2.2 SD 15 4 Mbps 5
3 SD 30/25 10 Mbps 5
3.1 1280x720p 30 14 Mbps 5
3.2 1280x720p 60 20 Mbps 4
4 HD 720p,1080i 60p/30i 20 Mbps 4
4.1 HD 720p,1080i 60p/30i 50 Mbps 4
4.2 1920x1080p 60p 50 Mbps 4
5 2k x 1k 72 135 Mbps 5
5.1 2k x 1k 120 240 Mbps 5
4k x 2k 30
l Baseline
l Cell phone/PDA
l Has error-resilience tools
l Main
l Broadcast, DVD+, digital cinema
l No error-resilience tools
l Interlace, B-slices and CABAC
l Extended
l IP Streaming, wireless
l Adds error resilience to Main profile but no CABAC.
l FRext
l High Fidelity extensions to Main Profile including lossless
l 10bit, 12bit, 4:2:2 and 4:4:4 modes
CAVLC x x x
Error Resilience x x
SP and SI Slices x
B Slices x x
Interlaced Coding x x
CABAC x
l Variable block-size:
l Every 16x16 macroblock can be subdivided
l Each sub-block gets predicted separately
l Quarter-pixel accuracy
l A gain of 1.5-2dB across the board over ½-pixel
I
I B
P
l MPEG2: l MPEG4:
l Frame Based Only l Slice Based
l Limited Reference Frames l Arbitrary Reference Fames
l ½ Pixel Accuracy l ¼ Pixel Accuracy
• e.g., Mode 3:
diagonal down/right prediction
a, f, k, p are predicted by
(A + 2Q + I + 2) >> 2
Original Vertically
Predicted
Horizontally Diagonally
Predicted Predicted
Discover DVB-T2 : MPEG4 and IPTV
David A. Smith : 1 November 2009 43
H.264 Intra-Prediction Examples
Original Vertically
Predicted
Horizontally Diagonally
Predicted Predicted
Discover DVB-T2 : MPEG4 and IPTV
David A. Smith : 1 November 2009 44
MPEG2 v MPEG4 Pictures, Slices and
Macroblocks
Slice 1 Slice 1
Slice 2 Slice 2
Slice 3 Slice 2
Slice 4
Slice 4 Slice 3
l MPEG2: l MPEG4:
l I, B or P Pictures l I, B or P Slices
l Contiguous Slices Only l Arbitrary Shaped Slices
l No Intra-Frame Prediction l Intra-Frame Prediction
T=1 T=2
T=1 T=2
T=1 T=2
l MPEG4:
l MPEG2:
l Variable Macroblock Size
l Fixed Macroblock Size
l 16 x 16 down to 4 x 4
l 16 x 16 Everywhere
l Important Detail Emphasised
l Equal weight to all areas
l 4 x 4 Integer Transform
l 8 x 8 DCT
2 N 1 N 1
(2x +1)u (2y +1)v 1 1 1 1
F(u, v) = C(u)C(v) f(x,y)cos cos
N x=0 y=0 2N 2N
2 1 1 2
H=
1
for u, v = 0 1 1 1 1
C( u), C(v ) = 2
1 otherwise 1 2 2 1
Horizontal edges
(Luma)
Horizontal edges
(Chroma)
Motion Entropy
Transform Quantization
Estimation Coding
Uncompressed
video Compressed
video
mb_type = 3
Step 1: encode ‘0’ using model C14
Step 2: encode ‘0’ using model C15
Step 3: encode ‘1’ using model C17
Spatial
MPEG4:
DCT
MPEG2
8 x 8 pixel blocks
H.264 / MPEG4 Part 10
Variable Block Size from
16 x16 to 4 x 4 pixels
50%
Integer 4 x4 transform
Adjacent frames only Multiple frames
Simple motion vector Weighted Motion vectors
Temporal
Inter frame only Intra frame prediction
I, B and P Frames I, B and P Slices
DCT Coefficient “In loop” deblocking filter
Visual Quantization and
Bit Rate
Rounding
General purpose RLE CABAC specifically
Statistical optimized for H.264
Reduction
Discover DVB-T2 : MPEG4 and IPTV
David A. Smith : 1 November 2009 61
H.264 vs MPEG-2, 4
Data Partitioning
Parameter
Sets Coded Slice/Partition
DTS
PES
PTS
Headers
l Wavelet Compression
l High frequency changes in small areas
l Low frequency changes in large areas
JPEG JPEG2000
l Significant Improvement over original JPEG
Datagram
Transport Specific
IP UDP or TCP
Elementary Stream
STB with a TV
No STB/TV Terrestrial
TV
PCTV TV
Cable over
TV Copper (xDSL)
Mobile or
TV Fiber (FTTH)
Satellite
TV
“Lean Forward”
TV “Lean Back”
TV
Video over IP
Internet TV
IPTV
l Internet TV Service:
l Provision of TV to anyone via the Internet anywhere
l E.g. Brightcove (USA), Axion (Germany)
l IPTV Service:
l Provision of TV to known subscribers within a controlled
network
l E.g. Verizon (USA), Deutsche Telekom (Germany)
l Internet TV:
l Transport Streams in IP Packets over Internet
l Internet Group Management Protocol (IGMP) allows general
access
l Open system with minimum quality of service control
l IPTV:
l Transport Streams in IP packets over “Final Mile”
l Real Time Streaming Protocol controls access by Set Top Boxes
in network
l Tightly managed closed system owned and operated by service
provider
Edge Server
Central Server
TS TS TS TS TS TS
IP UDP RTP Packet Packet Packet Packet Packet Packet
IP Router
IP Router IP Receiver
IP Server
IP Router
Inter-Packet
Gap
Inter-Packet
Gap
“Ideal” Stream
Real Stream
l Delay Factor
l Indicates Buffer Size needed to remove effects of packet jitter
l Usually expressed as time (=buffer size/stream bit rate)
MDI 3.5 : 0
l Bit rate everywhere = 3.75 Mb/s
l Maximum buffer size needed = 1 IP transport packet (1316bytes)
l 3.5 ms of delay required to subtract the jitter (=1316bytes/3.75Mbs)
l Zero Media (Packet) loss
MDI 266.67 : 0
l Maximum buffer size needed = 1 Mbit at 3.75Mbits/s
l 267ms of delay required to remove the jitter (=1Mbit/3.75Mbs)
IP-Based
Broadcast Networks
MUX
1 QPSK
Received Mod
Feeds
MUX TS
Physical 2
Wiring
Dedicated
Live Encoders
MUX Or TS
or 3 COFDM
Playout
SDH/ATM
Re-Mux
Server Feeds
Switch
Matix Local
MHP
Carousel Program
Generators
TS Insertion
Mod
MUX
n
QAM
MUX
1 QPSK
Received Mod
Feeds
MUX TS
256 x 256 2
ASI
Dedicated
Live Encoders Matrix MUX Or TS
Switch 3 COFDM
Playout
SDH/ATM
Re-Mux
Server Feeds
MHP Local
Carousel Program
Generators
TS Insertion
Mod
MUX
n
QAM
MUX
1 QPSK
Received Mod
Feeds
MUX TS
256 x 256 2
ASI Gigabit
Live Encoders Matrix Ethernet
MUX TS
Switch 3
IP-based COFDM
VOD Contribution
Server Network
TS Mod
MHP
Carousel
Generators Re-Mux
QAM
MUX
IP over DVB n Local
Internet Encapsulator Program
BackChannel Insertion
MUX
1 QPSK
Received Mod
Feeds
MUX TS
Headend 2
IP Gigabit
Live Encoders Network Ethernet
MUX TS
3
IP-based COFDM
VOD Contribution
Server Network
TS Mod
MHP
Carousel
Generators Re-Mux
QAM
MUX
IP over DVB n Local
Internet Encapsulator Program
BackChannel Insertion
DVB-S2
MPEG2 TS 1
Stream Physical RF
MPEG2 TS n Adaptation FEC IQ
Layer
Data Stream 1 & Encoding Modulator
Framing
Framing
GSE Stream n
32APSK
f
t
Multiple
TS or GSE(IP)
Streams
Input Pre-processors I
needed for TS or GSE
MPEG-2 Multiplexer
Encoder
Audio1
Video2 Program 2 (HDTV)
H.264
Encoder
Audio2 TS Packet Stream
Video3 DVB-H 3 (Mobile)
IP (MPE)
Encoder
Audio3
SI Table Data
Input PLP0
pre
Processor
SI Data
PLP0
HDTV
Input
PLP1
pre
Processor SDTV
PLP2
Mobile
PLP3
Video1
MPEG2 IP
Encoder Encapsulator
Audio1 IP Stream 1 (SDTV)
Video2
H.264 IP
Encoder Encapsulator
Audio2 IP Stream 2 (HDTV)
Video3
MPEG4 IP
Encoder Encapsulator
Audio3 IP Stream 3 (Mobile)
Discover DVB-T2 : MPEG4 and IPTV
David A. Smith : 1 November 2009 120
IP Stream Pre-Processing
SDTV
PLP1
Input
pre HDTV
Processor
PLP2
Mobile
PLP3
MUX IP
1
Received
Feeds
DVB-S2
MUX
Headend 2
IP Gigabit
Live Encoders Network Ethernet
MUX
3
IP-based IP
VOD Contribution DVB-T2
Server Network
MHP
Carousel
Generators Re-Mux IP
DVB-C2
MUX IP
IP over DVB n Local
Internet Encapsulator Program
BackChannel Insertion
Questions?