1

Inter-Layer FEC Aided Unequal Error Protection

for Multi-Layer Video Transmission in Mobile TV
Yongkai Huo, Mohammed El-Hajjar and Lajos Hanzo, Fellow, IEEE
School of ECS, University of Southampton, UK. Fax: +44-23-8059 4508
Email: {yh3g09, meh, lh}@ecs.soton.ac.uk, http://www-mobile.ecs.soton.ac.uk
AbstractLayered video coding creates multiple layers of
unequal importance, which enables us to progressively rene
the reconstructed video quality. When the base layer (BL) is
corrupted or lost during transmission, the enhancement layers
(ELs) must be dropped, regardless whether they are perfectly
decoded or not, which implies that the transmission power
assigned to the ELs is wasted. In this treatise, we propose an inter-
layer forward error correction (FEC) coded video transmission
scheme for mobile TV. At the transmitter, the proposed inter-
layer (IL) coding technique implants the systematic information
of the BL into the ELs by using exclusive-OR operations. At
the receiver, the implanted bits of the ELs may be utilized for
assisting in decoding the BL. Furthermore, the data partition
mode of H.264 video coding is utilized as the source encoder,
where the type B and type C partitions will assist in protecting the
type A partition. The IL coded bitstream will then be modulated
and transmitted over a multi-functional multiple-input multiple
output (MF-MIMO) scheme for the sake of improving the
systems performance in mobile environments. The proposed
system may be readily combined with the traditional unequal
error protection (UEP) technique, where extrinsic mutual in-
formation (MI) measurements are used for characterizing the
performance of our proposed technique. Finally, our simulation
results show that the proposed system model outperforms the
traditional UEP aided system by about 2.5 dB of E
b
/N0 or
3.4 dB of peak signal-to-noise ratio (PSNR) at the cost of
21% complexity increase, when employing a recursive systematic
convolutional code. Furthermore, unlike the traditional UEP
strategies, where typically stronger FEC-protection is assigned
to the more important layer, employing our proposed IL coding
technique requires weaker FEC to the more important layer.
For example, the system relying on channel coding rates of 0.85,
0.44 and 0.44 for the type A, type B and type C H.264 video
partitions, respectively, achieves the best system performance
when employing a recursive systematic convolutional (RSC) code.
I. INTRODUCTION
Layered video coding [1] was proposed and has been
adopted by a number of existing video coding standards
[2][5], which is capable of generating multiple layers of
unequal importance. Generally the most important layer and
the less important layers are referred to as the base layer
(BL) and enhancement layers (ELs), respectively. A multiview
prole (MVP) [2] was developed by the moving picture expert
group (MPEG)s [6] video coding standard, where the left
view and right view were encoded into a BL and an EL,
respectively. Another layered video coding standard referred to
as scalable video coding (SVC) [3], [4] was recently developed
Copyright (c) 2013 IEEE. Personal use of this material is permitted.
However, permission to use this material for any other purposes must be
obtained from the IEEE by sending an email to [email protected].
The nancial support of the EUs Concerto project, of the European
Research Councils Senior Fellow Grant, of the EPSRC under the auspices
of the China-UK Science Bridge and that of the RC-UK under the India-UK
Advanced Technology Centre (IU-ATC) is gratefully acknowledged.
as an extension of H.264/AVC [4], which encodes a video
sequence into multiple layers, where a reduced-size subset of
the bitstream may be extracted to meet the users specic
preferences. Moreover, the less important layers have lower
priority and hence may be dropped in the transmission scenario
of network congestion or buffer overow [7]. In layered video
transmission relying on SVC [3] streaming for example, when
the BL is corrupted or lost due to channel impairments, the
ELs must also be dropped by the video decoder even if they
are perfectly received.
Unequal error protection (UEP) was rstly proposed by
Masnick and Wolf in [8], which allocates stronger forward
error correction (FEC) to the more important data, while
dedicating weaker FEC to the less important video parameters.
Since then numerous UEP techniques have proposed. A novel
UEP modulation concept was investigated in [9] for the
specic scenarios, where channel coding cannot be employed.
Hence UEP was achieved by allocating different transmission
power to individual bits according to their bit error sensitivity
albeit in practice this remains a challenge. Additionally, the
UEP capabilities of convolutional codes (CC) were studied
in [10], while rate-compatible convolutional codes (RCPC)
were proposed by Hagenauer [11]. Furthermore, as a benet
of the outstanding performance of low-density parity-check
(LDPC) codes, a number of UEP design methodologies [12]
[15] have been investigated using LDPC codes. The so-
called UEP density evolution (UDE) technique of [12], [15]
was proposed for transmission of video streams over binary
erasure channels (BEC). The authors of [13] proposed a new
family of UEP codes, based on LDPC component codes,
where the component codes are decoded iteratively in multiple
stages, while the order of decoding and the choice of the
LDPC component codes jointly determine the level of error
protection. A practical UEP scheme using LDPC codes was
proposed in [14], where the high-signicance bits were more
strongly protected than low-signicance bits.
However, most of the above UEP studies considered arti-
cially generated signals of unequal signicance, rather than re-
alistic video signals. Naturally, the signicance differentiation
of practical video signals is more challenging. In compressed
video streams, as in layered video coding, different bits may
have different signicance. Therefore, again it is intuitive to
employ UEP for protecting the more important bits by stronger
FEC codecs than the less important bits, in order to achieve an
improved reconstructed video quality. Nonetheless, a number
of contributions have been made also in the eld of UEP
video communications relying on realistic video signals. For
example, an UEP scheme was conceived in [16] for object-
based video communications for achieving the best attainable
video quality under specic bitrate and delay constraints
in an error-prone network environment. A jointly optimized
2
turbo transceiver capable of providing UEP for wireless video
telephony was proposed in [17]. The performance of data-
partitioning [4] H.264/AVC video streaming using recursive
systematic convolutional (RSC) codes aided UEP was eval-
uated in [18]. In [19], UEP based turbo coded modulation
was investigated, where both the channel capacity and the
cutoff rates of UEP levels were determined. A novel UEP
method was proposed in [20] for SVC video transmission
over networks subject to packet-loss events. Firstly, the authors
presented an efcient performance metric, termed as the layer-
weighted expected zone of error propagation (LW-EZEP), for
quantifying the error propagation effects imposed by packet
loss events. A novel UEP scheme was proposed in [21], which
considered the unequal importance of the frames in a GOP,
as well as that of the macroblocks in a video frame. An
efcient FEC-coded scheme was also proposed by Chang et
al. in [21]. They also considered the different importance of
the intra-coded (I) frame and of the predicted (P) frames
within a group of pictures (GOP) [22]. The video bits of
different importance were mapped to the different-protection
bits of the modulation constellation points with the assistance
of hierarchical quadrature amplitude modulation (QAM). The
authors of [23] proposed cross-layer operation aided scal-
able video streaming, which aimed for the robust delivery
of the SVC-coded video stream over error-prone channels.
The video distortion endured was rst estimated based on
both the available bandwidth and the packet loss ratio (PLR)
experienced at the transmitter. The achievable video quality
was then further improved with the aid of content-aware bit-
rate allocation and a sophisticated bit detection technique
was conceived, which took into account the estimated video
distortion. Finally, a powerful error concealment method was
invoked at the receiver. An UEP scheme using Luby Transform
(LT) codes was developed in [24] for the sake of recovering
the video packets dropped at the routers, owing to tele-trafc
congestions, noting that the high delay of LT codecs is only
applicable to delay-tolerant broadcast-style video streaming
services.
In the traditional UEP schemes conceived for layered video
communication, variable-rate FEC was invoked for the differ-
ent layers. When the BL is corrupted or lost, the ELs also
have to be dropped, regardless whether they are perfectly
received or not, which implies that the transmission power
assigned to the ELs was wasted. The so-called layer-aware
FEC (LA-FEC) philosophy [25], [26] using a Raptor codec
was invoked for video transmission over the BEC. At the
transmitter, the channel encoding was performed right across
the BL and the ELs. As a benet, at the receiver, the parity
bits of the ELs may be additionally invoked for assisting
in correcting the errors within the BL. Motivated by these
advances, we developed an inter-layer operation aided FEC
(IL-FEC) scheme relying on a systematic FEC code in [27],
where the systematic bits of the BL were implanted into the
ELs. At the receiver, the above-mentioned implanted bit of
the ELs may be utilized for assisting in decoding the BL. The
IL-FEC technique of [27] was also combined with the UEP
philosophy for the sake of further improving the attainable
system performance. Our proposed technique is signicantly
different with the LA-FEC philosophy proposed in [25], [26],
as detailed below conceiving the following aspects. Firstly,
our technique is proposed for layered video communication
over wireless channels, while the LA-FEC of [25], [26] is
proposed for the BEC. Secondly, IL-FEC invokes the soft
decoding aided channel codecs, such as an RSC code, while
the LA-FEC of [25], [26] considered a hard-decoding based
Raptor codec. In this context, we note that Raptor codes
are less suitable for low-delay lip-synchronized interactive
multimedia communications, whilst our scheme is readily
applicable. Furthermore, it is important to note that the LA-
FEC cannot be readily applied in soft decoding aided channel
codecs. Finally, IL-FEC implants the systematic bits of the
BL into the ELs, while the LA-FEC [25], [26] generates the
parity bits across the BL and ELs.
At the time of writing, multimedia content is evolving
from traditional content to a range of rich, heterogeneous
media content, such as traditional TV, streaming audio and
video as well as image and text messaging. Furthermore,
in the current era of smart phones, mobile TV has become
an appealing extension of terrestrial TV. Additionally, in
order to meet the challenging performance requirements in
bandwidth-constrained environments, multiple input multiple
output (MIMO) systems constitute a promising transmission
solution. Layered steered space-time codes (LSSTC) [28], [29]
combine the benets of the vertical Bell Labs space-time
(VBLAST) scheme [30], of space-time block codes [31] and of
beamforming [32]. Hence LSSTCs are invoked for providing
both a diversity gain to achieve a high BER performance in
mobile environments as well as for attaining a multiplexing
gain in order to maintain a high data rate. In this treatise, we
propose a system for transmitting an IL-FEC encoded com-
pressed video bitstream with the aid of a LSSTC transceiver
structure (IL-FEC-LSSTC) for mobile TV broadcasting. This
scheme may be considered as an evolution of the traditional
UEP schemes exemplied by [20], [23]. The data partitioning
mode (PM) of the H.264 video codec is employed, where the
type B and type C partitions will be utilized for protecting the
type A partition
1
. The mutual information (MI) at the output
of the FEC decoder is measured [33] for the sake of analyzing
the performance of our proposed system. Finally, different-
rate, different-protection channel codecs will be employed as
FEC codes for improving the attainable system performance.
Against this background, the main rationale and novelty
of this paper can be summarized as follows. We conceive
an inter-layer FEC codec for layered video streaming, which
is combined with cutting-edge UEP and LSSTC schemes for
the sake of improving the attainable mobile TV performance
with the aid of mutual information analysis. Additionally, the
following conclusions transpire from our investigations:
1) Only a modest complexity increase is imposed by our
inter-layer protection technique, which guarantees the
practical feasibility of our proposed technique. Speci-
cally, 21% complexity increase is imposed by our inter-
layer decoding technique, when employing a RSC codec.
2) Intriguingly, we found that in the context of employing
our proposed technique, the more important layer should
be protected by less FEC-redundancy to achieve the best
overall system performance for H.264/AVC partitioning
mode aided compressed video streaming, which is un-
1
For brevity, we will often simply refer to them as A, B and C
3
expected in the light of the traditional unequal error
protection strategy. For example, the system relying on
the channel coding rates of 0.85, 0.44 and 0.44 for the A,
B and C H.264/AVC partitions, respectively, achieves the
best system performance when employing a RSC code
for the transmission of the Football sequence.
Again, we use the H.264/AVC data partitioning mode in
our simulations, but our proposed scheme is not limited to
partitioning based video, it may be readily applied in any
arbitrary system relying on layered video coding, such as
scalable video coding [34]. The rest of this paper is organized
as follows. In Section II, we briey review the state-of-the-art
layered video techniques. Section III details our proposed IL-
FEC-LSSTC system model and the related video transmission
techniques. Then the performance of our proposed system
is analyzed using mutual information in Section IV. The
performance of our IL-FEC-LSSTC scheme using a RSC
codec is benchmarked in Section V using two video sequences
having different motion characteristics. Finally, we offer our
conclusions in Section VI.
II. LAYERED VIDEO STREAMING
layer 0
layer 1

layer L-1
layer 0
layer 1

layer L-1

Fig. 1. Architecture of a layered video scheme [1], where the video quality
is rened progressively.
Layered video compression [1], [3], [26] encodes a video
sequence into multiple layers, which enable us to progressively
rene the reconstructed video quality at the receiver. Generally,
the most important layer is referred to as the BL, which may
be relied upon by multiple ELs. Furthermore, an EL may be
further relied upon by other ELs. Again, when the BL or an
EL is lost or corrupted during its transmission, the dependent
layers cannot be utilized by the decoder and must be dropped.
A layered video scheme is displayed in Fig. 1, where layer i
(0 < i L 1) depends on layer (i 1) for decoding, while
layer i improves the video quality of layer (i 1).
The subject of SVC [3] has been an active research eld for
over two decades. This terminology is also used in the Annex
G extension of the H.264/AVC video compression standard
[4]. Indeed, SVC is capable of generating several bitstreams
that may be decoded at a similar quality and compression ratio
to that of the existing H.264/AVC codec. When for example
low-cost, low-quality streaming is required by the users, some
of the ELs may be removed from the compressed video stream,
which facilitates exible bitrate-control based on the specic
preferences of the users.
Recently, the Joint Video Team (JVT) proposed multiview
video coding (MVC) as an amendment to the H.264/AVC
standard [4]. Apart from the classic techniques employed in
single-view coding, multiview video coding invokes the so-
called inter-view correction technique by jointly processing
the different views for the sake of reducing the bitrate. Hence,
the rst encoded view may be termed as the BL, while the
remaining views may be treated as the ELs.
A number of layered video coding schemes have been de-
veloped and some of them are adopted by recent video coding
standards, for example the scalable video coding [3] and data
partitioning (DP) [4], [35], [36]. In this treatise, we use data
partitioning based layered video coding in our simulations,
which is a benecial feature of the H.264/AVC codec [4].
In the data partitioning mode, the data streams representing
different semantic importance are categorized into a maximum
of three bitstreams/partitions [37] per video slice, namely type
A , type B and type C partitions. The header information,
such as macroblock (MB) types, quantization parameters and
motion vectors are carried by the A partition. The B partition
is also referred to as the intra-frame-coded partition, which
contains intra-frame-coded information, including the coded
block patterns (CPBs) and intra-frame coded coefcients. The
B partition is capable of prohibiting error propagation in the
scenario, when the reference frame of the current motion-
compensated frame is corrupted. In contrast to the B partition,
the C partition is the inter-frame-coded partition, which carries
the inter-CBPs and the inter-frame coded coefcients. The C
partition has to rely on the reference frame for reconstructing
the current picture. Hence, if the reference picture is corrupted,
errors may be propagated to the current frame. Amongst these
three partitions, the type A partition may be deemed to be
the most important one, which may be treated as the BL.
Correspondingly, the B and C partitions may be interpreted
as a pair of ELs, since they are dependent on the A partition
for decoding. Albeit the information in partitions B and C
cannot be used in the absence of A, partition B and C can be
used independently of each other, again, given the availability
of A. In this treatise, we will employ the partitioning mode of
H.264/AVC for benchmarking our system.
III. SYSTEM OVERVIEW
In this section, we will briey introduce the architecture
of the inter-layer FEC scheme [27] conceived for layered
video transmission over our LSSTC scheme for mobile TV
transmission. The systems structure is displayed in Fig. 2,
where data-partitioning aided H.264 [4] encoding and LSSTC
transmission are employed, while the structures of the variable
node decoder (VND) and check node decoder (CND) [38] are
further detailed in Fig. 3. Both the VND and CND blocks
may accept a maximum of three soft information inputs and
generate a maximum of three soft information outputs with the
goal of iteratively exploiting all IL dependencies amongst the
FEC coded layers A, B and C. Specically, assuming that u
1
,
u
2
and u
3
= u
1
u
2
are random binary variables, the action
of the VND of Fig. 3 sums two LLR inputs for generating
a more reliable LLR output, which may be formulated as
L
o3
(u
1
) = L
i1
(u
1
) + L
i2
(u
1
). The boxplus operation of
L(u
3
= u
1
u
2
) = L(u
1
) L(u
2
) [39] may be utilized for
deriving the condence of the bit u
3
, given that the condence
of the bits u
1
and u
2
is known. Specically, the boxplus
4
Encoder
H.264
s
A B C
Decoder
H.264
s
C B A
QPSK
Demod.
QPSK
x
a,p
x
a
x
ab
x
b,p
x
ac
x
c,p
x
a
FEC
x
a,p
x
ab
x
b
FEC
x
b,p
x
c
FEC
x
ac
Encod. C
Encod. A
Encod. B
D
E
M
U
X
x

1
C
o
n
c
a
t
e
n
a
t
e
D
e
c
o
n
c
a
t
e
n
a
t
e
y
a,p
Decod. A
FEC
FEC
Decod. C
y
ac
y
c,p
FEC
Decod. B y
b,p
L
a
(x
i
b
)
x
a M
U
X
x
-y
a
1
2
4
x
c
1
2
y
ab
y
a
y
c,p
y
ac
y
b,p
3
x
b
x
c,p
L
a
(x
i
c
)
L
e
(x
i
c
)
L
e
(x
i
b
)
L
a
(x
i
b
)
L
a
(x
i
a
)
L
a
(x
i
a
)
y
ab
L
e
(x
i
a
)
-y
a
L
a
(x
i
a
)
y
a,p
L
e
(x
i
c
)
L
e
(x
i
a
)
L
e
(x
i
a
)
L
a
(x
i
c
)
L
S
S
T
C
E
n
c
o
d
e
r
L
S
S
T
C
D
e
c
o
d
e
r
L
e
(x
i
b
)
L
e
(x
i
a
)
y
a
VND
CND
VND
VND
VND
CND
Fig. 2. IL-FEC encoding architecture of H.264 data partitioning mode coded video.
operation is dened as follows [40]
L(u
1
) L(u
2
) =
log
1 +e
L(u1)
e
L(u2)
e
L(u1)
+e
L(u2)
=sign [L(u
1
)] sign [L(u
2
)] min [|L(u
1
)|, |L(u
2
)|]
+log

1 +e
|L(u1)+L(u2)|

log

1 +e
|L(u1)L(u2)|

.
(1)
In contrast to the above-mentioned VND function, the CND
operation of Fig. 3 may be formulated as L
o
(u
3
) = L
i
(u
1
)
L
i
(u
2
) for extracting the condence of the bit u
3
, given the
LLR input of the bits u
1
and u
2
.
In Section III-A, we rst detail the techniques employed
at the transmitter. Then, our inter-layer H.264 decoding tech-
niques and the LSSTC receiver will be illustrated in Sec-
tion III-B, with special emphasis on how the VND and the
CND exchange their inter-layer redundancy for improving the
overall performance of the system. We assume that A is the
BL and B, C are the corresponding dependent layers, but
both partition B and C can be utilized for protecting the
partition A. In Section III-A and III-B, we assume that all the
layers A, B and C contain n bits for the sake of convenient
explanation, while in Section III-C we extend our algorithm
to the more general scenario, where the layers have unequal
length. Finally, Section III-D discusses the overheads imposed
by our proposed IL technique, including its delay, complexity
and its FEC-redundancy.
A. Transmitter Model
At the transmitter, the video source signal s is compressed
using the data partitioning mode of the H.264 encoder, gen-
erating partitions A, B and C. Then the output bitstream is
de-multiplexed into three bitstreams by the DEMUX block of
L
o2
(u
1
)
L
i3
(u
1
)
L
i1
(u
1
)
L
i2
(u
1
)
L
o1
(u
1
)
L
o3
(u
1
) L
o
(u
3
)
L
o
(u
2
)
L
i
(u
3
)
L
i
(u
2
)
L
i
(u
1
)
L
o
(u
1
)
Fig. 3. The structure of VND (left) and CND (right), where and indicate
the addition and boxplus operation, respectively. L
i
() and Lo () indicate
the input and output LLR, respectively.
Fig. 2, namely into streams A, B and C, carrying the A, B and
C partitions of all slices. The resultant binary sequences are
x
a
, x
b
and x
c
, representing three different layers, as shown in
Fig. 2. Then the resultant three layers are encoded as follows:
1) The BL bit sequence x
a
representing A will be encoded
by the FEC encoder A of Fig. 2, which results in the
encoded bits containing the systematic bits x
a
and parity
bits x
a,p
.
2) The bit sequence of the EL x
b
representing B will
rstly be encoded into the systematic bits x
b
and the
parity bits x
b,p
by the FEC encoder B. Then the XOR
operation will be utilized for implanting the systematic
information of x
a
into the systematic information of x
b
without changing the parity bits of the B partition x
b,p
.
Specically, the implantation process results in the check
bits x
i
ab
= x
i
a
x
i
b
. After this procedure, both the check
bits x
i
ab
and the parity bits x
b,p
are output.
3) Similar to the encoding process of the B partition, the
bit sequence of the EL x
c
representing the C partition
will be encoded into the check bits x
i
ac
= x
i
a
x
i
c
and
the parity bits x
c,p
.
5
A&B
Return
VND 4
Hard
Decision of
yes
no
yes
L
e
(b
i
0
)
L
a
(x
i
b
)
VND 4
L
e
(x
i
b
)
A&B
y
a,p
y
a
Decod. B
Error-Free
?
A
Error-Free
?
B
Return
A
no
yes
VND 3
no
decodes A
decodes B
?
Aordable
Iter
Return
null
VND 3
L
e
(x
i
b
)
Inter-Layer Feedback
Feedback L
a
(x
i
a
) FEC
Decod. A
CND 2
FEC
CND 2
Fig. 4. The ow chart for inter-layer aided FEC decoding of BL A and EL
B.
Finally, the bit sequences x
a
, x
a,p
, x
ab
, x
a,p
, x
ac
and x
c,p
are concatenated into a joint bitstream for transmission. Note
however that the layers x
a
and x
b
, x
c
may contain a different
number of bits. Again, the algorithm designed for this scenario
will be detailed in Section III-C. Additionally, the interleavers

1
and
2
are employed for interleaving the BL x
a
, before its
XOR-based implantation into the ELs x
b
and x
c
.
Following the IL-FEC encoding procedure, the resultant bits
are modulated by the quadrature phase-shift keying (QPSK)
modulator of Fig. 2 and then transmitted over the LSSTC
based MIMO transmitter architecture. Specically, the trans-
mission structure shown in Fig. 2 has N
t
= 4 transmit
antennas, which are spaced sufciently for apart in order to
encounter independent fading. The receiver is also equipped
with N
r
= 4 receive antennas, where the LSSTC system used
is characterized by a diversity order of 2 and multiplexing
order of 2. Hence the LSSTC used is capable of providing
twice the data rate of a single antenna system, while achieving
a diversity order of two.
B. Receiver Model
In this section, we exemplify the IL decoding process using
BL A and EL B, while the IL decoding process of BL A
and EL C is similar. At the receiver
2
, the LSSTC decoding
is performed [29]. Then the resultant soft signal will be
demodulated by the QPSK demodulator, which generates the
log-likelihood ratios (LLR). The LLR information contains the
2
The deinterleavers
1
and
2
are ignored at the receiver for the sake
of simplifying the system architecture.
systematic information y
a
, y
ab
, y
ac
and the parity information
y
a,p
, y
b,p
and y
c,p
, for the A, B and C partitions, respectively.
Following the demodulator, the IL-FEC decoder of Fig. 2 is
invoked for exchanging extrinsic information across the three
layers. The IL aided FEC decoding process is illustrated by the
ow-chart of Fig. 4. Firstly, the FEC decoder A will decode the
received information y
a
and y
a,p
for estimating the LLRs of
the bits x
a
of the BL A. Then, the resultant extrinsic LLR
information of BL A will be input to the "VND3-CND2-
VND4" block of Fig. 4 for extracting the a-priori LLRs
L
a
(x
i
b
)
3
of EL B, which is carried out by following the
processing of the LLRs in the VND 3, CND 2 and VND 4
components of Fig. 3. Specically, the "VND3-CND2-VND4"
block of Fig. 4 performs the following operations step-by-step:
1) VND 3 generates the information of BL A for CND 2.
The inputs to VND 3 block are constituted of the soft
information L
e

x
i
a

generated by the FEC decoder A

and the soft information L
a

x
i
a

generated by summing
the channel information y
a
and L
e

x
i
a

, where L
e

x
i
a

is generated by CND 2. The output of the VND 3

block is the soft information of A. The output can be
readily derived as detailed in Fig. 3. The extrinsic LLR
L
e

x
i
a

generated by the FEC decoder A is input to

the VND 3 block of Fig. 2, which extracts the extrinsic
LLR information L
e

x
i
a

and forwards it to the CND 2

block of Fig. 2. Since VND 3
4
has two input branches,
it simply duplicates the soft information L
e

x
i
a

.
2) CND 2 generates the information of layer B for VND
4. The inputs of the CND 2 block are the soft check
information y
ab
received from the channel, the soft
information L
e

x
i
a

of BL A generated by VND 3
and the soft information L
e

x
i
b

of EL B generated
by FEC decoder B of Fig. 2. The output of CND 2
is the soft information of EL B L
a

x
i
b

. The outputs
can be readily derived as detailed in Fig. 3. The LLR
information L
e

x
i
a

and the received check information

y
ab
is input to the CND 2 block of Fig. 2 for extracting
the LLR information of the systematic bit x
i
b
, namely
the soft input L
a

x
i
b

of VND 4.
3) VND 4 generates the information of EL B for FEC
decoder B. The inputs to the VND 4 block are the soft
information L
a

x
i
b

gleaned from CND 2 and the soft

information L
e

x
i
b

generated by FEC decoder B. The

output of VND 4 is the soft information of layer B.
The LLR information L
a

x
i
b

extracted by the CND 2

is input to the VND 4 block of Fig. 2, which extracts
the LLR information L
a

x
i
b

input to the FEC decoder

B of Fig. 2.
Then, the FEC decoder B of Fig. 4 will decode the EL B with
the aid of the resultant a-priori LLR L
a
(x
i
b
) and of the soft
parity information received from the channel, namely y
b,p
of
Fig. 2. Afterwards, the classic cyclic redundancy check (CRC)
is invoked for detecting, whether the recovered BL A is error-
free or not, as shown in Fig. 4. This check results in two
3
As usual, the subscripts "a" and "e" in La and Le stand for the apriori
information and extrinsic information [41], respectively.
4
All the VNDs of Fig. 2 have two input branches and three output branches,
resulting in a duplication process for two of the output branches. Note that
two LLR inputs will be summed by each VND for the third output branch,
which outputs the nal a-posteriori LLR for the estimation of xa, x
b
and xc.
6
possible decoding processes, as shown in Fig. 4 and described
as follows:
1) With Inter-Layer Feedback: When the bits x
a
of the
BL are not successfully decoded, the iterative IL technique
will be activated for exploiting the extrinsic information of
BL A fed back from the FEC decoder B. In this case, both
the solid lines and the dashed lines shown in the decoder of
Figs. 2 and 4 will be activated. More explicitly, the "VND4-
CND2-VND3" block of Fig. 4 will be utilized for extracting
the extra LLR information L
e
(x
i
a
) for BL A based on both
the extrinsic LLR L
e
(x
i
b
) and the soft check information
y
ab
. Generally, the "VND4-CND2-VND3" block of Fig. 4
represents a process similar to that of the "VND3-CND2-
VND4" block of Fig. 4. After this stage, improved a-priori
information is generated for the BL A, which concludes the
current IL decoding iteration. Afterwards, the receiver will
return to the beginning of the ow chart shown in Fig. 4. The
iterative IL decoding process continues, until the affordable
number of iterations is exhausted or the BL A is perfectly
recovered, as shown in Fig. 4.
2) Without Inter-Layer Feedback: When the BL A is suc-
cessfully recovered, the layers A and B will be estimated by
the hard decision block of Fig. 4. Afterwards, the receiver
may discard layer B, depending on whether it is deemed to
be error-free or not by the CRC check. In this case, only the
solid lines of Figs. 2 and 4 will be activated.
Moreover, after decoding BL A, the recovered error-free
hard bits x
a
may be represented using innite LLR values,
indicating the hard bits 0/1, respectively. Then, the CND 2
process invoked for generating the LLR L

x
i
b

shown in Fig.
2 may be derived as follows using the boxplus operation
L

x
i
b

= L(x
i
a
) L(x
i
ab
)
= sign

L(x
i
a
)

sign

L(x
i
ab
)

min

, |L(x
i
ab
)|

+ log

1 +e

log

1 +e

= sign

x
i
a

x
i
ab

,
(2)
where x
i
a
is the modulated version of the bit x
i
a
and the LLR
input L

x
i
ab

is obtained by soft demodulating the received

signal y
ab
.
Note that since the process of recovering y
b
from y
ab
expressed by Eq. (2) is essentially an LLR sign-ipping
operation, it does not affect the absolute value of the LLR
information of x
b
. This implies that in this scenario our
proposed IL technique is equivalent to the traditional UEP
techniques, where layers A and B are encoded and decoded
independently. Moreover, since BL A is decoded indepen-
dently without feedback from EL B, the two layers are only
decoded once, without any extra complexity imposed on the
receiver. Additionally, in practical applications, BL A may be
reconstructed immediately when it is received, without waiting
for the arrival of the EL B.
In both of the above cases, if the decoded bit sequence x
a
of the BL is corrupted after the IL-FEC decoding stage of
Fig. 2, it will be dropped together with the ELs x
b
and x
c
.
Otherwise they will all be forwarded to the H.264 decoder of
Fig. 2 for reconstructing the video signal s.
Note that in the above description, we have considered
decoding layers A and B only. The decoding of layer C is
carried out in the same way but we have excluded it for the
sake of simplifying our discussions.
x
j1
a
x
k1
a
x
a
T
b
x
jn
1
a x
kn
b
a
T
nb
b
T
1
b
(a)
x
1
a
T
v1
b
x
a
x
na
a
T
u1
b
T
b
T
una
b
T
vna
b
(b)
Fig. 5. Denition of T
1
b
, , T
n
b
b
when the BL sequence xa and the EL
sequence x
b
carry unequal length of bits.
C. Inter-Layer FEC Coding for Layers Having Unequal
Length
In the above discussions, we assumed that the A, B and
C partitions have an identical length. However, in practice
they may carry an unequal number of bits. Here we detail
the technique of applying our algorithm in the scenario, when
the three partitions have an unequal length. Let us commence
by assuming that the A, B, C partitions have the length of n
a
,
n
b
, n
c
bits, respectively.
For the case of implanting x
a
into the systematic bits of x
b
,
the basic philosophy of the algorithm is to map/encode x
a
into
a new bit sequence t
b
, which has the same number of bits as
the bitstream x
b
and will be implanted into the systematic bits
of x
b
using the algorithm discussed in Section III-A. In other
words, the bits x
a
will be replaced by the newly generated
bits t
b
for the implantation process. Specically, we introduce
the sets T
1
b
, , T
n
b
b
to assist in generating the stream t
b
,
where the relationship between T
1
b
, , T
n
b
b
and the sequence
x
b
is displayed in Fig. 5. For n
a
> n
b
, we split x
a
into n
b
number of groups on average as in Fig. 5 (a), each constituting
one of the sets T
1
b
, , T
n
b
b
. By contrast, for n
a
< n
b
, we
split T
1
b
, , T
n
b
b
into n
a
number of groups on average as in
Fig. 5 (b), where the sets T
1
b
, , T
n
b
b
within the same group
contain the same single bit of x
a
. So far the sets T
1
b
, , T
n
b
b
have been created from the bit sequence x
a
. Then, each bit
of the sequences t
b
will be generated from one of the sets
T
1
b
, , T
n
b
b
as follows:
t
i
b
=

x
r
a
T
i
b
x
r
a
, 0 < i n
b
. (3)
Given the sequence t
b
, we simply replace x
a
by t
b
, when
implanting the x
a
into the systematic bits of x
b
. Therefore,
x
ab
may be generated correspondingly using x
i
ab
= t
i
b
x
i
b
.
Similarly, the stream x
a
can be readily implanted into x
c
by
introducing the bit sequence t
c
and the sets T
1
c
, , T
nc
c
.
At the receiver, based on the technique detailed in Section
III-B, decoder A is able to generate the extrinsic information
of x
a
. Decoder B is able to generate the extrinsic information
of t
b
with the assistance of CND 2 of Fig. 2. Hence we design
the technique to convert the extrinsic information between
the sequence x
a
and t
b
for the sake of exchanging extrinsic
information among the decoder A, CND 2 and decoder B of
7
Fig. 2. Provided the LLR of x
a
and Eq. (3), the extrinsic LLR
of t
b
may be readily derived using the boxplus operation as
follows
L
e
(t
i
b
) = L

x
r
a
T
i
b
x
r
a

x
r
a
T
i
b
L(x
r
a
). (4)
Similarly, provided the a-priori LLR of x
a
and the LLR of
t
i
b
, the extrinsic LLR of x
a
may be derived as follows.
1) When n
a
> n
b
, the extrinsic information of x
a
may be
readily derived as
L
e
(x
i
a
) = L

x
r
a
T
i
b
\x
i
a
x
r
a
t
i
b

x
r
a
T
i
a
\x
i
a
L
e
(x
r
a
) L

t
i
b

.
(5)
2) When n
a
< n
b
, the extrinsic information of x
a
can be
expressed as
L
e
(x
i
a
) =

T
r
b
,x
i
a
T
r
b
L
r
e
(x
r
a
). (6)
Note that the basic idea of the above algorithm is to map
the bits x
a
into a new bit sequence t
b
, which is basically
an encoder having a variable coding rate encoder. Hence, a
number of codecs, such as low-density parity-check (LDPC)
codes [42] and Luby transform (LT) [43] codes may be
employed for the mapping of x
a
to the stream t
b
. However,
they may impose error-propagation in this specic scenario.
Hence, in this treatise we employ the method detailed in this
section to prevent error-propagation.
D. IL-FEC Overheads
The possible overheads imposed by our proposed technique
are listed as follows:
1) Delay: Our technique is implemented using the parti-
tioning mode of H.264, where each video frame may
be encoded into a number of slices. These slices may
be encoded into at most three partitions. Since the IL
encoding and decoding process is performed within
each slice, no extra delay is imposed by our proposed
technique.
2) Complexity: As detailed in Section III-B, the signal-
ows are based on low-complexity operations compared
to the FEC decoding. When the BL A can be recovered
in its own right, only sign-ipping is necessitated for
extracting the systematic LLR information of the ELs B
and C. Specically, we impose a 21% extra complexity
5
,
as it will be detailed in Section V-C.
3) FEC-redundancy: The BL A does not rely on the ELs
for its decoding operations and the systematic LLR
information of the ELs B and C can be extracted from
the received check information y
ab
and y
ac
without
any loss, provided that the BL is perfectly decoded.
Furthermore, since the transmitted bit sequences x
ab
and
x
ac
have the same length as that of the bit sequence
x
b
and x
c
, respectively, we do not impose any extra
5
According to our experiments, it is sufcient to use a single iteration,
which results in a low complexity.
protection bits. Hence the IL-FEC does not impose extra
FEC redundancy.
IV. MUTUAL INFORMATION ANALYSIS
System Parameters Value System Parameters Value
FEC RSC[1011, 1101, 1111] Number of Tx antennas 4
Modulation QPSK Elements Per AA 4
Channel Narrowband Rayleigh Number of Rx antennas 4
Fading Channel Overall Coding Rate 1/2
TABLE I
PARAMETERS EMPLOYED IN OUR SYSTEMS ,WHERE AA INDICATES
ANTENNA ARRAY.
Error Protection
Code Rates
Arrangements
L0 L1 Average
EEP 0.5 0.5 0.5
UEP1 0.54 0.46 0.5
UEP2 0.47 0.53 0.5
TABLE II
CODING RATES OF RSC CODEC ERROR PROTECTION ARRANGEMENTS FOR
THE BL L
0
AND THE EL L
1
. THE CODE-RATES WERE ADJUSTED BY
VARIABLE-RATE PUNCTURERS.
In this section, we analyze our proposed system using MI
6
.
For the sake of simplifying the analysis, we assume that
there are two layers: a BL L
0
and an EL L
1
. Furthermore,
we employed a 1/3 RSC having the generator polynomials
[1011, 1101, 1111]
7
. The system parameters used in our simu-
lations are summarized in Table IV. In the following analysis,
where two layers are considered, the BL is protected by the IL-
FEC codec. Hence, we consider the convergence behavior of
the BL. For the sake of analyzing our IL-FEC codec, different
error protection arrangements were considered, as shown in
Table II.
In Fig. 6, we plot the extrinsic MI at the output of the
RSC decoder for different E
b
/N
0
values for all the codes in
Table II. Observe from Fig. 6 that the schemes employing our
iterative inter-layer technique always acquire a higher MI value
than those dispensing with the IL-FEC technique. For example,
the RSC-EEP scheme and RSC-EEP-IL scheme generate 0.91
and 0.975
8
extrinsic information at -8 dB. This improvement
is attained by our proposed scheme due to the fact that extra
MI is fed back to the BL from the EL.
V. SYSTEM PERFORMANCE
Let us continue by benchmarking our proposed IL-FEC-
LSSTC system against the traditional UEP aided FEC-LSSTC
system using a RSC. Two 30-frame video sequences, namely
the Foreman and Football clips, represented in (352 288)-
pixel common intermediate format (CIF) and 4:2:0 YUV
format were encoded using the JM/AVC 15.1 H.264 reference
6
MI is known as a metric to represent the condence of a signal sequence.
Generally bigger MI indicates lower BER value of the measured signal
sequence, while lower BER normally indicates lower PLR.
7
The rst polynomial indicates the feedback parameter, while the other
two polynomials represent the feed-forward parameters. The code rates were
adjusted by variable-rate puncturers.
8
Larger amount of extrinsic information indicates a lower BER [44].
8
0.6
0.65
0.7
0.75
0.8
0.85
0.9
0.95
1.0
I
E
[
R
S
C
]
-10 -9 -8 -7 -6 -5 -4 -3
E
b
/N
0
(dB)
EXIT chart for the base layer L
0
RSC-EEP
RSC-EEP-IL
RSC-UEP1
RSC-UEP1-IL
RSC-UEP2
RSC-UEP2-IL
Fig. 6. Extrinsic information generated by the RSC decoders for all error
protection arrangements of Table II.
Football Foreman
Representation YUV 4:2:0 YUV 4:2:0
Format CIF CIF
Bits Per Pixel 8 8
FPS 15 30
Number of Frames 30 30
Video Codec H.264 PM H.264 PM
Bitrate 1522 kbps 655 kbps
Error-Free PSNR 37.6 dB 38.4 dB
Error Concealment Motion-Copy Motion-Copy
TABLE III
THE PARAMETERS OF THE VIDEO SEQUENCES EMPLOYED.
video codec operated in its data partitioning aided mode. The
video scanning rates expressed in frame per second (FPS)
were 30 and 15 for the Foreman and Football sequences,
respectively. The motion-copy
9
, based error concealment tool
built into the H.264 reference codec was employed for the sake
of combating the effects of channel impairments. Moreover,
the H.264 encoder was congured to generate xed-byte
10
slices, as dened in [4]. Both of the 30-frame video sequences
were encoded into an intra-coded (I) frame, followed by 29
predicted (P) frames. The bi-directionally predicted (B) frame
was disabled due to the fact that it relies on both previous
and future frames for decoding, which may introduce more
error propagation as well as additional delay. All the above
congurations jointly result in a bitrate of 655 kbps and an
error-free peak-signal to noise ratio (PSNR) of 38.4 dB for
the Foreman sequence. On the other hand, the coded Football
bitstream has a bitrate of 1522 kbps and an error-free PSNR of
37.6 dB. We employed the Foreman and Football sequences in
order to show the suitability of our scheme for the transmission
of both low-motion and high-motion video. The parameters of
the video sequences employed are shown in Table V, while
our system parametes are listed in Table IV.
9
When the information of a macroblock (MB) is lost, the motion vector
of this MB may be copied or estimated from its adjacent MBs or previously
decoded reference frames. Then, the MB may be reconstructed using the
estimated motion vector.
10
In this mode, the H.264/AVC codec will endeavor to encode a frame into
multiple slices, each having a xed number of bytes.
The H.264-compressed bitstream was FEC encoded and
transmitted on a network abstract layer unit (NALU) [4] basis,
which is the smallest element to be used by the source decoder.
At the receiver, each error-infested NALU must be dropped by
the video decoder, if errors are detected by the CRC check. All
experiments were repeated 100 times for the sake of generating
smooth performance curves.
Below, we will rstly describe the error-protection arrange-
ments in Section V-A. Then we will characterize the attainable
BER versus channel SNR performance and PSNR versus
channel SNR performance employing a lower-complexity RSC
codec in Section V-B. Finally, in Section V-C we will quan-
tify the systems computational complexity by counting the
number of decoding operations executed.
A. Error Protection Arrangements
Error Protection
Code Rates
Arrangements
Type A Type B Type C Average
EEP 0.5/0.5 0.5/0.5 0.5/0.5 0.5/0.5
UEP1 0.35/0.40 0.57/0.65 0.57/0.65 0.5/0.5
UEP2 0.45/0.55 0.52/0.46 0.52/0.46 0.5/0.5
UEP3 0.65/0.60 0.47/0.43 0.47/0.43 0.5/0.5
UEP4 0.75/0.70 0.45/0.39 0.45/0.39 0.5/0.5
UEP5 0.85/0.80 0.44/0.37 0.44/0.37 0.5/0.5
UEP6 0.95/0.90 0.43/0.35 0.43/0.35 0.5/0.5
TABLE IV
CODING RATES OF DIFFERENT ERROR PROTECTION ARRANGEMENTS FOR
THE FOOTBALL/FOREMAN SEQUENCE. THE CODE-RATES WERE
ADJUSTED BY VARIABLE-RATE PUNCTURERS.
In the simulations, we employ the overall coding rate
11
of
1/2 for both EEP and UEP schemes. For each compressed
bitstream, all NALUs were scanned to calculate the total
number of bits for the A, B, and C partitions. Let us assume
that the A, B and C partitions have a total N
a
, N
b
and N
c
bits, respectively. The A, B, C streams have coding rates of
r
a
, r
b
and r
c
, respectively. Then the following equation must
be satised for the sake of guaranteeing that the overall coding
rate remains 1/2:
2 (N
a
+N
b
+N
c
) =
N
a
r
a
+
N
b
r
b
+
N
c
r
c
. (7)
Again, the A stream is the most important layer, while the B
and type C bitstreams are the ELs, where the bitstream B and
C are similarly important. Hence in all the error protection
arrangements we have r
b
= r
c
. More specically, we rst
select a specic value to r
a
, then the value of r
b
= r
c
was
calculated as follows:
r
b
=
N
b
+N
c
2 (N
a
+N
b
+N
c
)
Na
ra
. (8)
Note that the total number of bits for each partitions of
the different video sequences may be different, which results
in different protection arrangements. Based on the above, the
ve error protection arrangements conceived for the Football
and Foreman sequences are shown in Table IV, which may
be readily combined with arbitrary EEP or UEP schemes,
where variable-rate puncturers were designed and employed
to achieve a specic coding rate.
11
Arbitrary overall coding rates such as 2/3, 1/3, 1/4, etc. can be readily
applied by changing the channel codec parameters and the puncturers.
9
10
-5
10
-4
10
-3
10
-2
10
-1
B
E
R
-10 -5 0 5
E
b
/N
0
(dB)
30-frame-Football sequence, FPS= 15, CIF
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
EEP-RSC-LSSTC
EEP-IL-RSC-LSSTC
UEP1-RSC-LSSTC
UEP1-IL-RSC-LSSTC
UEP2-RSC-LSSTC
UEP2-IL-RSC-LSSTC
UEP3-RSC-LSSTC
UEP3-IL-RSC-LSSTC
UEP5-RSC-LSSTC
UEP5-IL-RSC-LSSTC
UEP6-RSC-LSSTC
UEP6-IL-RSC-LSSTC
.
Opt-UEP-RSC-LSSTC
Fig. 7. BER versus E
b
/N
0
performance for the A partition of the Football
sequence, including the RSC coding schemes of Table IV and the Opt-UEP-
RSC-LSSTC [20].
B. System Performance using RSC Codec
In this section, we benchmark our proposed system using
the RSC codec of Table IV. All the error protection arrange-
ments of Section V-A will be utilized. Furthermore, in [20]
an UEP algorithm was proposed, which the authors of [20]
referred to as the optimal UEP. We used this scheme as a
benchmarker, which we refer to as the Opt-UEP-RSC-LSSTC
arrangement.
The BER curves of the A partition in the Football sequence
are displayed in Fig. 7, where the performance of the error
protection schemes of Table IV are illustrated. Observe in Fig.
7 that the schemes using the IL-RSC codec achieve a reduced
BER compared to their benchmarkers. Specically, the EEP-
IL-RSC-LSSTC scheme outperforms the EEP-RSC-LSSTC
benchmarker by about 7.2 dB at a BER of 10
5
. Furthermore,
among all the error protection arrangements, the UEP1-IL-
RSC-LSSTC scheme achieves the best BER performance due
to the strong error protection assigned for the A partition.
Hence, we may conclude that the UEP aided IL-RSC schemes
are capable of providing an improved system performance
compared to the traditional UEP aided RSC codec. On the
other hand, the Opt-UEP-RSC-LSSTC system achieves similar
BER performance to that of the EEP-RSC-LSSTC scheme.
The BER versus E
b
/N
0
performance of the B partition for
the Football sequence is presented in Fig. 8. Similar trends
were observed for the C partition as well, which are not
included here owing to space-economy. Observe in Fig. 8
that the performance of the schemes using IL-RSC is slightly
worse than that of their benchmarkers. This is due to the fact
that more errors may be introduced into the B partition, when
the A partition cannot be correctly decoded. In this scenario
the B partition must be dropped in the traditional UEP aided
RSC-LSSTC schemes. Hence the error propagation to the B
partition does not further degrade the situation.
The PSNR versus E
b
/N
0
performance recorded for the
Football sequence is shown in Fig. 9, where we observe that
the EEP-RSC-LSSTC scheme achieves the best performance
among all the systems without IL techniques, because the
A partition carries only the video header information and
fails to assist the H.264 decoder in concealing the residual
10
-5
10
-4
10
-3
10
-2
10
-1
B
E
R
-10 -5 0 5
E
b
/N
0
(dB)
30-frame-Football sequence, FPS= 15, CIF
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
.
EEP-RSC-LSSTC
EEP-IL-RSC-LSSTC
UEP1-RSC-LSSTC
UEP1-IL-RSC-LSSTC
UEP2-RSC-LSSTC
UEP2-IL-RSC-LSSTC
UEP3-RSC-LSSTC
UEP3-IL-RSC-LSSTC
UEP5-RSC-LSSTC
UEP5-IL-RSC-LSSTC
.
Opt-UEP-RSC-LSSTC
Fig. 8. BER versus E
b
/N
0
performance for the B partition of the Football
sequence, including the RSC coding schemes of Table IV and the Opt-UEP-
RSC-LSSTC [20].
26
28
30
32
34
36
38
Y
-
P
S
N
R
(
d
B
)
-4 -3 -2 -1 0 1 2 3 4 5
E
b
/N
0
(dB)
30-frame-Football sequence, FPS= 15, CIF
.
.
.
.
.
.
.
.
.
.
.
.
. . .
EEP-RSC-LSSTC
EEP-IL-RSC-LSSTC
UEP1-RSC-LSSTC
UEP1-IL-RSC-LSSTC
UEP2-RSC-LSSTC
UEP2-IL-RSC-LSSTC
UEP3-RSC-LSSTC
UEP3-IL-RSC-LSSTC
UEP5-RSC-LSSTC
UEP5-IL-RSC-LSSTC
UEP6-RSC-LSSTC
UEP6-IL-RSC-LSSTC
.
Opt-UEP-RSC-LSSTC
Fig. 9. PSNR versus E
b
/N
0
performance for the Football sequence,
including the RSC coding schemes of Table IV and the Opt-UEP-RSC-LSSTC
[20].
errors, when the B and C partitions are corrupted. Further-
more, the systems using our proposed IL-RSC-LSSTC model
outperformtheir corresponding benchmarkers. Specically, the
UEP5-IL-RSC-LSSTC constitutes the best protection arrange-
ment among all IL-RSC schemes, which achieves a power
reduction of about 3 dB
12
compared to the EEP-RSC-LSSTC
scheme at a PSNR of 36 dB. Alternatively, about 3.7 dB
of PSNR video quality improvement may be observed at a
channel SNR of 0 dB. On the other hand, the Opt-UEP-
RSC-LSSTC system dispensing with the IL technique slightly
outperforms the EEP-RSC-LSSTC scheme, namely by a power
reduction of about 0.5 dB at a PSNR of 36 dB. The UEP5-
IL-RSC-LSSTC substantially outperforms the Opt-UEP-RSC-
LSSTC arrangement, namely by a power reduction of about
2.5 dB at a PSNR of 36 dB or alternatively, about 3.4 dB
of PSNR video quality improvement may be observed at
an E
b
/N
0
of 0 dB. A subjective comparison of the UEP5-
IL-RSC-LSSTC and EEP-RSC-LSSTC arrangements for the
Football sequence is presented in Fig. 11.
12
The power reduction is read horizontally. Specically, the UEP5-IL-RSC-
LSSTC achieves the PSNR of 36 dB with 3 dB less power than the EEP-
RSC-LSSTC scheme.
10
26
28
30
32
34
36
38
Y
-
P
S
N
R
(
d
B
)
-4 -3 -2 -1 0 1 2 3 4 5
E
b
/N
0
(dB)
30-frame-Foreman sequence, FPS= 30, CIF
.
.
.
.
.
.
.
.
. .
. . .
.
. .
.
EEP-RSC-LSSTC
EEP-IL-RSC-LSSTC
UEP1-RSC-LSSTC
UEP1-IL-RSC-LSSTC
UEP3-RSC-LSSTC
UEP3-IL-RSC-LSSTC
UEP4-RSC-LSSTC
UEP4-IL-RSC-LSSTC
UEP5-RSC-LSSTC
UEP5-IL-RSC-LSSTC
UEP6-RSC-LSSTC
UEP6-IL-RSC-LSSTC
.
Opt-UEP-RSC-LSSTC
Fig. 10. PSNR versus E
b
/N
0
performance for the Foreman sequence,
including the RSC coding schemes of Table IV and the Opt-UEP-RSC-LSSTC
[20].
For providing further insights for video scenes having dif-
ferent motion-activity, the PSNR versus E
b
/N
0
performance
of the IL-RSC-LSSTC model is presented in Fig. 10 using
the Foreman sequence, when employing the protection ar-
rangements of Table IV. Similar to the Football sequence,
the traditional UEP technique can hardly improve the recon-
structed video quality by allocating more FEC redundancy
to the more important layers. By contrast, about 2 dB of
power reduction is achieved by the UEP3-IL-RSC-LSSTC
arrangement compared to the EEP-RSC-LSSTC scheme at
a PSNR of 37 dB. Alternatively, about 3.2 dB of PSNR
video quality improvement may be observed at a channel
SNR of -1 dB. Similar to the Football sequence, a limited
gain can be observed for the Opt-UEP-RSC-LSSTC system
compared to the EEP-RSC-LSSTC scheme, while the UEP5-
IL-RSC-LSSTC substantially outperforms the Opt-UEP-RSC-
LSSTC, namely by about 1.8 dB at a PSNR of 37 dB.
A subjective comparison of the UEP3-IL-RSC-LSSTC and
EEP-RSC-LSSTC arrangements for the Foreman sequence is
presented in Fig. 11.
We may conclude from the above discussion that the A
partition should be assigned a code-rate of 0.85 and 0.60
for the Football and Foreman sequence, respectively, for the
sake of achieving the best overall system performance, when
employing the RSC codec, which contradicts to the traditional
UEP strategy. The main reason for this is that the inter-layer
aided RSC decoder can still successfully recover the weaker
protected A partition relying on the extrinsic information fed
back from the B and C partitions with the aid of inter-layer
decoding, because B and C are more strongly protected than
the A partition.
C. Complexity Analysis
In order to provide insights into the complexity of our
scheme, we benchmark the complexity of our IL-FEC-LSSTC
scheme using both the RSC codec in Fig. 12. We emphasize
that if the A partition was corrupted, the corresponding com-
plexity imposed by the B and C partitions was not taken into
account, since they cannot be utilized by the video decoder
in this case. Therefore, the complexity of both the IL-FEC-
LSSTC system and of the benchmarkers is directly propor-
tional to the E
b
/N
0
value. Furthermore, in the simulations
each NALU was encoded by the FEC as a single packet. The
total computational complexity is dominated by that of FEC
decoding. Hence, the total number of FEC decoding operations
substantially affects the systems complexity, which was hence
used for comparing the systems complexity. The y-axis of Fig.
12 represents the average number of RSC decoding operations
per NALU, which was averaged over 2221 NALUs in the
H.264 encoded Football bitstream for the sake of statistical
relevance, where again each NALU was encoded as a single
packet in the experiments.
0.4
0.6
0.8
1.0
1.2
1.4
1.6
1.8
A
v
e
r
a
g
e
d
n
u
m
b
e
r
o
f
R
S
C
d
e
c
o
d
i
n
g
p
e
r
N
A
L
U
-10 -5 0 5
E
b
/N
0
(dB)
30-frame-Football sequence, FPS= 15, CIF
....
.
.
.
.
.
.
.
.
.
.
.
................
EEP-RSC-LSSTC
EEP-IL-RSC-LSSTC
UEP1-RSC-LSSTC
UEP1-IL-RSC-LSSTC
UEP2-RSC-LSSTC
UEP2-IL-RSC-LSSTC
UEP3-RSC-LSSTC
UEP3-IL-RSC-LSSTC
UEP5-RSC-LSSTC
UEP5-IL-RSC-LSSTC
UEP6-RSC-LSSTC
UEP6-IL-RSC-LSSTC
.
Opt-UEP-RSC-LSSTC
Fig. 12. Complexity comparison of the Opt-UEP-RSC-LSSTC system, the
IL-RSC-LSSTC schemes and the classic RSC-LSSTC schemes for the error
protection arrangements of Table IV for the Football sequence.
Observe from Fig. 12 that each curve of the IL-RSC-
LSSTC schemes may be divided into two regions, where the
complexity of the systems increases and decreases upon the
increasing E
b
/N
0
. For example, the curve of the UEP3-IL-
RSC-LSSTC scheme can be split at E
b
/N
0
of about -6.5
dB. Specically, in the E
b
/N
0
region of [10, 6.5] dB,
the complexity of the UEP3-IL-RSC-LSSTC scheme increases
upon increasing the E
b
/N
0
value. This is due to the fact
that the IL decoding technique was activated frequently for
assisting the decoding of A partition. By contrast, for higher
E
b
/N
0
values the A partition is more likely to be recovered
with the aid of the IL technique, which in turn results in
decoding the B and C partitions more than once. In the
E
b
/N
0
region of [6.5, 5] dB, the complexity of the UEP3-IL-
RSC-LSSTC scheme decreases upon increasing E
b
/N
0
value.
The reason for this phenomenon is that the IL decoding
technique is less frequently activated, when the A partition
is more likely to be perfectly decoded in its own right at
higher E
b
/N
0
values. Moreover, the complexity of all the
RSC-LSSTC schemes increases upon increasing E
b
/N
0
. This
may be attributed to the fact that at lower E
b
/N
0
the B and
C partition were more likely to be dropped by the decoder
due to the corruption of the A partition. Since low E
b
/N
0
results in unacceptable video quality, here we only focus on
higher E
b
/N
0
region. More specically, the UEP5-IL-RSC-
LSSTC scheme achieves E
b
/N
0
gains of 3 dB and 2.5 dB
by imposing about 21% higher complexity than the EEP-
RSC-LSSTC and Opt-UEP-RSC-LSSTC schemes at a video
11
Fig. 11. Video comparison at E
b
/N
0
= 2.5 dB for the Football and Foreman sequences. The rst column indicates the original frames. The second
column indicates the EEP-IL-RSC-LSSTC decoded frames. The third column indicates the Opt-UEP-RSC-LSSTC [20] decoded frames. The fourth column
represents the UEP5-IL-RSC-LSSTC and UEP3-IL-RSC-LSSTC decoded frames for the Football and Foreman sequences, respectively.
quality of 36 dB, respectively. Alternatively, the UEP5-IL-
RSC-LSSTC has PSNR gains of 3.7 dB and 3.4 dB at the
cost of a 21% complexity increase compared to the EEP-RSC-
LSSTC and Opt-UEP-RSC-LSSTC schemes at an E
b
/N
0
of
0 dB, respectively.
In conclusion of the Section V:
1) In the RSC based systems, the most important layer
should be assigned less redundancy than partitions B
and C for the sake of achieving the best overall system
performance, which is in contrast to the traditional UEP
strategy. For example, the system arrangement having
channel coding rates of 0.85, 0.44 and 0.44 for the
A, B and C partitions, respectively, achieves the best
system performance when employing the RSC code for
the transmission of the Football sequence.
2) As jointly observed from Fig. 9 of Section V-B and Fig.
12 of V-C, our proposed IL coding technique is capable
of achieving 2.5 dB of E
b
/N
0
again or alternatively, 3.4
dB of PSNR gain over the traditional UEP technique at
the cost of a 21% complexity increase.
VI. CONCLUSIONS
An IL-FEC coded video scheme relying on multi-functional
MIMOs was proposed for mobile TV broadcasting, where the
data partitioning mode of H.264 video coding was utilized
and the systematic bits of the A partition were incorporated
into the systematic bits of the B and C partitions using
an XOR operation. At the receiver, our IL-FEC decoding
technique of Fig. 2 was activated for the sake of attaining an
improved system performance. A RSC codec were invoked
for demonstrating that the proposed scheme is capable of
substantially outperforming the traditional UEP FEC codecs.
The system advocated was analyzed using mutual information
for providing insights into the gain attained using our IL-FEC
coding scheme.
In our future work, we will incorporate the IL-FEC scheme
into SVC and multiview video coding. Moreover, we will also
carry out further investigations for optimizing the inter-layer
coded system performance.
REFERENCES
[1] T. Zhang and Y. Xu, Unequal packet loss protection for layered video
transmission, IEEE Transactions on Broadcasting, vol. 45, pp. 243
252, June 1999.
[2] H. Imaizumi and A. Luthra, Three-Dimensional Television, Video, and
Display Technologies, ch. MPEG-2 Multiview Prole, pp. 169181.
Berlin, Heidelberg, and New York: Springer Verlag, 2002.
[3] H. Schwarz, D. Marpe, and T. Wiegand, Overview of the scalable video
coding extension of the H.264/AVC standard, IEEE Transactions on
Circuits and Systems for Video Technology, vol. 17, pp. 11031120,
September 2007.
[4] Joint Video Team (JVT) of ISO/IEC MPEG and ITU-T VCEG, ITU-
T Rec. H.264/ISO/IEC 14496-10 AVC: Advanced Video Coding for
Generic Audiovisual Services, March 2010.
[5] A. Vetro, T. Wiegand, and G. Sullivan, Overview of the stereo and mul-
tiview video coding extensions of the H.264/MPEG-4 AVC standard,
Proceedings of the IEEE, vol. 99, pp. 626642, April 2011.
[6] L. Hanzo, P. Cherriman, and J. Streit, Video Compression and Commu-
nications: From Basics to H.261, H.263, H.264, MPEG2, MPEG4 for
DVB and HSDPA-Style Adaptive Turbo-Transceivers. New York: John
Wiley, 2007.
[7] F. Yang, Q. Zhang, W. Zhu, and Y.-Q. Zhang, End-to-end TCP-friendly
streaming protocol and bit allocation for scalable video over wireless
Internet, IEEE Journal on Selected Areas in Communications, vol. 22,
pp. 777790, May 2004.
[8] B. Masnick and J. Wolf, On linear unequal error protection codes,
IEEE Transactions on Information Theory, vol. 13, pp. 600607, Octo-
ber 1967.
[9] T. Brggen and P. Vary, Unequal error protection by modulation
with unequal power allocation, IEEE Communications Letters, vol. 9,
pp. 484486, June 2005.
[10] V. Pavlushkov, R. Johannesson, and V. Zyablov, Unequal error protec-
tion for convolutional codes, IEEE Transactions on Information Theory,
vol. 52, pp. 700708, February 2006.
[11] J. Hagenauer, Rate-compatible puncture convolutional codes (RCPC)
and their application, IEEE Transactions on Communications, vol. 36,
pp. 389400, April 1988.
[12] N. Rahnavard and F. Fekri, New results on unequal error protection
using LDPC codes, IEEE Communications Letters, vol. 10, pp. 4345,
January 2006.
[13] V. Kumar and O. Milenkovic, On unequal error protection LDPC
codes based on Plotkin-type constructions, IEEE Transactions on
Communications, vol. 54, pp. 9941005, June 2006.
[14] C. Gong, G. Yue, and X. Wang, Message-wise unequal error protec-
tion based on low-density parity-check codes, IEEE Transactions on
Communications, vol. 59, pp. 10191030, April 2011.
12
[15] N. Rahnavard, H. Pishro-Nik, and F. Fekri, Unequal error protection
using partially regular LDPC codes, IEEE Transactions on Communi-
cations, vol. 55, pp. 387391, March 2007.
[16] H. Wang, F. Zhai, Y. Eisenberg, and A. Katsaggelos, Cost-distortion
optimized unequal error protection for object-based video communica-
tions, IEEE Transactions on Circuits and Systems for Video Technology,
vol. 15, pp. 15051516, December 2005.
[17] S. Ng, J. Chung, and L. Hanzo, Turbo-detected unequal protection
MPEG-4 wireless video telephony using multi-level coding, trellis coded
modulation and space-time trellis coding, IEE Proceedings Communi-
cations, vol. 152, pp. 11161124, December 2005.
[18] Nasruminallah, M. El-Hajjar, N. Othman, A. Quang, and L. Hanzo,
Over-complete mapping aided, soft-bit assisted iterative unequal er-
ror protection H.264 joint source and channel decoding, IEEE 68th
Vehicular Technology Conference, VTC 2008-Fall, pp. 15, September
2008.
[19] M. Aydinlik and M. Salehi, Turbo coded modulation for unequal error
protection, IEEE Transactions on Communications, vol. 56, pp. 555
564, April 2008.
[20] H. Ha and C. Yim, Layer-weighted unequal error protection for
scalable video coding extension of H.264/AVC, IEEE Transactions on
Consumer Electronics, vol. 54, pp. 736744, May 2008.
[21] Y. Chang, S. Lee, and R. Komyia, A fast forward error correction
allocation algorithm for unequal error protection of video transmission
over wireless channels, IEEE Transactions on Consumer Electronics,
vol. 54, pp. 10661073, August 2008.
[22] Y. C. Chang, S. W. Lee, and R. Komiya, A low complexity hierarchical
QAM symbol bits allocation algorithm for unequal error protection of
wireless video transmission, IEEE Transactions on Consumer Electron-
ics, vol. 55, pp. 10891097, August 2009.
[23] E. Maani and A. Katsaggelos, Unequal error protection for robust
streaming of scalable video over packet lossy networks, IEEE Trans-
actions on Circuits and Systems for Video Technology, vol. 20, pp. 407
416, March 2010.
[24] S. Ahmad, R. Hamzaoui, and M. Al-Akaidi, Unequal error protection
using fountain codes with applications to video communication, IEEE
Transactions on Multimedia, vol. 13, pp. 92101, February 2011.
[25] C. Hellge, T. Schierl, and T. Wiegand, Multidimensional layered
forward error correction using rateless codes, in IEEE International
Conference on Communications, 2008, pp. 480484, May 2008.
[26] C. Hellge, D. Gomez-Barquero, T. Schierl, and T. Wiegand, Layer-
aware forward error correction for mobile broadcast of layered media,
IEEE Transactions on Multimedia, vol. 13, pp. 551562, June 2011.
[27] Y. Huo, X. Zuo, R. G. Maunder, and L. Hanzo, Inter-
layer FEC decoded multi-layer video streaming, IEEE Global
Telecommunications Conference, 2012, to appear. Available at
http://eprints.soton.ac.uk/343750/1/1569586913.pdf.
[28] M. El-Hajjar and L. Hanzo, Layered steered space-time codes and their
capacity, Electronics Letters, vol. 43, pp. 680682, June 2007.
[29] L. Hanzo, O. Alamri, M. El-Hajjar, and N. Wu, Near-Capacity Multi-
Functional MIMO Systems: Sphere-Packing, Iterative Detection and
Cooperation. John Wiley & Sons, IEEE press, 2009.
[30] P. Wolniansky, G. Foschini, G. Golden, and R. Valenzuela, V-BLAST:
an architecture for realizing very high data rates over the rich-scattering
wireless channel, in International Symposium on Signals, Systems, and
Electronics, ISSSE, pp. 295300, September 1998.
[31] V. Tarokh, H. Jafarkhani, and A. Calderbank, Space-time block codes
from orthogonal designs, IEEE Transactions on Information Theory,
vol. 45, pp. 14561467, July 1999.
[32] J. S. Blogh and L. Hanzo, Third-Generation Systems and Intelligent
Wireless Networking: Smart Antennas and Adaptive Modulation. New
York, NY, USA: Halsted Press, 2002.
[33] S. ten Brink, Convergence behavior of iteratively decoded parallel
concatenated codes, IEEE Transactions on Communications, vol. 49,
pp. 17271737, October 2001.
[34] Y. Huo, M. EI-Hajjar, M. F. U. Butt, and L. Hanzo,
Inter-layer-decoding aided self-concatenated coded scalable
video transmission, IEEE Wireless Communications and
Networking Conference, WCNC 2013, accepted. Available at
http://eprints.soton.ac.uk/346610/2/IL_SECCC.pdf.
[35] Nasruminallah and L. Hanzo, EXIT-chart optimized short block codes
for iterative joint source and channel decoding in H.264 video tele-
phony, IEEE Transactions on Vehicular Technology, vol. 58, pp. 4306
4315, October 2009.
[36] Nasruminallah and L. Hanzo, Near-capacity H.264 multimedia commu-
nications using iterative joint source-channel decoding, IEEE Commu-
nications Surveys and Tutorials, vol. 14, pp. 538564, Second Quarter
2012.
[37] S. Wenger, H.264/AVC over IP, IEEE Transactions on Circuits and
Systems for Video Technology, vol. 13, pp. 645656, July 2003.
[38] S. ten Brink, G. Kramer, and A. Ashikhmin, Design of low-density
parity-check codes for modulation and detection, IEEE Transactions
on Communications, vol. 52, pp. 670678, April 2004.
[39] J. Hagenauer, E. Offer, and L. Papke, Iterative decoding of binary block
and convolutional codes, IEEE Transactions on Information Theory,
vol. 42, pp. 429445, March 1996.
[40] J. Chen, A. Dholakia, E. Eleftheriou, M. Fossorier, and X.-Y. Hu,
Reduced-complexity decoding of LDPC codes, IEEE Transactions on
Communications, vol. 53, pp. 12881299, August 2005.
[41] C. Berrou, A. Glavieux, and P. Thitimajshima, Near Shannon limit
error-correcting coding and decoding: Turbo codes, in Proceedings of
the International Conference on Communications, (Geneva, Switzer-
land), pp. 10641070, May 1993.
[42] R. Gallager, Low-density parity-check codes, IEEE Transactions on
Information Theory, pp. 2128, 1962.
[43] M. Luby, LT codes, in Proc. 43rd Annual IEEE Symposium Founda-
tions Computer Science (FOCS), pp. 271280, 2002.
[44] R. Otnes and M. Tchler, EXIT chart analysis applied to adaptive turbo
equalization, in in Proceedings of Nordic Signal Processing Symposium,
2002.
Yongkai Huo received the B.Eng.
degree with distinction in computer
science and technology from Hefei
University of Technology, Hefei,
China, in 2006 and the M.Eng.
degree in computer software and
theory from University of Science and
Technology of China, Hefei, China,
in 2009. He is currently working
toward the Ph.D. degree with the
Communications, Signal Processing
and Control Group, School of Electronics and Computer
Science, University of Southampton, Southampton, U.K. He
received a scholarship under the China-U.K. Scholarships
for Excellence Programme. His research interests include
distributed video coding, multiview video coding, robust
wireless video streaming and joint source-channel decoding.
Mohammed El-Hajjar is a lecturer
in the Electronics and Computer Sci-
ence in the University of Southamp-
ton. He received his BEng degree in
Electrical Engineering from the Amer-
ican University of Beirut, Lebanon in
2004. He then received an MSc in Ra-
dio Frequency Communication Sys-
tems and PhD in Wireless Commu-
nications both from the University of
Southampton, UK in 2005 and 2008,
respectively. Following the PhD, he joined Imagination Tech-
nologies as a research engineer, where he worked on designing
and developing the BICM peripherals in Imaginations multi-
standard communications platform, which resulted in several
patent applications. In January 2012, he joined the Electronics
and Computer Science in the University of Southampton as
a lecturer in the Communications, Signal Processing and
Control research group. He is the recipient of several academic
awards and has published a Wiley-IEEE book and in excess of
30 journal and international conference papers. His research
interests are mainly in the development of intelligent commu-
nications systems for the Internet of Things including massive
MIMO systems for mm-wave communications, cooperative
communications and Radio over bre systems.
13
Lajos Hanzo (http://www-
mobile.ecs.soton.ac.uk) FREng,
FIEEE, FIET, Fellow of EURASIP,
DSc received his degree in electronics
in 1976 and his doctorate in 1983. In
2009 he was awarded the honorary
doctorate Doctor Honoris Causa
by the Technical University of
Budapest. During his 35-year career
in telecommunications he has held
various research and academic posts in Hungary, Germany and
the UK. Since 1986 he has been with the School of Electronics
and Computer Science, University of Southampton, UK, where
he holds the chair in telecommunications. He has successfully
supervised 80 PhD students, co-authored 20 John Wiley/IEEE
Press books on mobile radio communications totalling in
excess of 10 000 pages, published 1300 research entries
at IEEE Xplore, acted both as TPC and General Chair of
IEEE conferences, presented keynote lectures and has been
awarded a number of distinctions. Currently he is directing
a 100-strong academic research team, working on a range
of research projects in the eld of wireless multimedia
communications sponsored by industry, the Engineering
and Physical Sciences Research Council (EPSRC) UK, the
European IST Programme and the Mobile Virtual Centre
of Excellence (VCE), UK. He is an enthusiastic supporter
of industrial and academic liaison and he offers a range of
industrial courses. He is also a Governor of the IEEE VTS.
During 2008 - 2012 he was the Editor-in-Chief of the IEEE
Press and since 2009 he has been a Chaired Professor also
at Tsinghua University, Beijing. For further information on
research in progress and associated publications please refer
to http://www-mobile.ecs.soton.ac.uk