Vietnam National University – University of Engineering and

Technology

Unit 15: Low – Density Parity – Check Codes

‘

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN Hanoi, 2017
Truyền thông số và mã hóa
 Content

1. Highlights

2. Code Construction

3. Minimum Distance

4. Probabilistic Decoding

5. Irregular Codes

6. Revision

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 2 Truyền thông số và mã hóa
1. Highlights
 Two most important advantages of LDPC Codes over Turbo codes:
 Absence of low-weight code words.
 Iterative decoding of lower complexity.
 LDPC codes can be easily constructed so that they do not have
such low-weight code words, therefore the error-floor problem in
turbo codes can be alleviated by careful design of the interleaver.
 LDPC codes use a simple parity-check trellis that has just two
states. So the decoders are significantly simpler than the turbo
decoders.
 However, the LDPC codes use large block lengths, then their
encoding complexity is high compared to turbo codes.

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 3 Truyền thông số và mã hóa
Construction of LDPC Codes
 LDPC codes are specified by 𝐴(𝑛 − 𝑘, 𝑛), a sparse parity-check
matrix A, it consists mainly of 0 and a small number of 1. Denote
(𝑛, 𝑡𝑐 < 𝑡𝑟) LDPC codes.
 𝑛 denotes the block length.
 𝑡𝑐 denotes the weight in each column of the matrix 𝐴, and
 𝑡𝑐 denotes the weight of each row with 𝑡𝑟 > 𝑡𝑐.
 The rate of such a LDPC code is 𝑘/𝑛:
𝑡𝑐 𝑡𝑐 𝑘
𝑟 =1− , =1−
𝑡𝑟 𝑡𝑟 𝑛
 The matrix 𝐴 is constructed by putting 1 in 𝐴 at random, subject to
the regularity constraints :
 Each column contains a small fixed number 𝑡𝑐 of 1.
 Each row contains a small fixed number, 𝑡𝑟 of 0.

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 4 Truyền thông số và mã hóa
Build the generator Matrix
 A of LDPC codes is not systematic (i.e., it does not have the parity-
check bits appearing in diagonal form).
 The generator matrix G may be derived by means of Gaussian
elimination performed in modulo-2 arithmetic.
 The 1-by-n code vector c is first partitioned as
𝐜= 𝐛⋮𝐦
where 𝑚 is the 𝑘 × 1 message vector, and 𝑏 is the (𝑛 − 𝑘) × 1 parity
vector.
𝐀1 𝐀1
 Note: 𝐀𝑻 = … 𝐛 ⋮ 𝐦 … =0 𝐛𝐀1 +𝐦𝐀2 = 0
𝐀2 𝐀2
 Due to:
𝐛 = 𝐦𝐏 𝐏𝐀1 +𝐀2 = 0 𝐏 = 𝐀1 𝐀−1
2 𝐆 = 𝐏 ⋮ 𝐈𝑘 = 𝐀2 𝐀−1
1 ⋮ 𝐈𝑘

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 5 Truyền thông số và mã hóa
Notes
 When the rows of 𝐀 are linearly independent, there will be
some set of (𝑛 − 𝑘) columns of 𝐀 that will make a
nonsingular 𝐀1 as illustrated in the following example.

 For some construction methods for LDPC codes the first

(𝑛 − 𝑘) columns of 𝐀 may be guaranteed to produce a
nonsingular 𝐀1 or at least do so with a high probability, but
that is not true in general.

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 6 Truyền thông số và mã hóa
Example
 𝑛 = 10, 𝑡𝑐 = 3, and 𝑡, = 5.

Figure: Bipartite graph of the (10, 3, 5) LDPC code

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 7 Truyền thông số và mã hóa
 To derive the inverse of matrix 𝐀1

 By using Gaussian elimination, the matrix 𝐀1 is transformed into

lower diagonal form (i.e., all the elements above the main
diagonal are zero),

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 8 Truyền thông số và mã hóa
 This transformation is achieved by the following
 Columns 1 and 2 are added to column 3.
 Column 2 is added to column 4.
 Columns 1 and 4 are added to column 5.
 Columns 1, 2 and 5 are added to column 6.

𝐮 → 𝑢0 , 𝑢1 , 𝑢0 + 𝑢1 + 𝑢2 , 𝑢1 + 𝑢3 , 𝑢0 + 𝑢3 + 𝑢4 , 𝑢0 + 𝑢1 + 𝑢4 + 𝑢5

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 9 Truyền thông số và mã hóa
 The inverse of matrix 𝐀, is therefore

 Finally,

 In practice, the block length 𝑛 is orders of magnitude larger than

that considered in this example. Moreover, in constructing the
matrix 𝐀, we may constrain all pairs of columns to have a matrix
overlap. (i.e., inner product of any two columns in matrix 𝐀) not
to exceed 1.

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 10 Truyền thông số và mã hóa
 Minimum Distance of LDPC Codes
 The block length of a LDPC code is large, ranging from 103 to
106 which means that the number of code words in a
particular code is correspondingly large.

 Over an ensemble of LDPC codes, the minimum distance of a

member code is naturally a random variable.

 As the block length 𝑛 increases, for fixed: 𝑡𝑐 ≥ 3 and 𝑡𝑟 >

𝑡𝑐 the probability distribution of the minimum distance can be
overbounded by a function that approaches a unit step
function at a fixed fraction ∆𝑡𝑐 𝑡𝑟 of the block length 𝑛. Thus, for
large 𝑛 , practically all the LDPC codes in the ensemble a
minimum distance of at least 𝑛∆𝑡𝑐 𝑡𝑟 .

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 11 Truyền thông số và mã hóa
 Table: The rate r and fractional term ∆𝒕𝒄 𝒕𝒓 of LDPC
codes for varying weights 𝒕𝒄 and 𝒕𝒓
𝒕𝒄 𝒕𝒓 Rate r ∆𝒕𝒄 𝒕𝒓
5 6 0.167 0.255
4 5 0.2 0.210
3 4 0.25 0.122
4 6 0.333 0.129
3 5 0.4 0.044
3 6 0.5 0.023

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 12 Truyền thông số và mã hóa
 Probabilistic Decoding of LDPC Codes
 The received vector 𝑟 = 𝑐 + 𝑒
 Due to 𝐀𝐆𝑻 = 0
 The bit-by-bit decoding problem is to find the most probable
vector 𝑐 that satisfies the condition 𝑐 𝐴𝑇 = 0 .
 Procedure:
 Initialization
The variables 𝑃𝑖𝑗0 = 𝑝𝑗0 and 𝑃𝑖𝑗1 = 𝑝𝑗1 are the priori probabilities
of symbols 0 and 1, respectively, with 𝑝𝑗0 + 𝑝𝑗1 = 1.
 Horizontal Step: in the check nodes 𝑖, calculate
∆𝑃𝑖𝑗 = 𝑃𝑖𝑗0 − 𝑃𝑖𝑗1
 For each weight-pair 𝑖, 𝑗 , compute ∆𝑄𝑖𝑗 = 𝑗∈ℑ 𝑖 \𝑗 ∆𝑃𝑖𝑗
0 1 1 1
 Hence, set 𝑄𝑖𝑗 = 1 + ∆𝑄𝑖𝑗 𝑄𝑖𝑗 = 1 − ∆𝑄𝑖𝑗
2 2

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 13 Truyền thông số và mã hóa
  Vertical Step: The values of the probabilities 𝑃𝑖𝑗0 and 𝑃𝑖𝑗1 are
updated using the quantities computed in the horizontal
step
For bit 𝑗, the scaling factor 𝛼 is chosen to make 𝑃𝑖𝑗0 + 𝑃𝑖𝑗1 = 1.
In the vertical step, we may also update the pseudo-posterior
probabilities:
𝑃𝑗0 = 𝛼𝑗 𝑝𝑗0 𝑖∈ℑ 𝑗
0
𝑄𝑖𝑗 𝑃𝑗1 = 𝛼𝑗 𝑝𝑗1 𝑖∈ℑ 𝑗
1
𝑄𝑖𝑗

𝑃𝑗0 + 𝑃𝑗1 = 1
 The quantities obtained in the vertical step are used to
compute a tentative estimate 𝑐 . If the condition 𝑐 𝐴𝑇 = 0
is satisfied, the decoding algorithm is terminated.
Khoa Điện tử - Viễn thông Digital Communications and Coding
Trường Đại học Công nghệ, ĐHQGHN 14 Truyền thông số và mã hóa
 Otherwise, the algorithm goes back to the horizontal step. If
after maximum number of iterations (e.g., 100 or 200) there
is no valid decoding, a decoding failure is declared.

 The decoding procedure herein is a special case of the

general low-complexity sum-product algorithm.

 Due to each parity-check constraint can be represented by a

simple convolutional coder with one bit of memory, the LDPC
decoders are simpler to implement than turbo decoders, as
stated earlier.
 However, experimental results reported in the literature:
Regular LDPC codes do not appear to come as close to
Shannon's limit as do their turbo code counterparts.

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 15 Truyền thông số và mã hóa
 Irregular Codes
 The performance of these codes can be improved substantially
by using their respective irregular forms.
 In a standard turbo code, the interleaver maps each
systematic bit to a unique input bit of convolutional encoder.
Systematic
Systematic
bits
bits xx

Irregular
Irregular Encode
Encode Parity-check
Parity-check
interleaver
interleaver 11 11 bits
bits z1
z1
Message
Message
bits
bits xx

Irregular
Irregular Encode
Encode Parity-check
Parity-check
Interleaver
Interleaver 22 22 bits
bits z2
z2

 In contrast, irregular turbo codes use a special design of

interleaver that maps some systematic bits to multiple input
bits of the convolutional encoder.
Khoa Điện tử - Viễn thông Digital Communications and Coding
Trường Đại học Công nghệ, ĐHQGHN 16 Truyền thông số và mã hóa
 To construct an irregular LDPC code, the degrees of the
variable and check nodes in the bipartite graph are chosen
according to some distribution. For example:

 One half of the variable nodes have degree 5 and the

other half of the variable nodes have degree 3.

 One half of the check nodes have degree 6 and the other
half of the check nodes have degree 8.

 For a given block length and a given degree sequence,

choosing the edges (i.e., the connections between the
variable and check nodes) in a random fashion.

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 17 Truyền thông số và mã hóa
Comparison
 Irregular LDPC code:
𝑘 = 50,000, 𝑛 = 100,000, 𝑟𝑎𝑡𝑒 = 1/2
 Turbo code (regular):
𝑘 = 65,536, 𝑛 = 131,072, 𝑟𝑎𝑡𝑒 = 1/2
 Irregular turbo code:
𝑘 = 65,536, 𝑛 = 131,072, 𝑟𝑎𝑡𝑒 = 1/2
 The generator polynomials for the two convolutional
encoders in the regular/irregular turbo codes are as
follows
Encoder 1: 𝑔 𝐷 = 1 + 𝐷 4
Encoder 2: 𝑔 𝐷 = 1 + 𝐷 + 𝐷 2 + 𝐷 3 + 𝐷 4

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 18 Truyền thông số và mã hóa
Comparison with Shannon limit

Figure: Noise performance compared to the Shannon limit for code rate 𝑟 = 1/2

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 19 Truyền thông số và mã hóa
 Summary
 Error-control coding techniques may be divided into two
families:
 Algebraic codes, which rely on algebraic structure for the
codes for decoding at the receiver.
 Probabilistic codes, which rely on probabilistic methods for
decoding at the receiver.
 Trellis codes combine linear convolutional encoding and
modulation to permit significant coding gains without
sacrificing bandwidth efficiency.
 Turbo codes and low-density parity-check codes share
the following properties:
 Random encoding of a linear block kind.
 Error performance close near Shannon's theoretical limit.
Khoa Điện tử - Viễn thông Digital Communications and Coding
Trường Đại học Công nghệ, ĐHQGHN 20 Truyền thông số và mã hóa
Revision
 What is the LDPC code and what is it’s difference to
classical code?
 How to build the LDPC matrix?
 How to decoding the LDPC code?

Khoa Điện tử - Viễn thông Digital Communications and Coding

Trường Đại học Công nghệ, ĐHQGHN 21 Truyền thông số và mã hóa