Information Theory Coding 6 Sem Ec Notes
Information Theory Coding 6 Sem Ec Notes
H(S).
FUNDAMENTAL LIMITS ON PERFORMANCE:
22. Calculate the capacity of a standard 4KHz telephone channél working in the range of
200 to 3300 KHz with aS/N ratio of 30dB.
23. What Is the meaning of the term™communication channel. Briefly explain data
‘communication channel, coding channel and modulation’channel.
24. Obtain the communication capacity of-a noiseless channel transmitting n discrete
message system/sec.
25. Explain external property_and additivity property.
26. Suppose that 1, S2 are two memory sources with probabilities p1,p2,p3,.....pn for
source si and qi,q2,.....4n for source s2 . Show that the entropy of the source si
a
H(s1)s Pk log (1/qk)
Kan
27. Explain the concept of B/W and S/N. trade-off, with reference to the communication
channel,
* 28. What are important properties of the codes?
29. What are the disadvantages of variable length coding?
30. Explain with examples:
. Uniquely decodable codes, Instantaneous codes
31, the Shannon-Fano coding procedure for the construction of an optimum
32. Explain clearly the procedure for the construction of compact Huffman code.
33.A discrete source transmits six messages symbols with probabilities of 0.3, 0.2, 0.2,
0.15, 0.1, and 0.05. Device suitable Fano and Huffman codes for the messages and
determine the average length and efficiency of each code.
34. Consider the messages given by the probabilities 1/16, 1/16, 1/8, %, Ye. Calculate H.
Use the Shannon-Fano algorithm to develop a efficient code and for that code,
calculate the average number of bits/message compared with H.
35. Consider a source with 8 alphabets and respective probabilities as shown:
eB Ce Dee ecrn
0.20 0.18 0.15 0.10 0.08 0.05 0.02 0.01
Construct the binary Huffman code for this. Construct the quaternary Huffmanand code and show that the efficiency of this code is worse than that of binary code
36. Define Noiseless channel and deterministic channel.
37.A source produces symbols X, Y,Z with equal probabilities at a rate of 100/sec.
Owing to noise on the channel, the probabilities of correct reception of the various
symbols are as shown:
P (fi) 1X ¥ Zz
x % %e 0
y Ye % Ye
2 0 Ye Me
Determine the rate at which information is being received.
38.Determine the rate of transmission I(x,y) through a channel whose noise
-characteristics is, shown in
fig. P(AL)=0.6,- -
P(A2)=0.3, i. a P(A3)=0.1
T
ad os ensemee
as os BS
39. For a discrete memory less source of entropy H(S), show that, the average code-
word length for any distortionless source encoding scheme is bounded as L > H(S).
40. Briefly discuss the classification of codes.
41. Show that HOY) = H(X/Y)+H(Y).
42. State the properties of mutual information.
43. Show that I(X,Y) = I(Y,X) for a discrete channel.
44, Show that for a discrete channel 1(%,Y) 2 0
36, Determine the capacity of the channel shown in fig. .
we
va
45. For a binary erasure channels show in figure below
Find the following:
The average mutual information in bits
The channel capacity
The values of p(x:) and p(x) for maximum mutual information
Lp46.A DMS. has an alphabet of seven symbols whose probabilities of occurrences are
described here:
Symbo SoS: Sz Sa Se Ss Ss
Prob 0.25 0.125 0.125 0.125 0.125. 0.0625" 0.0625
‘Compute the Huffman code for this source, moving a combined symbol as high as
possible, Explain why the computed source code has ani efficiency of 100%.
47. Consider a binary block code with 2" code words of same length n. Show that the
“Kraft inequality is satisfied for such a code:
48. Write short notes on the following:
Binary Symmetric Channels (BSC), Binary Erasure Channels (BEC)
Uniform Channel, Cascaded channels
49. Show that for a AWGN channel C.
density in watts/Hz.
50. Consider an AWGN channel with 4 KHz bandwidth with foise power.spectral density
a
2
the capacity of the channel. -
51. IF 10x, y) = 106)-1Gv/ys). Prove that I(x,y)=I(v)-yyx)
52. Consider a continuous random variable having a distribution as given below
4-448. wtierén/2°= noise power spectral
10-!2 watts/Hz. The signal power required at the réceiver is 0.1mW. Calculate
1 zg
ipl: baa .
0 obentise : :
Find the differential entropy H() =
- ERROR CONTROL CODING:
53. Design a'single error correcting code with a message block size of 11 and show that
by an example that it can correct single error.
54.1 Gand C, an two code vectors in a (n,k) linear block code, show that their sum is
also a code vector.
55, Show CH'=0 for a linear block code.
56, Prove that the minimum distance of a linear block tode is the smallest weight of the
non-zero code vector in the code.
57.What is error control coding? Which ‘are the functional. blocks.of a. communication
system that accomplish: this? Indicaté the function of-each block. What is the error
detection and correction on the performance of communication system?
58. Explain bref the following: (a). Gotay code (b). BCH Code
59. Explain the methods of controlling errors
60. List out the properties of linear codes.
64. Explain the importance of hamming codes & how these can be used for error
detection and correction.
62. Write a standard array for (7.4) code
63. Explain the properties of binary cyclic cods.
64. With a neat diagrams explain Une bittary cyclic encoding and decoding
65. Explain how meggit decoder can be used for decoding the cyclic codes.
66. Write short notes on the following -(i) BCH codes (it) Majority Logic decodable codes (iii) Maximum length codes, R-S
codes ;
(iv) Golay codes (v) Burst errors & its correction
67. Draw the generalized encoder diagram for convolutional codes.
68. Explain the following :
(a) Veterbi algorithm
(b) Stack algorithm
(©) Fano algorithm
69. Differentiate the systematic and nonsymmetric codes
70. Find F1(x) £2(x) if FL(x)=x-F1; F260
71, Prove with an example that [f(x)]2=f(x2)
72. Draw the circuit to get an output c(x)=ai (x). b1(x)-+a2(x).b2(x)
. 73. Draw one divider circuit which gives A(x)/B(x). a
74, Draw encoder and decoder circuit for a (6,3) group code.
75. Write a standard array for (6,2) code
76. Draw and explain the matched filter decoder circuit for (6,3) group code. :
77. Explain the following terms. ,
Hamming distance code rate
Free distance weight of codess (pa nels doh. Fagemiatiay
—Fihty po, ees —the._fuabsbidihg cp deuitinuse. 9 on.
babitdiy cp
tenth _an_eventt that has teaat pint
— —OEUaE NER Cav di .
| eget. ‘ntl uot get ol
7 jets Hae binee ied saa ingen a
a
F
© od Lexy = ty os
F tog, Hd =ia. : a : ee ee - ai +
fle einenage.” ingormafiiatemmaling oe i
-Lheunanaittiaof toy given: natant 4, kime ‘ =a
| the tarprrchad vole op bas, Fnpor 5b id | |
poe gat badena = = fknown 08 = Enlapiy op
Tew) POU) + F 0g) PLaIt+: Team? Plan)
ma
Led Puy te
Hie ag fafa taming uty tn DE on,
Hees Dounce. "+t
= hieee te in oven tmveryede logy
soe ete Cop =
Sog,52. = . bel +
52
reLe Ne q days = 60 X00. Si 1°
=
eps 164 note — eS|}. thie weather _comdidions
fo Glen a Lipical _tode.
| Sol Since each sistas conilontoequsndly uly Ji’.
aot ee he
errr tag mn weal
teen ta = todd
Rolay 00 .
Sisnny. a}
| loudly ‘Lo 5
eh i
ade yfuolby likely, whak to a
tt mpn no 4 binasy floes Aeguisacl pment
~fattiwe using binary drulats.: GQ?
“fet Pag & he tir |
orgy mY & iT
ne pep. To t : : i
ory Lat anaes
ola 4 F o c
Pts aly a tig og eg apy
7 = a J ji
[ - ae
[tee Preemie fo-eetner boxers qeePre rare vey |
| nag Oe enpee baproavrep or rporpartes porary rag
- —— eae
[oc pspeadror pot ery LEK === tK RY = PHOa
sd ag she bays Ba pea -
aa ade figs 8 by bog
“yt FT Tea
__ Be — —lercatetin the mumage dani Cop 81 Saye
Vyoo | 8 Me fal Tid Tiios -
- Jit -
04 — Soap ty tie T it —fijt+e+— = :
5 aly ote areas eee i
Yo Visl WL Hl un[| aiday tte tang
- Sy NA eg ML tM
dn, bs tp 4 BEY
KL F Bnd + 3K +
5 ,
!of0 Sa way |
(Ol al
1p Sr Tal
tO Sa , Is
Win es, 2%,
=area
BE Te one Bone eA 2B
| tia fan at Fadsny io |
| Code Syeabols Probl kt fir
en oy . 8 Fey “Ele : _ _
{
woo Ya fy\ Glas a
tro ep 8 sb eee
The Ss Ys TRE Ty ti
emt Sy Yoo Lge ha tid —ffgne ——|
Tine Se - vac TE Taio
a PP Pa es
Se le (Te TelPiguin
Fyannal inghua {Hae peg Tee ee EE
Ls 875 - Binile /sy bol
i pigs
7 o # é
+ a Ts 3G 3G 3G Ia
- ——aep iifee1 2 : ;
7 eA Vel eM 0
ol | & nS, ae
- YO > Ya 0 Ys
ae
000 |. & : ime =
“COT a ant oe
— i ¥%-1001 a
1x
Seeneos a a‘Leading = Hw) sts
0.00).
aL
1
tT
|
Syd
00)
ay ode. wond_tupgi Ls fang }ta foo dna
: be 88 bint Jaynai
i
PeRS HT bine w= CN
d (89-94 }Oary so-ox b+
ee)
rave Bop rarer cra ee — ag
ia (lono
ae ae [Soo 9 1F ~ong-
[ERROR HORE HTS Dl >
i phe | sag > sa0 IS TOV
i oo | no
Peels ro. ag d010
. hea mY F OFi
: ee |) oi
81 + 6-0
90101
La 2-4 btnids / aaprnlock
NG) = 232192 bila /erpnboot
|
|ee
ee Hie. _2rhobion — aa aed baat
tee ta ny Tali hs dn teat Ata.
epeeee ee eT ee
_ ~esding = 2464 400
2 Log 3
tx
= 100
€ - at1 ooa +Bxt ie = 15833 Winks,
i 7 1 ——
4 __ +t Rey fog bt ah tog!
HO) = a347 bils /expenbol_
8a = 315, Jop_s_44 ’
tga hawaa Mie pach tie a ~, Ouebibiwnce glen
Vg Ls X%
L 4 2 pe bog Np. ———
(0g, 2 ee}
An yo < U-}
puts VapiBat dea ounce exibtng oypnlanla a ain
Coe er ee Tawa xt,
H——-ohen all tae Symbols ase eau probable,
| nadtiptie metind
tn apie Aa tpeicbn gi
=F pt to9, AALS = Prat)
Plog. Bi ACP AP
Ss
> P= ae ere
gg co
ENOL F-Pt P=
ei eee cae 4G HM), are a :
= Yang 2 4 Pt * ads bog Pi xt { +r>xt =o
{
| => dx bog Pr-= d hog 2
i a tet! —| 2s Pi ia, Oa) 20
res
Hex) 20 o_o $ ROD. Aey and:
A souct huadtuct ng.
Hi), _dlun tte enfin A mi —caatin deel a
[Source
Fagin by Ha) = nm h@ [tobe ae lotr)
Dh A Source. Pundicen tos sym tg Pablge
— i ee ee
l gtun PCs) 3/y Resad = Vy.
bat te i 0 9 gs 43) +
Se ee ar ra ar
|-.Syonbols patty py qo extension
7 184 WS) Pls) = Se He?) = "2 95 C'¥4)
SiSo. PLS Pla) = He +
1 + AK 106
i SP C603 Pts) = He = ea CVy
So$o.- PCS PLS = Yr ee) wo =
7 $2) tt To aa
ae : ~ YN 2 TEAR Dileymbd
re HG => 2 Hs)
ard ~
Tie ny ace —
SS aye (8%).
ese a sss
2581 VY + 3°x3 do 1
8 S38) * Yeu 4 ey
SiS)
$2 $1
a $2 $285
{ $2328,
{
epersereeeae:
: ee
LB Gouice pvoduced obphateti hs aan ey Mowing
Sia ual ie Yall Yu, Vo Ye Sr M2 RNG)
TT atwin POD. lag) = Yu, PO): Play) = Y,
a bs aby ote —Souset = HUBD--4 tog 2 + 1 tag 4 +
“Rx Dwg 8 | a
: H(s):2 MTG BIS /sumbol.
IP ee tt fad tts enti nd ion_op tee.
al ay Mie xt Log 6y
i son 64 a
we _! Vai = :
ay You Hs?) = 3:5 bile /sif cabal,
tid aw Yeu : .
My ek Ue DO)
yl 2D. . Yar: -
tad a You mt
yay Voy Ca
ae astot = 4 —- a—blempt}—diswal tungth odes A fined fang in code to. a tnd.
| phone tack nerd 1 cin
———_L fn.
Chmpaii” Vasiabte longi Caden - A vasiatle dungla sods torre
_ hinge Lode. tund lng its te not ffxedk, Ab cota
wo en ea
Glnjed} Disk osen — A madi dink. 5beroid alete
a
i lnfeclana tity Kit déeonnt chow us how te obtain, ned
Pasir f 7 7
|Pabe i
LSE cyntnl lode toda Gad ST.
So o OW 0.
2 lo. ; o}
= S 0. 0) OL
| sa no aole io
1 8y yuy OO ty)— ks las Pl ee
“ode fs oe ae rosa a
pe Kix tras 2}
Min no:oak a +
1 aaad Bsn sg cocetwerd_lengtl an
— itn tam —ending pebame, 22 bp
~ ———
ZF tog Fs)