UNIT 2

LECTURE 3 : Parametric statistical model

Hi there,

This is your pal Scooby

Following are my notes for the lecture. These are my handwritten detailed notes. I felt that Prof. R
taught a lot more in the videos than he wrote in the slides. I tried to make them without any
mistakes and cover most of the video material.

But as Beyonce has rightly said : 'To err is human'

I however DO NOT TAKE any responsibilty if you fail your exams following my notes. So please
check the actual lecture slides and videos if you suspect something is wrong.

Also tell me about it at : [email protected]

The notes DO NOT CONTAIN solutions to any homework problems.Only the problems and
exercises solved in the videos are included in these notes.

Here's a motivational quote

'You don't know what pain is unless you have gone through a terrible breakup'
~ Every other teenager on Insta after being dumped

Here's a cute photo I googled

I hope these notes will help you too

Let's get started

LEc +3: ?anauETRrc SrarisricAL llopeI 3,1
V fW" roti,onaLe behind stctts{i,cal rnode,llin{
t* tf.. obsewaH
Dtrrt^tng ony kind o-[ stattsdcqL expertirnent ,AVtz. VanLtculort 6;114&

vneasure is modetted as a l.andorra vartiobte usuo-\. dTA %

{*f*r[ +61t r* +l^"ct
Xi . We assume all tae n obgervattovuc nre {ake ove iid te identtca}I4-
oncl trrcte penaer,rfiS dis+ri bule d"

Xt ,Xz , Xa Xtn- o.ye. iiol vvLeovl&

. ,fur! X',. {otlows -the same proloabiti}5 d'rstribr'tHor't

. facl,u y.v. xi rr'',dupL;"i + V i'i t lr't"'n\ o\hev Y'v' x5
^/' "q aI[ thece K'l
We atnr to -FUncL the c.o7nmon. dtstnbultt''n- +
vurtieku *uonl-;" ;*; 1ir\n" {^ose g.arne_b &t 1V.,a* cornrlebeQ d**
ts",'te exomptei'
a drsh/tbutuor once*pu.Jq"!i:.c" t;
t

Suppose -[cx sonae r€ason h]e rnodeL X as Qcusston'

C
Xi '\, N ( t, o-')
'il \'Me
a vtormat / vor,rdonr, varciabte
Ne knor,r: -tl"re pd{
"I (x- r{l' 6ousstavt
1 g- A 1)t-<-@
(x) + 9-cz
+x €W
und' r \^)e LNn-
6l- q
Owee hle kvtow 1l/\8, value *'r*
?arao..z(e:'t
randortwaxidoLz Xi
EnShleY aLL orreluasliorn s tl- nnitit re3arddi
S*f.Posa heasolt h/e rnodel X ru ?otsso't RandorrVailoL:le
t" SoYYlR"

yi '\) PoB ( X)
Itue pdf l-odk" l''ke
,.k g-I
t*. ke fo,t,Lrb'4"" 3
b
lx
(x) =
kr
Once hle knoro' + I -l"u
patanno,tev vu€ can anshle'1
va\.rra
aq pv&otrUshc lueshor.u vt.g**at:2 Xi
-{orc
lvrz distribuhons choose X; neeA not b be *lae- s+anf,*'.d" 6n'%
@ r,.,,.rc

lw tne eu btiqs axonrupte Xi = rro . { sibtin-gs itft p"v*on. had

)L; I 2- \ 1 5 6 7r7

P(X=x)' pr lz le [a ?' l+ Ps
\en r^!e r^rc0 hole 10 k.vrorar 7 poro metevg ie lr ,lz,l>,1+ ,Y, ,Y, , ?'
'Ne a\so how tw- ansbraint"thob
ft+ pa+ fs + P.* P' = L
12r+l2z +

We leayn )il*,se yovarvrrators uslrr* Ilr"z olata co\JzcYed arud {u +l,r^t

wL twed a sXdtevrtnUc pocedul re
 3.1
> Etattsfi cal, rnodel
We have Ia i,id oloserva,hona . We Nant.o make probabttslcc cdc[loltovr,s
ftvst step lo do *ln-is is *o
-m" spectfS the statisttcal rnorlel

De{,i : A Stahsttcal model in fts rvrost genertl 6ense is a p*i,

(e , (B)r.
Lets explore each teYrn
.E : .T[re
Somple SPace
Tne szt {rsm \^}hich" valuer { X* coryla 6r \ colLe;c{.ol 6t cill po34'ibl€
v.ure^ 4"a xlo}rgy.set like .R {ii olso vohd, t+ ispere{fieredtot<ee{a E as
{ftho-r6h-
pree-rde as Possible

' (F") seo t fa'*Q d f,ol"t'itit drsbibuu.',1 .'r'\ E

{nes" cnvt ba +^our&t' as al.L pssibte fwtctts'rns ^d, o.
*ptt'!'
frrm that """ bl rnode *v9...r[ tlae farameterg-*akeg
it{!.re"t r.ralr-r er [n O
o .. ?avaft\fh1,'r Seb

Ttnr rnc.t&mohcal sel +o wtnrcL d betongs

sr plL posstbte voluu

+ partomehit d
A wetL spe.,{t"d model
we fov ouY PuYgoses o-S sume that ouv stqtstrcal rnoctet is rocllsPeci$oa
De{'n ; fie de{tned, s.t. *here.
moctel is er so *lnat .P lPer
exist =
rUhot 'tlnis mB,aYx is ane Sluaxe (ivr, Venn dolvam vepresert'ha'ltot')
cortoiuts at[ pvobatoiUt dbtribctton a *h,at ravt
be de,fmed orr g
{9, seO {n n is 'Pitr- true P 'that genevated' otr'Y
dato.
Tne.rna ls ako the, subseb t F*, e e O t
ivtside this a1trarre. (denoH bY 4he P^tcl^)
(t^rs rs the " {^-.ih + qlt clbhibutbha
t r^rtth yarcvnr:bd'r U
Allprobabttitl dirbtbutiq crn E

{" flrx, wwdol 4t bo nrell specit-rzd ^le assur,vLe -than our &stnl>ulio'+
P ig ir,v thcL subset 4 J,^*"+r.r^ / atsb:Lau*tc,rnr
{n, porttcular g ta called tta -lrue parcanr,ete-f
P ?ayam eftic, Noru pavarlretrtc & Semipara me,iric modd.*
We recall" !^ri,!et panamekex Spacer,c {ts *he sPace nhevc the poutorv\iltt+ J^^ e
Once a vnoo\et U t"lelJu spectfted, y\!e worut *o knoto 9 , i e*te -lrue pazarneterc
*?"Gt {s unknowr}. Tiajs I betorgs lrc tlaeset -re, -tl.re pavavno*t:<
spa,Co
Ovu the bo^r&
+ p axam e*zr S et hle dr* w e- 4h,e rnodaL o(
lI) Pcrrc arYrLt.iL : E Rd {ov sozte d>,
fz) Non Qavame*tic t O i'e t"uf+roite elimenslorral
IE> Semi 9ararnetric : O - Or- X O, nrhere O r jt"ite d*rrnerrrio-rtnl
a tru{tnite, dirnet*r o/\^l
p Lets r"rri*e sorne stafrshcal rnoc{el.( Qarurnthic)
Arcgeneval.prrw & (e, tq1".o) NOT9
, chongu bn
()
r3 and
bolhmenvtslrru

{", r\ %ernojttrl,-*rtats 'l

we know +l^-at .,r.v xi -con-talce onlg *too varua o 6r L Ao : E = to, l3

Iw pn omphu { ir^terest ltzve [s g = p 0s it characte^r,tzu r^rhote Pdf
fVrz drbtri.buh.orr ts t/stjtten as Bern(p) which" U elt,valant lo Fs
tde atso knor,.o *ltat p ts a vtltmbex 6e-t,uoe?A, o on1d, I a$ L+ is *t"e.
p,robafiUq q Xi =1 8o p € (o, t) . bo 6 = (o,r)
Jhe co.se. p= o and p= | corresponcl -to *he case *
+% *rroaed coin Eo rare rm\b{hcrn
Aw ,^odeL iE
/
(fo,rJ, (eerrpl)f e6o,r1 )

@ tl X,x1,x. Xn Xd ?"*a(,1) fan s*,,a A )o

rde knor,.r Xi Pot!s(f) i-pLr Yi c-an*ake orntg not n?a*tue valuu ft can *ake
^,
O {oo . Ao ; E = Nf ie wahfiqL nurrr[s],g

{1ni gx?trrv,tex ,t irnfurest hz}z U A 8o g = }"

fl"e drstrlbutcozt ts ravritten ag ?otst,f1 hlhidl, ts elu,t,zalent 1o Pg
Y the de{initton r} Pc,tsgons dt.tribuhon ,I. is g rentuc tharr, O
d* Pu
}"e an-d, h.le lr.ave .\ 7 6
Bo O = (orm)
t\r* rvrode[ fs

NII , ( P'tss(x,))
1)o
 J.1

@ Xr , Xr, Xa . . . Xn }.9J, (tl,,-') for some ,le? srld rt lo

\de. knon a .y.v Xi etn l6tke aru1 va)ue iw tho veal )inz E = fi
$outua,v
\ere haw *trco unknovozrs t"
u.re arn d" rr, Ao now g iA ,wl a AV" tumbw
rr t
vardcrbte , tE a tn:o dirnensio;d lre.ctolc rue nyrite Lt a*
d = (,I- , rL) we also kyroru t e R,
( \
,ts-teleryrenb "and e,tcnr,tnt o-2 e K+ w bro)
ge r^s nsri{ter, a,r ,il
1, nr)
Tra,e fuur.honat futm f
We aLreod,g d.tscrolA@d vsh,ere onoL trt bet Eo tlte gr wctor
1 ryf
ie the (.r1r a-2) wctart toetongs io JR,x {o,oo1 ie tlui u
e
Ine mrdut A .'

( rB. , (N 01,n")),yn,
) e & Xb,a)
)

6 {t Xr,Xr,xa.* xtuH Mffi#@} A)r, e,ra) f* Ssne unkown

,/ € R,d
N\oUe, Jh^at lrerg hle cu€ fakng orbotrt a
d drrrrensioraL mtJhvo*abo
Ao Xt, &,Xt - -. ... -X* aeLe ,ronAwt vec*tta 1atgtaw
4l*, 41Ac. vaxiana Sro,no6x ia aLreod,6 tfu,fd O, tduth!
i:;:,:erutonal
T6,- rartdorrt vetlor Xi WiA be 0- VecJw i,yt, d dtmentioruAo E = _&d
Iyr," unknor"vn, pavam,ebex tverv u il'te ont6 ?neon vzc;tar .1 aA +hn
s-2-
yutatrix {^ aLrenAX specifted,. ,t vedou iz ilselol a aL d,weruiona,t w,*or
{vu^odel tl

( #, ( Na (,,[ ,ril),1ewd)
)

we tut[1 1ow l,ooL" ub sow. +-xaynflu nor- yaror^ulric wrodu).t

d
 3,5
l) Ue+s t{rite Soyvre Statts+ieo.l. Modet.e (Xton - poycrvr,\e\ric)
Xrr Xz, X3.,. .. x6 € IB o.re itd h'i,+|tu unknornwr
uruwrodal. pdf
-Nfobe : A unimodo-0 {unclton inoneases orr (-o,a) the d.ecreaser on{(a.ror)t*Srrnr
valle "J a >o
Ge sornptq space E ts f,n n .to be -B
1fr1 U he,v€ ts *rn&y. I+s {he pdf + ilserf . becauAe arnly r^*lar
.parometev
r,oe icnoyo the exaat
J do r^x u.(auo +r^e {*n.ir"la prrri ts Crl *he arcb.,bufdrn
.,i'"
H-i
-The
ml:fflni#:*.; {TilT iffix Ee

poran'reLr sef O is the eottechort sl o\L -fum}toTrrg {hat can ba

aloweivev its ts uwtmodo.L sD
f
5i*en f {"t lik*e :

4 a o. d,

P'ne cue{ 6 = { l.* unirnod.al N*ma''sed !

{trnc+tans
frug ywodel [rlor,.r be-,ovner, ,*ty nr.r-e,rd -{r n:rtfe iln, $r* of a. p"iy

(n (
' q )f n Lee,qruuvumo-datnorwrolised f,^.t-,^t)
/
pglrtbukout
f,wn the BdJ ,
f
Xr, Xz,Xl - -- .. .Yq e [o,tl ore iid. \^siHA uvrknohm tnlnevttbte cdf tr
E 'b E *n [0., t]
Portorrte.tir+ rs futwrkbte cd{ (

Q : gek s+ Gtt invertibte- cdf(

(fo,tl ,(Pr),
n se+sl ou rnveqlibracDrfunc+r*.)
 B.(
V firv++ret L>+am{"o
l)near rc4rc.ss\ott' modeL
(XrrYr ) r,.-..... (Xn,Yr,) are ttd {ta'n lin e-aY wrodeL Cre'gr€AAldn)
gd X IB
" Xi'.r-tlrfo,R)
Y[ = ptxi + Li" , t\ n ]i(o,l) {or an unkswn ?*F.d;
Here *taa ra,,rdorn vaxrqble is
i*se'l{ a pair (Xi'Yi) , Xt eRd or-rd Y€JR/
X1 ,rs a d divrztavrstonc.L <ectrr Yt ds N rngal no -llnot dapayrds on Xi kxrt t^
otgo r"artdo.rr dtfe fu *lal- ti (vrotsel .3o f = ndXB,
fie povometer V here. is B tu'ucft ,r.s o. d dimensionoJ wcw i. pn-Rd
I

5o O=F,d
-t* +rr" lPs <* ts *he Jourut d[s]ribuh6n + (Xi,Yf ) ie a 3 a"Ldinnna^iJ
4p
. Nolz on ty -lhat we kruor^r Yt deryends svr, Xi
$un clrcn
So X; ^ rf,a [o,Ia)
Yi ulrU deaoanfl or\- X{, sD fe+s {txst f-tx Xi = x
Y= {x -r-Li 3o EtYl =
fx
I
\,
4t is & o- constu& vax[Y ] = I
o^Le- Xi x 4"nd
9a
Yr ")a ( p'r, L)

Oox frepor*uo,ta[ *fazarcL rnodeL

(Kr,Yr) ., txr,yr) (Xn,Yn) u gd X R, '. (ne osndidonal dislriu*k*'' r+ Y
s*n X = re hos CDF F d +rre .term
Ft+) = I * u*t t- J*a*re(f,x1 d*)

E ,, rdx r,
V t\eszU )?, h)\
!
-Rdue,tor unLnohm
{unrhor

o; KA X i. Atl pa,rtbte h $nc,+t*,.]

fr= YF {* f t*n a borre-
 2,7
> I-ar*ifraEtut u-
{o mor,ne {orroarrd, .we-
Izte U{,1 t-ror,,.: ,#u" decidecl ho^l-llne
h,orne w obsevaHo.tc
d"qssu*P[31 frh,o.t)A {rry so-hte.
X1, Xz, XD - ---....Xn ct-vt
'dt+l'L
C{**itr,^tt-, Fy we cayL ca.LcrrLo6. qrofuohi6+1lA about Xi . T., do *lnjs rnre wdl
eshyr&ate, g,; valge {om 1*tz cla4o- rn}e wUe'ctred '
But suppos. r^)e Eet sorne valu" e-s-lirnaLe
+.9 tf I daUa. qn d'rn:e h.rrw choosevr,
astlr^atoy t?r-ot it witl men€, and rn6re. prectse!.t6- o*
"""" "ln
vr.te tn.r.g6gs tlr-e rrurnber ol, poi'tt3 rL
v&,av 6euavankees +t^-ot of""t' +'nd s r'rre r'uttt
* qw i 1:11:.-9: :X*l
{o tttat* par*icuLar gr, fltfn all, vrle uuttt haw to
q6sla}ex' prclca}rrisHc qurattol a
Tegavdi,v.rg Xt an^d *lnose uwU be cmsw *^ *tvra
tru dis*zilrirthrlr+
fun'c'fcn /
g
1o evtsure -thjs ,tesue does
lrot bothe.r r+ I^)e, want uderrfufhbit\ '{orc
Iv"e garamebeY g ts calfed tdevr,tt{tob\e itt {lee. vrrap rycO*lP,4A
.twjzcttw,ie, g +U, g, Ip_ +ps,
gr e-1utv oJ*ilY

V=d
9* *J 7- \r,an, i$tnLie avrnsutt o[ a,ata { +9.:,1:, e-s\t'na?e' u4n" larnti-fi.d-
+z cornect and l;+" Fs r^t}6 i't 1 uaw teicnl^t^b-u&d
Exarnple
xi = il (Y" >o) , y,,y, Y^ iid 1r.1, .'r) {* Aome ,{ e K
Xf {s o. lerrraultr its arr in{tcafrrJunc+ta-rn (r,v) ra )o
^s
ELxi] = p in
^* Bern(p) ao
here E[xi] = E tttvi >,o) ] f (:t >, o) -L*-4t'
= z Jlo t e ,2rL
o. rJzl
v..:lruich is v^te,nd +o {.ha -,1-oUo t^!iy ort &

[t ts ^, (Nl ( o, t)

o
1
\0e lzyrorO
v f t Yi )o) 1- F(yi 40) =I - P( Y-r{ < -{ )
ff'
* L-? t L< -t )
6-/

+ r- f r*)
So X; ^, B* (f t \i>o)) Ber !, -f, 1 rr
= t )
 Z,g
3o

XI - be'r ( r - 0 t*)) E Ber(p)

1 -6t,--rr-
P -\ rJt
Ss tcon d.o dds {
u.
g- 0- (r- p)

Norrc t (3/I cshrnali val.u?.i

+ F u etn6" *he abore, e1,^ahot
8,
!- u"rt, $^e j4 ; th-ts "rako . B\* .l1 an.d fi cute
s ^ot tr.Ji\rcduat|
ider'{r$bb\e .

r-- ea
loy.
c tI l,ft
hle ge-f & value
r
1= 1,_,
*d r= % *he
ralt o {a ond hence Jr = -O'(,_p)
q q
tJo'^sgwx ut = iI, t r = Z q ll stiil-
")-i 6t're the. sa.me
?oho L"lu l,/o =3A, =
e6i
v
6'T

tr Engvcise,s sr1 fa**t1 ixt>+tt$

(c) xi = Yi" whn^ Yt, Yr. , -- -.Yn N 1&(lo,al) -f"* 36.,n?- wVnouu't a'
B & iclenrlt'S'da)e
l*t's t o,l,u*!oXe cd+ + Xt
{*m t IP(xL%) : g,(Yzr<r.) F ( tvl _<^Fc) = IP(_la <Y( n)
Pdt- + Y %-
L
cyr oly
Jcrl'/t.r
Va,
_ i 1 d
0- V
Lo-
o
o (/\,
-,,,- a-

o ile ,L<o
{yCnl = Lo- it VfW* o(>r <aL
.1-
a.f f<x
Once @L ts knorarr,r, ExLu) ts ul^r\ueq LdcvtJr-l<red
 3.1

(ol) Let Xi= n(,Vi Zq/z) usi-rerr. Y,,Yz,Yg..-Yn ^9 qfCo,a:) F" so.me

unknovtrvt c or€ unobser*ed . G o- tdentr$able I

Xi ^, Bevn ( p)
rcrhere p = P ( yr >a/z)
q
i" dF 1
Jalz o o..

Ao Xt' rutBer(p) : Bern(Ltz)

Uy d.tn r+ rlt r^fi'{d^},nlihd- &
i+ e +s/
+inil+ Ys * Ps,
Lvrw ,i+ 4r = I(sag ) 4 != un",!,"
hL = z tso"p) * For----?r,Yllt,)
''11t+ 4;dtl^il,.t
aW '
tr\crrce