UNIT 3

LECTURE 12 : M - estimation

Hi there,

This is your pal Scooby

Following are my notes for the lecture. These are my handwritten detailed notes. I felt that Prof. R
taught a lot more in the videos than he wrote in the slides. I tried to make them without any
mistakes and cover most of the video material.

But as Beyonce has rightly said : 'To err is human'

I however DO NOT TAKE any responsibilty if you fail your exams following my notes. So please
check the actual lecture slides and videos if you suspect something is wrong.

Also tell me about it at : [email protected]

The notes DO NOT CONTAIN solutions to any homework problems.Only the problems and
exercises solved in the videos are included in these notes.

Here's a motivational quote

'If you can't handle me at my worst, you dont deserve me at my best'

~Every entitled chick with a horrible attitude

Here's a cute photo I googled

I hope these notes will help you too

Let's get started

Lec + 12 : M-es*imattoru TZ.L
'fte tcleq ol M-estma.ti6n {a a li*lte tricfg- so hte undevstanct f+ s\ovlily r{.pbysbep
> M - estimators
> lhe dqto"
[ost d {hn }trne, the orrtg tt rin6
Jt,gt rrle l.uo,rc ra doto ie- obsevva*tcnna . lrru
tlrre pretv,r,ts discusstot , os soon o.S rare hod slosevVAt(cr'ls
hle o'a^r-tmed
(i,n *6e rnodel ossumptdan) dn-ot .tlae doto tnoy /or't* Sehevo.ted ltonL
a porhautay $arm\y ol c\dstributtorn, like -N (1,<ra) 6f !xp(l)
Ne caflecl *he Pararneter o] ivr*erest V and o,Lt eur ttr,vestSa*tarr'r J'
es$rvrah'ons h.)eyp to t"^i^i A o,^ ctsse {o g* (ie true porame*er) a.t
possibte . -Iho tclea ntas oyxce-uue t<,new g* rs es+tm cte a t^)€ co,rL ol1sl^xrc
an y
luestror+ usim.{ ffre pdf / pr^t 4 t rr. X .t{ererue us€ a di{{errent agproach.
^ Laf Xl, Xr,Xg . . .. ,Xn be iid cqes { scnrre y'andozruectoy X
E is o,s usu^L the gaynpte space ; E = Rd frr sorrr e d,lz|,
h)e axe *akir,rq'the vnos* general cose, I t.rhen d:]. x ia jur*t a-
Yondom vovfoble
, + d >J X is a cotteclton q$ d vandorn warcia,bter
aYr671gqt1 \w a enLurnn up trruttL
hon: nothii^g vt;i-w vr,a* h,app*,rA
Let Xr, Xr , XB - . .. Xn,, be itr d rnlt +h sorne unktnowra. dtshibqH,oru tr
hre do ,Lot q.ssunxe ilr11y'ru"t ahouL r*re
etistrtbuh* 'r. Not6 +hay
X.
hle 0're nok arnerr. usln6. "+',2"
r.tn, g ..ie "l
pararnet&t L".f= *u
rnal<tv\4 r^,, qseu.t
pHols $r vtf,tat fa*i[ dis\rtbuHon doer p,be)or,,*o.
ttt
r*e
1 d
'Ut orre, Uo1ln{ ri, tax have Sorne yandoan otata X,rX.2,Xg. ._.Xyy
o" distribultor,r cElted -P qenevated ib, that,s it.

-Ilrr,
is 1 caru ask
tdea non: $to1y parornetevll.tat tvtteresh nne
luesfiot
tjke uvhal's lhe wwaft , varriartce., med,i6[, asT,luantle , Frobakrti{g
o1 X hetr,g gr.eatey*hayu o , ,Bkrare ne/A, qq-fl,r morv-ten*, etc.

fod occorcl[nq t the questcoru )h,at f ocl<,Ah]t,tl a,6Aoct&,e landlhe

{rue value
* ing fova*ryiete v afi t^terrst v',vltl be denoTed ta2- ,t*
trd tJ ,1 cavt }ce arry povometev itateregt , it Lan, ey€n be U iz
"J
v,noaJu if t u.rat,tl it {x, be uoltt crrne back-
orl,r pararne\fr 4 ahoi+^+,s+i.al
4o +ws potrnt a
u-li+H, )-o.her

NoU" +hat a\L ltne queshons, -fLke, vlelil, vox tytedhn . qqth monrevttelc.
crrw a\so be {ouna . a! rn}e hore *ne a:x-cict dis+rilcukon fe* #14}
sur daf& arrcl v ue url osltmaff- hem wi"ttr, P5 . L, uulmle

iwt
^e.rr';ro;\RA
)'t-a,st*attot v..le arc agproacrrrnl bamz problern' d*l*-h
 LZ^Z

8o rare horne ot/Jd porome[ez d interesk ,t Ne sog .rt e )'[ ie { A *he set-
to urft1'q1n orv porarmeter -t boto-,aga ' I^le moke
g(X, 1) : A vandorr, functiorr, te o ravrdcrln, vanjable. (raecnr.tae i( a Junctt*
a a$ X ) ovtd. ttot€ thakil,s a yondorrrvcruob\e
finc*tma {. 1 noFvecfor

Nith .Hl,ir. p .toss {unckrn hrs cALc,rrlat e q,$ exgec\attbra atrud cqll tt Qt1)

Q(tr '.= Et?(x,,1)l

*'ln-ot qq) ts such,'t1a"o,b it attafh"s ilfs
4pd w* war,r.* +o sort +ln-e {uvtc*ton
uNtQuE MIN\Muh4 ab -tf t' re

oY,,1ieyLq(t =d
is ,ure l,tayre o.partamfu, 4 iv,,tertst .r1.Wu *Cu !g
/So rarhat vrre are dowrg
Sag,tvr6 4^Bl ,in ordL +o t i" true gavawe-)cer lust r'tirum'tae
^d .d
qlrt,l"= E [ ((x,1)] -t^i* is tnot *t,'e wre.thod'
(ne wrel{,tod {s ,lot' *he Paramekr o'+
t*t::t : {u-"*t'n'
1, -t* is mruinirn'rsed
*i$"c{t-n
(X, ti Alcl^ 'that 1We ex pectaUrru +
O

ffi;il;f; ;-tt"Ltu 4rue pavameiet {n\;,,,r,,- ^.,.,^

hovc to {+nd. ? &r ^ parlrc''La}c ?a}.ovn'e'\er
bo ihe questtort becornes
,l trtte res.t

E xavrnples
j*t do asimple exornpte ',^" ha\^e 61 simgte rnandorvrvariobleX
Lek $tvSu
y\re harne a ovre d'rrnets't-lctL prameker 1 e 1n
hlhjcl^'A
E =B ond
otro-S*^b be & ' flence E=Jut=$'7
or.,t *lr,tr 'tunch"n
AovnBvwQ. g6mcs up {o us arrl, sa5r, fl**y
((*rt) = ().-r1) Y:ceK& tcts
-tl^rs tuwc.h"iz e>tf.cba#'sw
+\d ast<g \we, \^)l44t 1v,u,nl,,ruses
\ree ul|Lat '^"t^^*'sesu E [X-'1)"1
1
w uoVs do ib
lzullose )Me
PA{ *
X 'is fC*l i (*'e could h'aw'loket'r' thc pmf )

E [ (x-{)"] = tx- ry' {txl ax

J
e
 12.b^
Nole (x- {2 is o. conve)c Junctton * l, ond *}te E [(x-r1l'] [g also conveD(
.go vrle cc.r^ j..sd
pi,l'rirnise i+ plHtn{ ttre devtvattue = Q

Qttl = E [(*-t:"]
q (t =
JE ,*-t)" {t*r
d,c

t E^t) = ? [ t.-1rz
t
jcxt dx 1vr. tr,.le qra-ltorr is rnvr| rt. so ta.re
o1 Q\Jn cah- {ake- *lte ctev[vatr,re inside

G8(rl) = t (r.-
^P'frxr dx = Arr(-tt-r){rr.l ax = ,J (1-x) ft*l d'x
a1 a1
-Lt J
E JEE
'lu.rg tg ,1*"t 1, o at {= f
dx : o
J,,i-x)ftxr
Y

,tofi*,,a* -Jrc$">dx =o
ttr-]s ie I *{nrs is E[x]

tn -Etxl -o ,t+ = E[xl

uJe did vtsk il.csd, {b evew go tnto -the lu^tegrakorr step

qt) = E [fx-11'l
QQtlr= votvs cm,npr.\aftort
I xlroctztltsrn
?rIrx-4'1
01
in
'01 s\neY X So hse can {o.ke d.e.riva}iw
instde E [" ]

8,*'.'= fitx--1r'l = E [*c*-{r-r)]= -r.E[rx-1:l

E

{tug ts O o*,
4-- t*
E [ *-{l -o = r[xl -{ :+ {= rlxl
8, t?.e J.rr,"ch,on a({ E [t*-l )'] ts vnirurr*sed. ah t = E[X]
 Lz.\
Noaf rue rno\ne oru *o ll, e vuex-t eXample
Here E =Bd ie X tS a d d.twrerrs,iana[ randorvlrector r-[e a c'stl.tmvr.
hxth d vandorvt vartalc\es 0.6 its elemenla
Tt tg atso tr,.^ .tfrdb ,1 € ).,1 and It= Rd 4e hhte paYotvne\cv a!,
iwtevvsL is "6tso oL rrech* (dx t)
Some ovle curr\eg 1o trs oncl eaysrT girneyotr *ha func*ton

g (x,r1) = ll :c- tlli {or aLL xeRd ard ,ftKd

/,nd ast<s me ,r,?to* -l1 wuir+inises *his $nc{t-.t's e.xroecto{torr-

iJ uvhaL ,t r,vtivulnaists E I lt x - tili ]

hle wrrlst
A^ urder to solrne rruhat lilTe aboq, eltlahon YneansrrccA-rr
A,tn z
under stoncl vuhat lt z lll : *hls tevvrn zrreans frr sorne
Suppose $r rhe soke d. exomfle z ts 6.3 dirnenst6nal rec'lur
y:
Li I Z, ] e K3 1:,',,e q*T*,+g ig OueLideaw
,llzt\rc-ouedt-rvrcrrnn)
Trlteathe
L?l hor"nd z (a\go
1 tsoaicalty ils *he concept o!
nZllr =
:r2
L.1 +zi+ s.2
Lz d,istottce , m mtg'n i'tude o*
rrec{ar tr,r. 3 dirnenstcrns

Irzltl is [ust Ue s1uarc ol e xP vEsgt6n

"Uo^,^e

ttz u| = 2,2+ zl*4

6. cr generaL d dtvvtevtstr^ol co-Ae

z=. r Z-t

\z eEd
L
4
llztr| = z?nfi+zt* ' ,i
!6* 0.s \Ne lrtnov,c {"r cur probte'nr, X (vec\'or) e Rd
-r1 (v€cto/) € Frd

Lzts deno(c
(r) ?tlase dona[ crrutuga

t
X ,r{r
X {r as E [xtt)1
x (2) t
^)
6

a arrd so ow.
.ocl
lytesz Arc
a lu.St
(d)
Vaxiab\es tn &
X 1d upuntttt yton {hevehar,
been no latk { f [X,.!,1
 12. 5
3o by the de{r6
"l LEuct"ideaw nsrrn,
p(x,{) = ll x-1ll )L = (>(rrltz:ag xd)T
,^t",tn -
2-
,t = (.r1r
1o)t

3(x,1)

P[X,4
-thtas a yarn-dorn vaxiable noL a lneel'raa
,.t )'
*his is o. {un"tcon "& -1 = (1r t\zt\, " -

Q,(rt)= Q(r,1, ,\,,{r..--Ial =,E (e(X,rt))

reol valled {uncton
e(tl is a rrur[ltvamab\e
c{ clivnerrsi.snat vectov or,ttput'ts a YeclL valle
i; { 19 o. ,
,go *re p.robtero t,row %oe-orrtes q
y!^ir ir^irl^q. o d d[vrnehs.tsv\a[ Junc]i*lQ(S
{.om otrr +herr.errt obouL eXpecta*t.st'"{ c,arrve:<-;tuvtcttarr" So
Ne kUoW
tcvtotr:
fale Q == E L ll X-r1ft|] is 4,6^\^eX V^uence has a" unilue minirruizer

h/e op{ain t't by aimpt5 Pu+iq1 VQ(1) = o

o I-ooL aI thc exFressi6n ol ptx,*1)
#n,Qt,.t' & rit ,..le Jtna +vr" otpcctatcern
I
o11
q t1 )
2-
0
; r Ie(x,1)]
I = a[tx"I {,i]+ e ftxeJrt"f j+
a- a ({
ald o
@ ,.t= {.*
r I t x(dl rtr il
[ ? ( x, 1) I wrt 1P soy oll -khe
{r
/ro { \^}e eLi fJeretn"ttorLe, E
rt4nar ter rns hJant t.,rtatter as *hey dor, t havr teYms t"$t+h '1s:
tru general {or ort6 1_<l < d B

a Qt,t) jt
arli L
a. E [(X(ir-
a.tj 4 )'] -28 t X
rjl
errch c,ornpcmeraf C VQtrl) ts zero at t= t*
 t2"6
8 hrrnu \^)e f.*tty ha,,ne our ans1^1ef

Q
(.,t) E I e(x, 1) 1 = E I I x-,,tll'^] is,r,,,ui,u,vrised. at
(r)
E fxctll X
t+= E I xtz>1
^v?n.te
X x(z)

x ta)
L Elx("1)l

Nonr tn, ttnjs example X tg a vandarrr variable eR ruso l,(e ts

E = {= R Sorne one gove ut 7(zc:1) = t".-'t! Y xets -\e ts
'Ttr,e

L?-t
t pdf
{hat q(t) -- f [f x-r1tl is {o be f *r.r.d
vniniywuizes
x be {,*t
"J oo

E I tx-1tl J t*-1\ {(zr) dx

-@

aUus (!.n be pased trntp *tn:o cri.mpo.revu[a

de csan
r1 avud **. 1
rt't\e & funaamcntal
After that 4a ch port carhe dtflerc.nttohe ustnq grocluct
*lneovzwt 4 caUluLus . [Tne di]ferenttatton
ts wrt' rt )
tf Q(') ts u"vi{ovwrlS c-orrtivtrroxsrt [a'l']
Qtyl = dt o,,.{r" 4i{$cr^arcttabte san (a, b) and xe (a,b)
J* &tt)
Ou

Gen
q'(51 = E,gl
t w,,^lt be i\ a.s i+ is u iw lea+ure exeycise "

t^," nrue d[an, "&

E is the func]twr, *'ia'ot vrti'nimirer Ettx-{tl
utodich, is 'the 5oZ luavu{ite tlrle hgill
n-e:ct tee cr {unr*ton certted,
Tw
$nO any luon ltle Jr"t r
)dna methoO
cl'uecte. {uncli or- 'lhat helPs \la
6l M es+iw\ aticrr,
c
)1"* t.rre o^goin !"r-are a yarndorn va:+ioble X e 8., , ,t g )'t
r ,rt R, P(*' ^t) = Co (*-t) rnyhe*-e

#,.TI &
-(t- x) cc i$ rc< o
Q,t .> ig ca[ed, t?ue' cdzcL ]u'n"c*toa' d"X 'tf >L), o
 12. 7

- (r - *) (ec- {) if x-{ <o

C
"c
(x - 1) wttt be t o( (x -.1) is x'\>/o
@

EI c*tx-t)l = Q(^t) -- J c,.(x-r1) t,*>

-@
uyhev.. {c*l -- ?At 01 x

be decsnnpsse d in{o {roo pav{s accqvc\i\t

*:^it ivtie SvaL Can,
to the d4 t ,* C,. tN - 1)
rocluct rulp, and

aYt)aL )*tzorem CoLcrrl,u,g , lnenput to zero

functav,n 4
d AHt 9) . GcSl + H(9) ' $ec1:
aX
(Hrer Gtyl) dg d3

r
tl
.l t+ qt') 'ts tr^^t**! c,cr,r*inr.rou.l orl [a, bl
d,t
Qt1)
JJ =\ 6tt) ara,d difle-rerthable qlL (a, b) arn d :te ta, b)
tu

tl ) - 8(*)
el D'l'Y iti a iw 'tec-tuha ?-v't><sge

pr,^t o^1,1" -Lt^ia hle geY

,1*,
d v
& Jtx:
dx

-@
1*
 12" s

> MLE : an- jwL esl,rnato,,c

3o v1orc t! rnle Ee* {he ideo oJ. M eskrrra*ts,n- . Let( ve,membo+ "'&"o+ "'"<'
clic{ eoxti e.v iw c.o.se o{. "nH.rrirnurn )r!'<eUhood esttn^attsn'
-+ N€ computed \(u (lP6* , Ba)
-> vue got KL(Pg*, Bs) - cor''rs*avtb E+[u6- f*rxll
*hat yn ,twuuylt ,silt b& ^!'
I rce v\^ rrrrtised -flnis {ur,ta*trn c-!'aim\'{-{n"iswi\l
= g* . (acfuaty hle jusf ctaime) *tnab
O happen)

+ Flwvatentely hle cavr, 9o1 hre rnirruirvrised

- Eu* t rtfr(x)l
+ le ure rnzti rui ra,rised - E L U, ( X , *) I
-+ Cf we See +f4is proceduvu ,Iot^ ure cavt r^e{rame the {uer*ccn-
h)e harea vrnoclel (E,ttseleeO)
{$ {=O & 4=v
A rue Jnoue the l.osg fur,r,ch"n ( (*rrt) = -L,(gort)
bJtruat is lvw I $na'* *vtjrTiraruiser Q ({) urhere

E [-t-,(x,
a fl.fe
1)l
Ot{gt/$€J{, dS
=Q(1)

ffi
ttt,* dd"A oll -these. caLc-,ilaHfl9, vn arno- *o reaUss 'll4a+ 'this ^rr',r'-ld'to1 *y
1:+owcle
sorl^e 'tnstghls 'ilnerrittcattS v*r,at goool u it i{
{. aorr+ knor: P
becoWe olt ea\orlo{torrs truc did {or Wr€-c"rw, rynediana
the l,es cara,
orrlg be conrp,trted knouring 4te dtstrib-r.rhorn IP. Bu* 't{taL lr 41rue v^aittu
\z,rolDtem
4 cto not knoul JP. -fvrolf n:\ 1. am doinn e-thrr,r-d*i6v1,s.
,{ko iI you nol-tce vxevle.r *ravvr +J,* bqr'n-irp
-t{ruf !.et\uvz haw we
j*r og es.[r^a-[tovrr . fe. r^re l',ave r.n,ni jorrned"[ *n1 o*):"w p.lors
. ltorn
*Vtes.^Pl€, dcba X,rXerXl----K* v\>z lrr'aw'
v{etL llr*\ i.s absrrt +o cwouge, , L i+
,"ntL 'truvotve our o\d recepre
{o--^.

K"$tu*-
\t
{*V.l,o{r'rn-,
h- A""Y^X* i
 tz. q

> )"Snrdtort
we hlrtl cted rne Nre wv ts*trv"ator -i^ o"s 'tl^-e minirniser
4
Q" t1l ',= tZ f(x;,{)
4[ tf^. co-Lc.u\aUoa, h]e &Ld b** ffw .T.'{h"is*tv
t,uttl tlsr- nesulli
\
taut r^)€ trge +he roorcl eltpivicaL' as La\cuhtto'rn done rn sa,vrpte
.Eke , e*?*ical rned*an, e*?tical
QxL
evr,r,pirucal vneon luardite

We n^oy vtsf rreoti[ but et -l-oL.$ anotysis]l"* *T dsrre Jor is becanrse

tlae' case
-ot- -'LLE es+tyna+ovs can be lvansl*:tecl
here {* *.^ changes - Jrnis
ease ol M- esttwra*irn t"vhere P = -Li (x,e) . !o tlt light "1
flr;;;;-;peclaL can tttkitiuety predtet the pvoper}ror
{lte pvevierr.s clisursstons h}e

I l) Consistanay
ep,mew\bzr {or
hle dicl -lhe cose + MLE . \'oe had" KL ( &*, Ft)
lho.l had c^. rzr,tntmuvrrr, at = 9* . VOe "?t li"d *he $rrvnu\ato $vd
'^$h^+
O

We drd'ttok knor'r: how *o raklrta&e

*ttis
KL o brrs{ant, *t Je tn 1
%f d,Y eg{twra*[dt gtrat"'EY
*.$out p+tor kvror,,cted,Ea'; *+ ) so ule invoked set out b rnninih^ise
ttasteod ol *r-*"tg- inr* [1"? $o(x)l T
-*Y\,1 tgUd'{ntx;) L, L.N .^Jr*a }t..t +his iwdeed cs,nverged -tvr"
i
'{o -Ee* [ lt ] 0\s irs i*t an oveYtr{t +
probabt uly $e txl
t{^rft 'titis conveYgence betr':eevu
yorztdom fuwctorts. We ilr.u, ctaivrned trcrns\aVcd tD {te c,ovrt'"vgat*o e{
)fie, luro ij t ;; {r^"+t* , can be
; iw the x- o:trls. ktevue t^)€. cwrctr'rded thor 6Mue t^'os
,T 6
"GAs+tvilo.{pr + 9,*. \de ?i\'e tlrc same aYgumerif,s lrLeve
g, csrnsisteht

,*+ 1,tl; esltrnate q(1) by Q"t{ = X ? txi,t)

*
r+ ql (Xi, t) is just a ranelsryt {+rnc*ttrn ,hence a rondcrvtvartr'able
e L.L.N fttk us * X, g(xi,,1) # E I e 1x,n)1
{o ?cexts
r+,iJ^is Crnverg evg-. t^ y axJs {uvtcl.tuns Con be {rons\obad
undey go me {zch.rn-icol csndittcrrrs . frnerreSrc

+
A P
n+ oo f
 12. {c
It) No'rnal,ity,(syrngtottc
NorD r.u.steo.d, + hecalLin3 urhab nle dicl {r, the cose ol Uue , ,^4) dont we iusY
rrepeat,{o sse rnrhat l,uoggavtS

- > t^e 1:iyst step tnvotved {he. exg.msitr, ol g/txf ,{l axound ^t+ Jitt
csnvinievrce rnr rrtl\ use qlt{t = g/(xiri1:'r"* +^" raytcrr e,xpangisn hos
-{o do Nttt^, Y.V. 8i
"o*l"t^q
eit{) = glrt*) + (,t-{:g/r1*r+
-l
Tcrtce su mnaa+ttrn bolth sides t ott.g that 4hese toYvrtg tEltoreo\

*Zslrilr -o as ,t n r,iwivrriges *he Junctton Q"trt)

{tso E t git{t1 =o as ,t + is {he vrninirniser ol f te (X,tl

,&o {he aborne eluakcrn botnvnes

*f.qi't{r =Tfltitto) +t{-{' ryt

n\
p i"(
t* )

-lru's will be rn)e hlilt add

zevo4@ qnother4erm @ here calse i 6. zevo
,

O = Jff (*x (t'{r rrrJ,{rl) + Sri,i: z qit-,f I

n
t0e Lwvoke ouv $vmrvlte
]rcdc vLot/o
-
CLT & L. L.N

97,'" .In"
(* teit{r r I gi 11*11
;:}'n ('' Vuxrri{fl)

LrN: Lt qr" l1*r ,--IP -r, E L gi'Crt*l l

rt Vt4 oo

+ tt*tp ft aff {o-gether t^'c wt\\ h'ar't'

1ltr,thr$

+) [ (,'rr1*11)
"Ji [0,
(d)
JA( ,t Vqr(
A

^t n4o
E t ?;" t't*' I
Or ivtothev t^lords

.tfr, (t -
1n)
(d)
lnl a I 0, va,r l?{

EtgirL+)Jz
tq*l] i
 -l a .)
Ir,. 1I
r--+ f,6 fis ts rchat Pr{ B. tatked otcoub 2 h}e see +he vq}dahce {ern,t, t}ua[ L .,

volt [ ?t '(1* ) I raf,tevr iw case -t'4LE I^]€. tT,ad,

"]
E Iei, ({)1" ? f.% u, (x, e)
So ule Vr,ao[ t'tne, ex prtesstott

-E[ ?'7 = Var I ?'l , = a(o)

tt *" 4n:e l'\^d o.
ice cr,'hcellattorn

velr t p; t^t+) I f (0) -t

{.('s)
E [?i',(rt*ll' a.(e)z
-br.tk {tais worit be 'the, ccrse fov ony levrevat ? (xrt
j.d ttr. de$vte
,+ 4ro nofi, ttvat 4hjs dortvafio,r J* td iw vr,tove {hovr"

The J and K matrices :

Let X1 , … , Xn be i.i.d. random vector in R

k
with some unknown distribution P with some
associated parameter on some sample space Let for
∗ d
μ⃗  ∈ R E. Q (μ⃗ ) = E [ρ (X, μ⃗ )]

some function ρ : E × M → R, where M is the set of all possible values of the unknown
true parameter μ⃗  .
∗

Then the matrices J and K are de ned as

2 2
∂ ρ ∂ ρ
⎡⎛ ⃗ 
(X1 , μ) … ⃗  ⎞⎤
(X1 , μ)
∂μ1 ∂μ1 ∂μ1 ∂μd
⎢⎜ ⎟⎥
⎢⎜ ⎟⎥
J = E [Hρ] = E ⎢⎜ ⎟⎥ (d × d)
⎢⎜ ⋮ ⋱ ⋮ ⎟⎥
⎢⎜ ⎟⎥
2 2
∂ ρ ∂ ρ
⎣⎝ ⃗ 
(X1 , μ) … ⃗  ⎠⎦
(X1 , μ)
∂μd ∂μ1 ∂μd ∂μd

∂ρ
⎡⎛ ⃗  ⎞⎤
(X1 , μ)
∂μ1

⎢⎜ ⎟⎥
⃗  ]
K = Cov [∇ρ (X1 , μ) = Cov ⎢ ⎜ ⎟⎥ (d × d) .
⎢⎜ ⋮ ⎟⎥
⎢⎜ ⎟⎥
∂ρ
⎣⎝ ⃗  ⎠⎦
(X1 , μ)
∂μd

In one dimension, i.e. d = 1 , the matrices reduce to the following:

2
∂ ρ
J (μ) = E[ (X1 , μ)]
2
∂μ

∂ρ
K (μ) = Var [ (X1 , μ)]
∂μ
 12. r2

,b $wo\\y \de can soY

Led ,tu e, I,.( hue poromeler) . A,sguvrte the fotto*"*
(+ne

l, ,t* ig lilae onlY rntnirnis u' 4 q

2,. {,*t) ts tnvsvfi'ble S' o[ t t )'t
Z. t fevo m ore *echnical condi*trnl
1,u* t" sattsf ies

&vtsis*ency
,t^
A P
t4->6 t )k

r'rcn naaf]Y
{s5^np*o*ta -L
Jvt (
t t +
)
rd)
w+ q)
i(., rtt) K(t) (,{ )

horre lhe dis+ritor'rtan- of

6lL huiLd, cor$ae"tce irntervat $t'(+ "t T
ll911ce ,nle
{lae probtern 1n}ewaL fr dependino oY1'
,{^ we wl\\ (Corr) agaf^ {ace "l
eth"ds
itse\f " t^Je ^:tt\ the I'taw tlaree
nT

ttne pa,ronnetev to
ro5ts "1 t
t. Gotu. : Sotve some eluot{ton +o $"d
'- f^voke some lnel*oti[+hat boundg I
,-, tuouna/c,o'rserva.*tvg
iw the vcrrionce berrru'
B. {Futg.irr' i Sdrstih'tte t
iI^' lLace 4{
 72"LV

F Robus+ sLattsHoe I
S uppose we hovg some observotton' =.) !W* a stanclard elorr*iar.r"

t -r.8, 2,a, . qB, . g, -b-2, L.o6 :'OB !

lne meort here ts fokatsum
=
.31-
,/ 3 'oot4
7
Jhe rnectian, 'ts coleutdled by ovraqgilg -lta.e rn-umbevs i4
qscending ordac
avrd {+utdtrl" -tlae rniddte one

t-r,Y, -1.5 ) .oB,'zlB, .g, {.oF, 2.3t

4nu l.vredion ,ts -l,/,ew .4B
l^reav\v avtd wre&aru q.r€ ctose +o zero (o)
^{L.id"t
Now te6 so) hle gei {too nero obss./valio4a L2,5 arLCL 14
lfrr's cayr be due to some \P" -the obseyvey naade
Tr,l.is cau\dtJ.co Le due *o fact +had 'tl^ese
1* r^"?v -lou: prolcabitilr realisahorru

+ Qarr,r.l,.r,ar'u : thot hre toF u"$"tuv,'atel2

(ne wleorn {"s = @- = 2"'1t{6

I
Ttte rnae d tat'r is t O" 9

t -ts.3 ,-1.8, .oo, .48, .g ) 1.o5 ) L,Z ) 12.5 ., llt t

Not^o See 'the reotiza*rrrr \^)ere {."r., N (o,f )
1l* yneovr, t* O

TVr" y\^eah chanqed +D 2-,q,46 {av at^rqy t*

T"-., .ool4 O
g{t\\ close *o O
fnlhile 'the *ecl,iai changed l-"'" '4O *o '9

is Se crr^s" vvtedlow la a robus'k s'lrrtisic xl,.ile h4eon n ndL

J6is
,, tf^nr* a,flect 'the rnem, 3r""tb r;fvdLe lha meAian is olot qluf."i-
"it.
wtr,r,cl,r, q

[orne]tmes y ou J,ralr,e *o d" rco[ust es'ltyyr,o*t on . Le\5 louL ak

Couchg Eosln\"rt*
d
 lo
ll-.
-1
I
./)
I

r,r)e next sfudy the &.,"V clistrrbutton Tr.,e rneon is not

^'ill
deJtnect. for *hts dts+riUutictru x>e \^r-t\t see a {+rnado'rr
cotted Huber/s Lrss -++1r.+ hetps us \^ery neat\y
,,oe w1l oFfrJ Hubers Loss +o ovr ol{re c\tstributco'n co\ted

Touble 6xgo,.url+.ot crr l-optoca dktribil"ttLor\ '

.'e use
1l. t{uber,s Loss $r c^^.u {s dcvne in futrriats
"t look
Bufo* q-t rNe \^)il\ see hor"r the $rvtchons
$rv.royc\
It> Cauchy distributton
' la> Doubte exPctnerrhoL

\ts) Huberzs Loss

Let's do it, '.
 Cauchy distribution - Wikipedia

12.15
Cauchy distribution
The Cauchy distribution, named after Augustin
Cauchy, is a continuous probability distribution. It is Cauchy
also known, especially among physicists, as the Probability density function
Lorentz distribution (after Hendrik Lorentz),
Cauchy–Lorentz distribution, Lorentz(ian)
function, or Breit–Wigner distribution. The
Cauchy distribution is the distribution of
the x-intercept of a ray issuing from with a
uniformly distributed angle. It is also the distribution
of the ratio of two independent normally distributed
random variables with mean zero.

The Cauchy distribution is often used in statistics as

the canonical example of a "pathological" distribution
since both its expected value and its variance are
undefined (but see § Explanation of undefined
moments below). The Cauchy distribution does not The purple curve is the standard Cauchy distribution
have finite moments of order greater than or equal to Cumulative distribution function
one; only fractional absolute moments exist.[1] The
Cauchy distribution has no moment generating
function.

In mathematics, it is closely related to the Poisson

kernel, which is the fundamental solution for the
Laplace equation in the upper half-plane.

It is one of the few distributions that is stable and has a

probability density function that can be expressed
analytically, the others being the normal distribution
and the Lévy distribution.

PDF
Parameters location (real)
scale (real)
Support
PDF

where is the location parameter, specifying the

location of the peak of the distribution, and is the scale CDF
parameter which specifies the half-width at half-
maximum (HWHM), alternatively is full width at half Quantile
maximum (FWHM). is also equal to half the
interquartile range and is sometimes called the Mean undefined
probable error. Augustin-Louis Cauchy exploited such Median
a density function in 1827 with an infinitesimal scale Mode
parameter, defining what would now be called a Dirac
Variance undefined
delta function.

https://en.wikipedia.org/wiki/Cauchy_distribution
 12.16
The special case when and is called the standard Cauchy distribution with the
probability density function[4][5]

History
Functions with the form of the density function of the Cauchy distribution were studied by
mathematicians in the 17th century, but in a different context and under the title of the witch of
Agnesi. Despite its name, the first explicit analysis of the properties of the Cauchy distribution was
published by the French mathematician Poisson in 1824, with Cauchy only becoming associated with
it during an academic controversy in 1853.[2] As such, the name of the distribution is a case of
Stigler's Law of Eponymy. Poisson noted that if the mean of observations following such a
distribution were taken, the mean error did not converge to any finite number. As such, Laplace's use
of the Central Limit Theorem with such a distribution was inappropriate, as it assumed a finite mean
and variance. Despite this, Poisson did not regard the issue as important, in contrast to Bienaymé,
who was to engage Cauchy in a long dispute over the matter.

Explanation of undefined moments

Mean

If a probability distribution has a density function , then the mean, if it exists, is given by

We may evaluate this two-sided improper integral by computing the sum of two one-sided improper
integrals. That is,

for an arbitrary real number .

For the integral to exist (even as an infinite value), at least one of the terms in this sum should be
finite, or both should be infinite and have the same sign. But in the case of the Cauchy distribution,
both the terms in this sum (2) are infinite and have opposite sign. Hence (1) is undefined, and thus so
is the mean.[12]

Note that the Cauchy principal value of the mean of the Cauchy distribution is

which is zero. On the other hand, the related integral

 12.17

is not zero, as can be seen easily by computing the integral. This again shows that the mean (1) cannot
exist.

Various results in probability theory about expected values, such as the strong law of large numbers,
fail to hold for the Cauchy distribution.[12]

Higher moments

The Cauchy distribution does not have finite moments of any order. Some of the higher raw moments
do exist and have a value of infinity, for example the raw second moment:

By re-arranging the formula, one can see that the second moment is essentially the infinite integral of
a constant (here 1). Higher even-powered raw moments will also evaluate to infinity. Odd-powered
raw moments, however, are undefined, which is distinctly different from existing with the value of
infinity. The odd-powered raw moments are undefined because their values are essentially equivalent
to since the two halves of the integral both diverge and have opposite signs. The first raw
moment is the mean, which, being odd, does not exist. (See also the discussion above about this.) This
in turn means that all of the central moments and standardized moments are undefined, since they
are all based on the mean. The variance—which is the second central moment—is likewise non-
existent (despite the fact that the raw second moment exists with the value infinity).

The results for higher moments follow from Hölder's inequality, which implies that higher moments
(or halves of moments) diverge if lower ones do.

3/13
 Laplace distribution - Wikipedia
12.18

Laplace distribution
In probability theory and statistics, the Laplace
distribution is a continuous probability distribution Laplace
named after Pierre-Simon Laplace. It is also sometimes Probability density function
called the double exponential distribution, because
it can be thought of as two exponential distributions
(with an additional location parameter) spliced together
back-to-back, although the term is also sometimes used
to refer to the Gumbel distribution. The difference
between two independent identically distributed
exponential random variables is governed by a Laplace
distribution, as is a Brownian motion evaluated at an
exponentially distributed random time. Increments of
Laplace motion or a variance gamma process evaluated
over the time scale also have a Laplace distribution.

Definitions Cumulative distribution function

Probability density function

A random variable has a distribution if its

probability density function is

Parameters location (real)

scale (real)
Support
Here, is a location parameter and , which is
sometimes referred to as the diversity, is a scale PDF
parameter. If and , the positive half-line is
exactly an exponential distribution scaled by 1/2. CDF

The probability density function of the Laplace

distribution is also reminiscent of the normal Quantile
distribution; however, whereas the normal
distribution is expressed in terms of the squared
difference from the mean , the Laplace density is
Mean
expressed in terms of the absolute difference from the
mean. Consequently, the Laplace distribution has Median
fatter tails than the normal distribution. Mode
Variance
Skewness

https://en.wikipedia.org/wiki/Laplace_distribution
 12. 11
F Hubev/s Logs
Ne wtll inlroduce *tue Huber/s Loss Junctiot lrr.5(x) , fhis is a good subs{iful'e
{or l?el as +}Lis fuvrc{twr con be d*}erenhated and drruble di$ferenhqted"
i^ contrask to l>cl . we wtll xee how the c[evivattw,s ld

2.5 t
,(.
f$ t:tl a S
2.0 h 5t,,)
t(rxt-*) i{ rxt> 5
1.5

1.0

0.5

-3 -2 -1 1 2 3

1.0
% it l)tl4 5
0.5
LJs tr.l +5 if x >6
-( 6 I
$
x<-t
-3 -2 -1 1 2 3

-0.5

-1.0

1.0

0.8

0.6
t 'T txl4 6
h"rt*l
0.4 o othevr^nse

0.2

-3 -2 -1 1 2 3

Here delta = 1 for the graphs

 12

10 delta : 4

delta : 2
8
delta : 1.5

6 modulus of x

delta : 1

4
delta : 0.5

delta : 0.1
2

-4 -2 2 4

This is a comparison graph of |x| with Huber's function. See how the green one most closely resembles |x|
 t2.zo

{vre doubte e etPo nento os des gribed ia 6-i'ne

n by p df
t eb
- lx-,t{t ,L (rn, b)
$t": =
rb =
her€" J{:= .l,ocattern garamehr
b '.= secile Porameter
Ne witl foke b= I Jor our catcul^ttsrL
Lets for J.t *, 1) co-leuloLa M-o>drnurn Uke[hood Esttm otor
Ir {1x;)
11

+ L(xr,XlrX3.,..Xn, m) = Jm
t

L e lxr -ryr,l .!d lXr- ml

_
. -lXn-ml
o<
1e
oa -?,

t
qVL
-E
gL
lx;-nn,l

- n logZ { tx;- rn \
) lr+ Ii keti hood' =
lD jh,t" exprrc,ssitrru rnrr* rY1. . on,d-
ane wext step is
ctaffete,ntto}e
+t,ot b.oo.,*L ol $.te q[getufe vaLueA ' But \dol-ltE nolr
hle co.n,t clo
\,rre har€ Une naethocl .t { - es*e m attcrn
axiu^uvt4, Jf ketlhaod es{imateozr
+ Suppose trrstead d. ealcuLattnl h
;
nle w outd Vrqr,e ' stated {h; problenzr' as Jottot+s
We havt soy)4e dtshd'b ut0on' 9
= rn,
UUhrfo vr,tlytivrrr,tsas E I eXrrn]J
tOl,rerf
L t)Yn) lx-ml * >t e-&
et = ueshan, . -Jl,rl,t tnc,r tt"e $*t
8, t^le kvtotrl +lrue ' cLrl,s hgev fo 'tlnas 1
exarnp\e ol eStrnaHurL .

median \e 50"/o q uatt+i\.,.e, vl'tiYtiYrn"tses E [ \x-wrll

t=
& t^yhr,w t.l& esttntate, it
Ilt,.re
Alo
,t = Er.npivicaL Ynec[ian ,h1f vtrn t30g .t,

lxi- vrn,\
3ut {h,ts is j*t -lt^p. mk-utcrtlsr,r *hat needed. fu M,
nrua2irmum ltkelihood Laso . \zwet l|rue )4cstwra+r:l, t* lx-nrl
t
aA hrcll ae {l're Mt E es*irraatov ,[ L (m,1) is
u ?Lx,m)=
mzd,.o,t" enrfric,ai .

Now hle ryrsl^e ow \o cotrrsfster,nc! & asymg*otc v atiatn ce,

 L2,2L
,b(oro voe r.uar€ jFourad oub {tat 4he esHmabr rvL t:s emV\rlca).
. As ik avr Mt-E es*trnotor "$
c$ rn co nsistencg is prety
{uanlrle
slrcrrgtut lfonoard
A P * ie Csrts&ls n"tu h
{tL ryL alrxttrc,d,
v1,+ 6 a

l^" proble rn ^1e. nyill ehcountey is wh-r'te e aleulotiLa1 -++12 asgmptotic

Vo.njayuce . we can, qo {he }41-E vcnrke to $^d &r.z f,ihev {n(ovmahot
'}4- lo
{" tvre
olr hle can estiyraahq. roube caLculabe K(m) & d(rrn)
Oot79 wil gi; *tte soyl4e vesulf as JyLlS is I speciaL case *

{ - es*trrt^}tr.r, . }0e wtil do eryr l es*tm atton axarnple os we howr,r4

I

dJtne anp - Le\( beqtrn, ) \Ne harre.

.ie ,* a*t \, os
oun" hel^e
) ; **, rnedian d -Ul" diA+nbt'tkon

h -
ts ttne egh,tato*ts lt'e evwpr-ica\ wtedtoru
t '"rf
Ne a\sokvtov.r ((*rrr^) = lx-ml here
'C^uttslst"^q b-__-__.+
F -.lr
r'tly', YN
n ----) oo

voe atso had a thenre,yr.r, th/-t Soid r

fi (.,1& r ll m+6
(d)
n t, , r=(l^u) K(*+) f,lr+r )
hlhere {tn,. ) -- r Irr g(m)] = EI q't-tl = E t d P(nn) l 2

dmL
K (rn1 = Cov[v Q('n)] : Vqx I ('rr)l = V*t I (rn)]
t.,f
Nol,o hle haut. 4h"z Vroblevvt
"r*
trn. {-*r* ,* ug

* e('vra) d lr.
= dm jru* l>e
t '-- -ml r.ntt, ca,LcrLqted. aA
I

dm
a- ctosed frr.ra.red Sotrttr"orr"
and so car(t be -.|*-re duuble
cler[v attle
hle are g{uclc . dr.r, ts rotn e.re, err s L-,t^ c*rrlz
*** ff
,trruto pr\
 12"L2
Ao tte harna e(x, m) = lx-rnl ond t,tle cavrit afJferenhate il ,be*us<
*ne funcli61 is naugh*y. C+ hqs a kink . And *he kink 'r,e prccisc-l;r
of tL= rrt rarhevre {:lne Jttncksvt ,t.s raoh eli{Jer+table . we &rc 't}.o*
\^erg flu.rpoEe hre I'r^o.rc, *he *lt;ehr-rrs
lm-x I
L"s hs(.)
we wtll use hg ( rn- x)
,L m a:tts
os it sl"u,{6 {he furucfrcrr {o ?n-
t rn- xt Jire rndtr'va*fcn l.,ere ,i the dsrivah.rne
Gt m = 2L is. hrett defined. . The doubte
h r(^- *l
derivattrae howeuer disesrdtngstrS
is discornttrulous at r,,,s t g and
x.
m a>qs =
nob f = frL ,Eo r,rrc ctWl
,ueed to
* orrg alao rt -+f4 +
5, LeL e(x,nru) .

=hg(x-m)
Here are ir,^"texfed trr
^E
t/)e know (m-x) 7- elerivahues wrt vy1,

z ; lvn-x1+ 6 Ao tnre See hS(rn_x)

l'r.s (*-)r) 3S a Junc*toru "d
m'
L litls uara be a tf{ttY. corn{urw
E(r,n-xt -*) ; lm-xl> 6 as'tradrtomaLy x L*r^u'
vo.idable t ,.t-{h,^'s tA
rn-)t ; rrn-xl4 t j*t notnhon '
f, hgtm- x)
dm
E rn-:c > $

a
d m- x (-6

g' 1 I ryr--:rl4 t 1 'l-ln . ,-rt cowrpaaltT be

h6(m-r1 r^sri{t-en a s
drnz o lm ->cl > $ J

Ni+l^" tf"is dra.-.$rnaattoyr {t\R, O,YP, -]o co.Lc^.late-

W
f trnf I = Et dmz\ t' p(,n)l J
@ m- rv*
K (rt*) !- s tm)
= Var I-L drn ] @ m--Yf
 T7.Lb

T(m) = E t
t'
dnt
hs (m-x )
]=EIrtt--xrns)l
we know E ( r(a)) = t(a) So
Ttrn) = lP(lm-x|4s) : whiah caru he. co\cuta\zd. as f, ru
JJ*, t)
ylr+ E

Jtrm)= Jft,.rc{x
rn- [
m ^nS*
4tne frd$ is, symrnetri")o
( \ * haw,
m-$
= * J firtro- : . rut",*-*, dx
nn+ 6
T(rrt)
o a" 'o
> l((rrt) m+E
j (r-m)
K(m) e- 4x
- \/orc [ #ttm-x) ] 0
m+E
\tv/le mbev m
+hah e dx dx
m-X ; lm-x la L J
0
-d hlr,n-x; =[ t ; x (rn-[ = em L-e-x1m+t
dvn L* -6 o
; x)nt+6 (dI ,;'m"-s;
=- em
uets cqU Y - d \tg1nn-x)
drn L -e-6
valL(y) = E[y"l -Elyla So T(m*1= 1-t6
W* colculate eu,ch L s+e oy srup
m-6
Pot
t @
Elvl = j 6 ftx:dr +
rrc+
(m -cc) f(1)dx
- J 6 ft*l a*
-@
m-6 rn+ 6
Srl ftrrl

14 * vvl
>x
^4r' \ m+6
m+t m+t
m+t
Jtr-xlf
rn-
t)t) = rn j ft*:drc J
zt*t") rt^ttr__3_,FT.{
$
rn- t ,n- t L{"J

=e
rrt

ft-ii'"J jT d'*'H. -i'i' r

%y *''l riSouvoug t-ft "y
lt*s in Malhe*n+i,P
0
 LZ" L4

e [v"] rn 5 rn- m+ t
(fi''?
@
e- (rn-x)
-(rm >t) dx + -
( x-m) e dx + (cc-m) I
z- -(:(-m)
e [v'1 = E, e dr *
L J ) .2_
-b 2- rn-E nL 2, rn+5
@ @
obtaira"ed
o @
By \€? rery rig oryous C,olc-uJatteryr
irn' UtaitheMaHcl 1nre hcrc
ry I^?''.t ti ^3 e,o,drr ex?r€si,isvt
s
=t & s"e-

1- ! (z+ tt + t") e E

t
t - a1, (z-+(6 + t")d

4.,
L, !"eI
2-€
t( 5
-1-s) t< 0v* )
E LYtl = C *@ *@n@ Q.

We gee {he f #f K(nru+) {-t( ,r# 1 )haL.islhe asyntptc,ttc

$ncttorr-
Y&luo,vtce * il
{-{rt,,*l K(,rf I {-'(*n) 2. € ttdt-1-.s)
(r - e-t)'
4n, li [< e '.
f,rrrlicna
^"alzs
Asymptotic variance
2.0

1.5

1.0

0.5

delta
-15 -10 -5 5 10 15
 72_.L5

Novo noYz {nuk h}e set ouL +o rrtinirnise

r,r.)l =u[x- - ll
E [e ( X,
oyt {p es*tmqbe the rninirniser "l Z I Xi-rnl
h)e kyrernr ti.lot tre. rninimiser roould be enrpiricol rrnedton.
We couldrrat ta[t< aboub ih osgrnphoH.c vq]o'qnce becogse tarc cotrldvr't
Catctrtabe 1lne deylvalt\rea . So nre used hS ( x - rn) tn ploce (x, m)
{ p
F"t *he e,s*irTrqtov & -lhe emp,*"icat because
(x, .^o $ovttrte
fi) has changed f"""r, \ lx-rvrl +o hs(xlnn)
^lonq
P
bc e ca[L
YI
A
m6 s hs(*-x)
2YqW t w Lt
1 t=1
ryx

8nd hle se e *hat as S -> o *re J"r.c*ior.r fs

go rnre hone
t
Jn lmt m)
(d)
n+@ it ) Z(t-6e6-a L)
(t- e-t).
thte
know 6 r-s define Such *hat 8>o (ie
pos,"rfi\rre)

tt 6=O t^)eho\,€to $ nd liwr & (r- 5e6-dt)

(As o76 5+o
&r-) 1+ e-2E _Z6t
ttm 2 (e'6 - teE - et)
u*y*
S+o Ly ezE
ezE + t -z,et
Vue kroro by Tlytor Exponsiorr et= I+t+f+tt+
1l {'t 5L
So the cxpression hecomeg

a ti* ['*
S-)o L
# * 1S' +
EL a
863
ts! - (s+ t) f n* I +t*{
rt rr 3L
+---.-.
L
1+ z6 n tf,* !S. . +1-z [,* 6 + 6' + 6.3
u *to 3t r! 2,t 31.
t
 t2,26
St rtther stt^apfify
f(r-i,-
'"(*; t-il.
2 li- 5 ( a-rJ +
6-ro wr) h).
2-
+ s"(t,-h) s'(L3!- L\*
6 (
lto t) +
\ z\)

Ivt" {st +er(m "{ hf & \a alo- is zero

N e car c^an czl, 6)- and le,f (zsl o|+xnar Ao{o zeyo a^ 6-+o
s2 (S +gr ( I I -i)+ --
t ti* 8+o t t) RI o( I

*"lL
\21
e- \
AL] + a\ I
g -L
b t3J ?rl\
z-r-
o
( ,")
2*I
{
I
[{, t )

proceadtr,tq lel vnz 'evise l'ohn+ rue dfd' ar h}e

"fl{ace
-bf,o,^, ba{ove {orruarcl
ev1, uwexpecLel' plrotaten'," ndr^)
-) yrle had {tar, Loplace ( m , f ) qrd we r^.: onted f. J.^d t{ uue
"lts+rtbukon
+ 4Vte vPyocess boiled dor^.rn {p rrr-irumisinq E tXt - m \ urrt m-
+ 1n1e corr,,t dD *ho*, os hre c^r,6t +"1..e +rrt a*lro+.r€s 04 abso\n{e va\ues
? Werrrg nob , i* \^)e sgs {he. grob\ern %y ane Wera:por"wto{ M-esttma}ion . nnz
frabtem is eluivoLevtt
to $^dr*t {lne vrtinirntzer ,} e * 2, g(xt',rn)
rohercg(xf'vrn)=lxr-wu\oi'rrnrea\'rneac\qktaowlllal4lnisisachienned
\
b1 ',ft ; ev^fiicaL rnz'diovt
movls orn, lo itg a sywrg*ottc vatto'rr,e ' tie co^t Jirud'
l:Lt'aV
-> NexL rale
aE f(*d ) ornct K (,^*) {r-votve otoubte derivahr'er' * \ m- x 1

4r.vg,er Huber( Logs : lnJe soid wG $^""1 r+t6 {.* h5trn-x) -= g (m,x)
d"v,tz wilL tate-r put S= o ara,d thsn A ,lartt\ be trngivtcat rnedia.ra, OA

\.rs(ra^-x) willba lvn-Xl &r 6=o

+ We caLcuta6d asy*PtDttc variara'ce 1* anry delka(E ) . lr.r,ts on\ vne Pobl*
l\vgv{s Loss ts NDT EQuAL fo Irn-xl -for $= o .*'s O
=
 L2-, 27

Remember
(rn- x)' lm-x\S g
,
hs ( no- x)
\m-x\ ) 5

rnrherl 6 + O

he(m-X) : o. ,(
( \/n - Xl - z)\ =o
-0 m

0
d o \^re &re g+ac.k,., Bub not re at\Y
'(
^+ We wonter\to minirntge L U,5[m-xi)
i i ht(m-xi)
+ .sv ony 6 > o hle ccrvl s qS the minimtser h)il\ a\sovryrinr"rtue
i5
1e vr,r-tyrivnizing rwUrutrvttztnt

ti httm-xl) is el"iv"tentto t hs tnru-xi)

5
+ 1+., rn tir,tivltise.r otr\1.t is A6 as hle ca\cu\ated arr,d so i* L
the rninirvtizer + e hg("n-xi)
6
-> 4";" rr.ol seem *rtv[al up urtttt noh) put norr the
rf ny,roal c hrro+>tvtz,
1
{t*,'%u' fus
(nr- x) e.loes vwV
%u
b lv,t-xl ^L 5 +o . BL*
( lrn-xl - )
g
hg (m- xt g.oes ie t,iwt, Irn- x\
7 &
8+o
E

+ hewce we Law ,ht'r^tt1 ,% dt = g h4Prrr caL 1Io

whLe at E=o
rl
\^t€ knotnc {te asymgtottc vav[avr,ce lLs L
44d

.m A,tt +
YVL

+ B* lln*'s vne*hod
4 vayv1 t vtae I ts
1r**
. Lzl{ Sza r.rfti
 L?^"28

LeK g?,Q. r,u0'r,at l,r,oppene vtLwn b ,-+ a

l,us(n -X) ll"n (v^- x)' lm-Xl L 6
8+a
@6+a) 2,

-
{\€ i*b tx- m\'
^e 2,

&fl^ rs e
is the $ot exornple ie r,..:hen ?(r, ,fr) = (x- ,n)'
-t4ne hA+ = ftx] :l6,?avL
eslinnatton ll K sannple qv€r
-'lk."

e. hle know -\hot

rn:i {hevt Qrw
1
asrqnyb\ic vaeL
YYL ".

[trn {(l- Sgt-et)

t"
6+o (1 - ts)z
=2
o,Lth ougll, uv\€an is woV MLE Awe sample X6
A, 7
r,rreatT,

{""as +v asyu,tflo*tc vouiowce ,* *r,rr" . Bu} nye eau wrl,te- C LT

usrn q rne thods o f lt - e s*, rvr ati o-n
1.r

Jru (n Etxl) vv
(d)
1a
,il(o , L)

4]J [r'*"t is . }4ox tit< Eg*irvrqtov chooses -lhe unbaised.

no(e l'uere
e'h,na-ator wt+h *Le teost asgmgtottc vo.vianc€, ' l'*
rnecliavt hoc,t

asy rrtpfoftc Vart ora'ce aE I \"Jhi le *h"e, vf&oll -had vard anes 2
,
8, MLE Chose wwdlan-.