QUANTUM SERIES For B.Tech Students of Third Year ofAll Engineering Colleges Affiliated to Dr. A.P.J. Abdul Kalam ‘Technical University, Uttar Pradesh, Lucknow {formerly Uta Pradesh Technical University) Machine Learning Techniques By Kanika Dhama K Quantum ~Page—_ QUANTUM PAGE PVT. LTD. Ghaziabad @ New Delhi Scanned with CamScanner> ee Apram Sina puousieD BY! QU ntum Publications? Quant Quantum Page Pt-Ltd) ave gor, Site Indus Area, ‘anibabad, Ghaziabod-201 010, Phone 0120-4160479 napa poe: 12 a sitcom Webster wwweauantumpage cain Robins Nagar, Shahdara, Delhi-110092 (6 Au Rem Rese Feon mov Feral or roomie, Ty ay mars witout poisson, Separator nay fo ee (ocmton conte in is werk is derived from sources | eters bert veyeflerths bean made tersure tno, however either de publisher nor the authors anne he acuraey or cmpleteness of ny information | Pent een and nether Se pulsher nor the authors | Sate repo any eres, emisions, or damages rig tof eof is information. Machine Learning Techni 1 Bain = 202021 pas INTRODUCTION MTEL INTRODUCTION ing we efned earning peb Dugan» Len peekine seaing AP Cen ember evn Dee Base a ong and Data Scene Vs Machine Leming oer neengSeION E BAVESIANLEARNING | (2-11L¢02-281) aoe ona epreson ad Lait Regresion REGRESSION ST Tpyen tore, Concept aming, Bayes aaa ene fe dare jean te reben CR ea opORT VECTOR MACHINE: ltoaucton, EM are ect keel (Lines het polynomial Poe re ett ygepiane ~ (Decision <0 Peper Sa ns VAL UNIT, DECISION THEE LEARNING. @1Lw3270) aoe Rin LEARNING «Deion te fxg lori DECI Janda neers with dein es, EHOPY rsa nation ain TD Algor, bones aaa erring INSTANCE BASED LEARNING b- ee a Lamang Lvally Weg Repessin. Ri SSUES Niverte Chena amin Unig ARTIFICIAL NEURAL NETWORKS (itessty) 1a TCINL NEURAL NENWORKS - Pereptren's, Muiayer rien sce the De le lye retorts, Caan eee sesrmacion Algor, Generaration, Deetieed tomagy SOM Alpin nd vant DEEP LEA omc a Peas eomehanel ners Arsen ont pki ERMeL Comapestconann idan oer Fring SPST Case sad af CNW for eg on Dubetc Reap Shing asa speie, Seeering ete. UNITS : REINFORCEMENT LEARNING (1 L105-301) ENFORCEMENT LEARNING Induction to Reinforcement [ian tre sori ce {oom ode foe Reinfrtement = (Sastow Decision proces, Q {Smarty tt Lama Agen) Apso Fanta isang ccon Bop 0 Loran GENE ALCOMITINS Intoducion Components, CA cycle Moueis of Evolution and Lesening, Applications * ‘SHORT QUESTIONS ($Q-1Lt05Q-191) Scanned with CamScanner{ CONTENTS a 1h oa, Tearing. Toes of Lee mon Well Dein Leaisg mem Pratens, Designs 3 Eesrine stem Hltry of My Introdution Gritehice Learaiog Approaches (rial Neoral Network, Clasense, Renforcenent Tearing. Decisnn Tree Leaening, Boyce Network, Support Veetor Msthive, Geet Algorithm) rts Machine Learning ed Dota Sdeace Machine Learning oo ATL to 1-81, —~ H9L to Lee, | | Part 1 1-241 to 1-261, MILcstTsem.5) Introduction 2L(cstt sem) “Learning, Types of Learnind Fae] Detine the term earning. What are the components of 2 | Taser 1 Learning refers tothe change ‘brought by repeated experi ina subject's behaviour toagiven situation mots in that situation, provided that the ree chengesconnet be explained onthe basis of native response Tanlenios matriculation artemporary states ofthe subject. Tearing agent canbe thought of s containing a performance elem crarteckicewhat actions to take anda learning element that modifies {he performance element so that itmakes better decisions. ‘The design ofa learning element i ffeted by three major issues: ‘a. Components of the performance element 1b Feedback of components. ‘e Representation ofthe components, {important components of learning are = she | sti set ae] Pett 8 flood —— Hl rk =o = a Perfrmance | a — vELiL Gaaauemess ‘Aciuiston tne knead 1 One cimponent of learning is the acquisition of new knowledge. Scanned with CamScanner—————— sige JLICSIT Sem) set chine teeing ) zislasiie 124 me F Mhetincpor sel saeottn te Tr eth pattern cane seed tte ok US TD Incheon of), econ can euvaenty Bes Fe see Basen ering. Bxalnn dose mal “stem eineqalies ctassietion Fa. patepatoy>pte|apet0) a [ewer 1e|o,) ply) All? 2 paloy gl) al Ry: ploy |2)> fo, |) fhe regions RR covers allthe space, we have ae dee [ hoy ace = 1 07 on Tay ti WaeBIT | Derine Bayes classifier. Explain how classification is done by using Bayes classifier, (273) and (2.7.9, we get, Pr=Hay) { ploys)-plopled plaids (2 a ‘ iyo eror ie mizimizedifR, 1 Allayes classifier iea simple probabilistic classifier bared ot Bayes theorem (from Bayesian statistics) with strong (Naive) Independence assumptions Scanned with CamScannerMachine Leaning Technique 291(C8T-Semal 1icSATSem-5) Regreston fe Bayesian Learning 5 a 2% ANaive Bayes classifier asumen that the presen or bene ot cular feature efacaesisunteatedto the prose nce (a absence) vin etoclnss sy other feature PT ayestee asta &_Dependingon the presse natureotthe poly model, Naive Bay lasers canbe tained very ctcnty a aaupervised learning t Bev Inmany practical applications parameter eatmation fr Naive ‘models uses the method of maximum ikelioed: in olber words, og ‘an work withthe Nave Mayor mel without believing in Bayeaag| robbs ong ny Byeron method 5 Anadvantagef the Naive Mayes lassie that it requires a small mount of training data to etinate the parameters (means ang ‘arancerefthe vanables)neerar fr elation, | '& The perceptron bears a certain relationship to eassical patter efirKaown asthe Raye cea “a 7. When te environment ix Gaussian, the Baye clstifie reduces ta lear cassie, lathe Bape clasifer, or Bayes hypothesis testing procedure, we ‘Biinite th average ik. detted by R Fora twesiuns protien ‘preseaid by dases Cy an Cth average rks dafined ke oyAf nicrOpeouh, | Race bere the various terms are defied as folows: 2, Poor potty race Heth sand Pe eet Gat fect in nour els represented | shenclas Gc tranjmthije,2 mented sterce Pyse | ea] nero of ree ee {at the observation vector x i drawn from Conditional rab density faction the random vector X | Fe 29:10 depicts ablck gram reprcetation othe By representation ofthe Bayes cassie, | The important poiatsin thistok diagram ae twofold | Te dt ning i dvi Bae clase ote ithe computation! the lo eats & Thicomputaton it complete letely invariant othe valor signed to the ror prbsbitenand involved in thedechow making eee ese quniiesmert afc te vacst the rehee © Fromacomputatinal pointt view we nit mor i Fork with logarithm ofthe Ikelite ratio shee Wee likelihood ratio itself, menheed Tiktbood | ie) ites Comparatt =) hernia, assign ittedane’y w Aasgn ets clay Tama hapa toh cone ites’ meet a teed lt) st classifier using some example in detail ier Refer Q. 29, Page 2-1, Unit 2. Forexample 1 [Lat Dis atraining st of fstures an ber azsciated casas. Each Feature is represented by an mdimensional attribute rector Haley ty oot) depicting measurement madeon the featre rom Arattelates respectively Ay Aye dy ‘oppose tha there are eases Cy, Cys item feature X the ‘lassie wll predict that X belong tthe lass having the highest posterior probity, conditioned en X. Tht caner predicts that X Eoingsteclaes€sFand ony, mei tse) me ee sana Degas rete beat en ain 1 nt th RG NE ee face pet ete Sere te ese eet es mene iacdaat Cente Sgnttemteae 1 Tgp een Eocene ribates the computation X10) Scanned with CamScannerMachine Learning Techniquet EM L(C8TE-Hom-8) Th Tissue tt he or of arte are odio Pie bere the leat he tr. ows, pixie» [#10 =n) ig IC) Se Me pobiiapli16Ceny)arey extiata seein tie Wave’ th valet tote so mdgate tn chedted ether the strat eget oroninun ale st Forexmpl comput piX|C) we comer, A ta teeter then |) te number estar of [,6 SaDhninethe voles ford aed 16, the sume feature sl in 1 Ita ncotinoouvalued then otineoavaldatebte ie {ipa cmupete anes Goouan duration wih meen ‘ind standard vation ined sa hat p10) et Vi Thereise need to compute the mean wand the standard deviation 1 atthe value of atibute A for training ac of clase C, Thees ‘aloes aruned to estate 10). xample let X (5, Rs. 40,000) where A, and Ay are the strbuten age ndincome, spectively Let the das label ttebute ‘betuyecompute. vid Th sociated cls label or Xi 0, buy-computer = ye Lets appoe tat age has 0! ben discretized and therefore exists stacoollnsouralued etibute ‘x Suppose that from the rising et we find that customer in D who buyacompaterare36«12yeara fags Inether words for atroata ge and ths clase, we bave = Band o» 1 1 Inorder opr the lass ube o Xp XC) ps evaluated for each © gape To sites pric tha tn cat atl a is thle Ce POICIRC)> pOLIC) AG) tors jemjei, _Bhe predicted clans abel ie the clase, for which piX|C) IG) the fetta i it gesty Pen -rigeeer te sen a re Vr ‘probabilities ofthese objects be given as follow ier to lasly pene pon ond pape : U3 Pipenigreen)= U2 Plpupesigreen) = U8 iUuiued= U2 Plpenfblue)= U6 Pipeperbfe ‘eed =U8——-Plbewred) =i Plpaperpeat yer rule: Pipeenpeni = Pipencil green) Mareen) ‘Fipencid green) Pgreen+ Pipena Boe) ibe) + pens red) Pred) 1 . ah =o. Ped Lt) 038 060 Sata ey Plpenei blue) ban) ‘Pipes grees) Mareen) + pena Dan) Ptblue)+ Pencil re) Prod) Pibiue/penci = at = 242, 274 -0378 Pipe red) Prd) PreecrenclD = Ciena red) Pred) + Ppenc Bw) Piblue) + Pencil green) Pieces) ala 24M, ga “9.35 “048 Since, lgrew/penil has the highest vale therfore pei tlang to clas green. Pigreen/pen) = =-—ipen/ aren) Paren) Pipeal green) Preen) + Ppen/ Bie) Plblue + Ppen/ red Pred) Scanned with CamScanner‘Machine Learning Techniques LISL(CSAT Se apo/ Mua) ibobeal= aaa Ren) pew a) ‘Piblue) + Pipers red) Pred) 12 4. Wom codtpn) » ‘iredipes)= pipe green) Pigreen) + Pipe! Blue) ‘ible + Pipen! red) Pred) riod ia ‘Siace Pigrenfpes) has the highest value therfore, pen belongs elas green gape eta) Pree) ——_ ‘oad ren Pen Pape) Phun pape ed Pred) ‘Pilue« paper ed Pied) es ost” a2 =°7%° Pibue) + Ppapee re) Peed) riod oat gai“ ‘Since, iretpaer has the ight ale there =a. Expin Naive Bayes classifier paper belongs Regression & Bayesian Learaing Naive Bayes model is the mest common Bi Fnachine learning. ere, the cass variable Cathe root which 2 Anat variables aretheleven, nS? Bde andthe ‘The model is Naive because it assumes thatthe attri © Mes toaly independent fenhather ree ues ae 1 avesian network model wsed 09 08 on Proportion corret om est st ig ART ear tt 4 Assuming Boolean variables, the parameters are = AC = tr), 0, = AX, = tre |C mtr), 4g PAX, tre | C= False) 4 Naiveayes model canbe viewed a Bajetianoeborkawhicheach [has asthe cole peat and Chas no parent 4. ANsve Bayes model with gaussian PY) isequivalet to mistre tf goussan with agonal covariance matrices 1. Walemisaresofaanan reed for deny etintinin cots dine Naive Byes models usedin darts and nite doosin, & Naive Boyes models allow fr ver ficient ference o arial and ‘onda distributions 9. Naive Baya learaing ha ao diay wth aly dat asi can gre "ore appropriate prbabisti:preditions 3. Consider a two-class (Tasty or non-Tasty problem with the following training data, Use Naive Bayes classifier to classify the pattera "Cook » Asha, Health Status = Bad, Cuisine = Costinental”. Scanned with CamScanner1 (CSTSem Machine Larne Tebninet po tilke Rogreson & Bayesian Learning oak] Hearsiaton [Osten ae ia _| Yer a ni —| Soest | tsa Sia sot] le = inca oo 03x26 = Gort Yes -Terfore the proditin stat. a a_i E va ee io GSEAE] Poplain EM algorithm with eps Ne na = ed] Content St Gond__| Contneoal_[_¥ eevee | redan Yer he Becton Maxiniation EM) algyrithm san trative way to ta Good cman ikelood estimates fr mnt paraetrs when the rm ao {8 Mpomieeorhan ang datas aatome biden varie fee rom afr mig pit entire seweet oda 41 Thewe reve vals are then recursively ued to etinae better fist Eat acy tng upraveng pot until Oe velos petted. vr | Ne cw] na] | 4 Toeenarethetwo bse stops ofthe EM algrthm Ste fe te sa Estimation Step: : i Iii Yad random vlc orby Kean csterng «fe omnes | eeu ory heretical tering esl 4 “Then for thot given parameter value, etimate he val of thetatent variables, th Maximication Stop: Uplate thn valet the parameters by Tyand calaated using ML method 1 Inte themean pe the covariance matrix and ea] the mixingcoctientsm, ty random vlue, (or other vale) Yes Ne hes [ ne Coins the» value forall Be [saan — Jue 18 i Agnincatimate llth parameters wing the creat vases Coats n6 | 34 jn, Computelogikeiiod funtion se Puteome convergence een. the log-likelihood valve converges to some valve (or fall the parameters converge to some valves) then stop, ‘else return to Step 2 Scanned with CamScanner_— ani Leasing Tecan “FeeTE ] Desert the usage EM agri. leas Canetti : ate ved ofiltbe misng detain asa ea athe asso unsupervised learning fle. mating the parameters of Hi avantages and disadvantage, 1 Oe a Meantoued {3 Teanbe wed far te puror oft Maron Mode BOO. {ented dior the vas fae arable 1. Wisabwajs guaranteed that dip etn ty ran olen 1 ABagean trek ott wit qua 2 Metatopecitcaats af hate se dra Me mt te ser raph in with each pode obability information, a erewsconncts pairs esos. ther ised oles parentofy re ditional probability a i lity distribution the elect of parents onthe node, ea (and hence isa directed aeyeie Bach node Pana) ae jan Leoraing Regression & Bayes crption of the domalt, te desert ne yan network provides a compl ion of A Baventay in he fal st probaly dint Baer maton nen run matwerks provide a concise a7 Bagi ee reltonsiprin the oma networks fen exponential salt to represent conditions! than the full joint “rewant to determin tho possi gasping wet oY Seppe Tcrurene of erent 8008 soe eather has thre sate: Suny, Coy, Boe esther the grse: Wet or Dr. wet but iit sro pemilercanbe oof 1 rainy the grass ges wet ut ease nt grace et by posrng water fom apriake? Few pat the acs wes Tie cold contributed by one othe Sarre esi ening. Secondly, he pear ar ning the Bayes rule, we can deduce the most contributing factor cease eaten and Rainy. There are ca Spriakler yesian network possesses the following merits in uncertainty romledge representation: Bayesian network can conveniently handle incomplete data, ‘Bayesian network can learn the eatua relation of variables. In data {nals casual relation is help for fel knowledge understanding, it an alo easily lead to precise prediction even under much interference. “The combination of bayesian network and bayesian statisti ean take fall advantage of eld knowledge and information from data, ‘The combination of bayesian network and other models can effectively void over Sting prebem, Scanned with CamScanner— Machine Learaing Techniques WaERIT] Explain the role of prior probability probability in bayesian classificaio ae] Rote of pri probability i 1 Te pro robuilty is sed to compute the probity of ll tefore the oleton of new data vette It is used to capture our assumptions / domain knowled, ii ‘independent of the data. * =f 1s theunceeitioal probability thats assigned ore evidence is taken into account. ‘07 reer Roleot posterior probability: 1 Paster probability edo compute th probaly ofan event ap tales data Ieisuedo captre oth the asuptins/ domain knowledge a ty 2191 (Csny, Poste 2 a 2 patter in observed data 3, Itis the conditional probability that isassigned after therelevant evidea orbackground is taken into account {Qae238] Explain the method of handling approximate inference in Bayesian networks. sven] 1. Approximate inference methods can be sd when tte led ouacepabe competation timer i very lag densely conesed 2 Methods handing approximateineence i Simulation methods: This metho eh orto enerte oa peabiydtrlon nde finer when henambe cases is method express the inference tase 1 fs s numerical optimizstion problem and then find vpper and lower bounds ofthe probabiltovaf interest by sling naimpied ‘version of this optimization problem. Vector Machine Introduction Types of Sapport | supper (uinear Kernel Polynomial Kerel, and Gaussian Monel Linea Berne Poyronl Rend and Go wht ta ab ai wet tnt ner er main jee 1 Page 1a Ua Ea] winters of suport vectr machine? wel Pr eyes vr nahi along GVM : Linear SVM is used for linearly separable dts, which 1 near SV jataset can be classified into two classes by using single a eel say er tated es edad as Linear SVM cassie i Ne Liner SVM edo on ner ral extn ee cut eddy ang gt i rh ne Sere anon en ata ad csr aes ‘alledas Noa-linear SVM classifier. aH] What is polynomial kernel? Explain polynomial kernel siagone dimensional and two dimensionsl. Tamer , veer «epi! Kernels a Kernel faction sed wth Snr Vs ae Sia and other kernel model that represent the Scanned with CamScanner