Advanced Quantitative Techniques in the Social Sciences VOLUMES IN THE SERIES [HIERARCHICAL LINEAR MODELS: Applications and Dsta Analysis, Methods 2nd Eton Stephen W,Raudenbush and Antony 5, Bek “MULTIVARIATE ANALVSIS OF CATEGORICAL DATA: Theory Job Bran de Geer “MULTIVARIATE ANALYSIS OF CATEGORICAL DATA: Applications John van de Geer STATISTICAL MODELS FOR ORDINAL VARIABLES ifort C. Clg and Edvard, Shitadeh FACET THEORY Form and Content Ingwer Borg and Saul Sie [ATINT CLASS AND DISCRETE LATENT TRAIT MODELS, Similar nd Differences Ten Heaen [REGRESSION MODELS FOR CATEGORICAL AND LIMITED DEPENDENT VARIABLES 1.Scot Lang LOG-LINEAR MODELS FOR EVENT HISTORIES Jeroen K.Vermunt MULTIVARIATE TAXOMETRIC PROCEDURES, Distinguishing ‘Types From Contnas Niels G. Wale aad Poul E, Mech ‘STRUCTURAL EQUATION MODELING: Foundations and Extensions, 2nd Eton David Keplan ‘PROPENSITY SCORE ANALYSIS: Statistical Methods and Applications SSenjang Goo ad Mark W. rae PROPENSITY SCORE ANALYSIS Statistical Methods and Applications SHENYANG GUO University of Nort Carolina at Chapel Hit MARK W. FRASER University of North Carolina at Chapel Hill ‘Advanced Quantitative Techniques ‘nthe Social Sciences Series wv) @SAGECoprigh ©2010 by SAGE Pabtons Ins A rights rere No prof hack may be rpodicel ola a ty frm, cy ary men, dtr lading photoapring crn by Sy bron toe and ter ym, th rn ig fo he erinfrmation 6 SAQE Pulses, ne SAGE Puen nia Prt i 255 Teer Rod Bult eben Coopertne ‘housed Oa, Corin 9150 India foe ‘Ea orde@sngepohcom Matha Road New ei 11004 In SAGE Peon Le SAGE Pblion Ais Pie Pe, ta "Ole nd BBPekn See 40201 5 cy Rand fea Sq Lendon ECIY 1S? ‘Sogrpore | ned Kingdom Pedi he Used uss Ameri, inary of Congres Coin Pubeion Da ,shemang Pope anti mac spe Semag i Mat W Pema quanta cigs othe elms 1) Inca bitigepil fence sad ine ISaN oe -ta.s0-6(eeh). 1, Social incest aad 2 Ana of rae Fae, Mate, 146: I. Tie naaacorrs2010 siass—aaa aeoso1ans ‘This ookls pte ae see paper @ wun w9e76s4a7 ‘tions Bos Vie Right ‘Aoweite ior Sets Coney “iri Astouee Laren ta Pru Ets Kee Wey Capp Er QUADS Pees) ed pee (ant igi ak Prete: Seat One Idee ater Fpachoe ner Deer Gena Voge Mareig Manage: Steams AS1BI99 Contents Lise of ables it of Figures Acknowledgments 1. Trodaction 111 Observational Sadie: 12 History and Development 13 Randomized Bxperiments 712.1 Faker’ Rando Esperiment 1132 Tper of Randomized Experiments and Stati Tes 13 Crue of Socal Experimentation 1.4 Why and When ropensty Score Anais Is Needed 2 Counterfctual Framework and Assumptions 21 Casa tral Validity and Thess 22 Coumericuals and the Neyman-Robin Countracual Feamewore 23 The poral Team Asgnment Assmption 24 The Sable Uni exten: Vue Assumption 25 Methods to Estimate Teament ees 251 The our Models 1252 Other loncing Methods 26 The Undatying Logi of Sasa Inference 27 Types of Teste Ect 28 Hedkma’s Beonometric Model of Causal 29 Concusone 2 Conventional Methods for Dat Blanlag 5.1 Why Is Dat Bolaclg Necesry A Heuristic Example 532 Thee Methods of Daa Balancing "321 The Ordinary Least Squares Regression a sess usaeeasa 322 Matching 32.3 Smafaon 38 Desig ofthe Data Sialaton 3.4 Ress ofthe Dat Simulation 3. implications of the Data Simulation 3. Key sues Regarding the Application of OLS Regeeion 137 Condsions 4. Sample Selection and Related Modes “61 The Sample Selection Model, 44.11 Trncaton, Cenorng, and Incidental Tneation 444.2 Why Ie inporont to Made! Sample Slcton? 413 Momons of Ideally Tunoted Bivariate ‘Neral Ditton 414 The Hock Model ands To StpEsiator 442 Tretment Bet Mode 423 Intumentl Vaal atimator 414 Oneview ofthe Stata Programs nd Main Festus of tere 45 bamples 451 Applaton of th Testa Bfoct Motel ‘0 Analysis of Orr Data 1452 Bolton of Tetent Bf From a Progr Wiha Group Randoms Deg 453 Ruming the Teme Efc Model Afr Maile Inputs of Missing Date |46 Concasons 4. Propensity Score Matching and Related Models ‘31 Overview 52 The Poblen of Dimensional and the Proper of Propensity Scores, 5.3 Bsinating Propensity Scores 3.31 Binary Lage Regresion ‘5.1.2 Sri Spay a Cart Model ‘Pricing Propensity Scans 543 Hiren ad Imbonst Mato for Spciing ‘Preicon elying om Predetermined Cit Vlues 53.4 Generlied Booted Moding SaNahing ‘S41 Grey Matching 5:42 Oprinal Matching 1543 Fine Balone ‘5 Posimatching Anais ‘551 Malariate Anal After Gedy Matching ‘5.5.2 Stratification After Greedy Moaching Sea cevesag ease 100 1065 us 4 ws Ds 2 135 2s 1 Mo 6 Me ive Ww 153 ir) 154 ie 5.53 Computing Ides of Covariates 554 Outcome hls igh Hodes imo lg ‘Rank Te or Optinal Matching 4555 Ravin Ament Bolo Semple Crate by Opin! ae Mating Ss Rasim Adusnet Uig Hae ahaa ‘Alig Rok Sees fer Optna Making 56 Popes Sore Weg 5:7 Modaing Does of Tament 5.8 Overview of the Sits ad Programs 59 Boamples 21 Greet Maching a Susu Aa of ser Retr 592 Optinal aching 59. Poul Marching Anas ig he Hoe Lehane ark Tat 5.944 PoP hing ays ang Reresion of Dice ares 495 Propo Sore Weighing 56 Maden Des of Tamed 59,7 Campari of Mads and Conon ofthe ‘Sy of fe gt of over ot Chl fate Achevoent 58:8 Capuron of Random and State's Bont Alito 5.10 Conlsons 6, Matching Estimators ‘51 Overview 152 Methods of Marching Estimators 1621 Simple Matching Estimator (622 Bias Cored Matching Exinator (£23 Variance BstimatorAsoming Homescadasticiy (624 Variance xtimator lowing for Heerscdatty £25 Lange Sample Popris and Cavection 63 Oveviw a the Sats Progam mumatch 4 Bamples| (61 Matching With Bia Cored and Rabat Veron Eximatars (642 Bf Suet Analy With Machin Eins 155 Coneusone 17, ropensty Score Anayss With Nonparametric Regression 721 Oneview 7 Metis of Propensity Sore Ansys With Nonparametric Regressions met Ldshine te 197 19 10 161 18 19 m5 19 95 195, 7 19 a2 26 2s a 9 230 26 ery 2s 5NN 17.22 Review fhe Basic Concepts of La ins ‘i a a 12 pt a ie Sept Pepi el ee a ac x0 730nmtorae Si Pn mlz bop 2 ie fi foe a List of Tables *hadpned Rn Be ange mn mm ‘acontow a 1, Sain Bint Sey A zs otter Bas won a 11 satand RP yA Meas o 8.11 Sources of Selection Bins 26 ‘Table 1 Bhawan Rrneedes by 13 orc en as = feat Anal ap 15 Coun Seno a 2 ‘in Een (7) w aD) * trast reo Ba ze 22 none eons eae ° '8.2.A Monte Carlo Study Comparing Corrective Models 85 ‘Tible22 Benn {Sieg es Gi Say = ries Compton of ray Ros fore Sting Tooele ete Say 3 Gaus The Deere ® 2 ne Be nee ‘nr Say Ani = Tile32 Aci Do ory Rae . “83.1 The Basic Idea cd nee Eencking Gres 8.3.2 Illustration of the Wilcxom’s Signed-Rank Test ‘Table 33 Adjusted Mortality Rates Using the Age Standardization ‘eso debacle 9 ‘ia hemr Boone Come ‘ona afte en pon ere = Seton i Db a Remind i ae iy abso tfondftattpam 2S 34D Dacon nd Eine’ hy . 85.2 Sensitivity Analysis for the Study Using Pair Matching 317 ‘Three Methods: Scenario 1 ss conte 3 ‘ieAS Da Peron nd ited by 9. Concluding Remarks 2 ‘Three Methods: Scenario 2 a ‘941 Common Pitfalls in Observational Studies: ‘Table 36 Data Description and Estimated Efects by Achat tc sa Preece x 2a opty pins Wh Pope ‘Sere Aprons ¥ 26 ‘able? Dat Descipton nd Hamad Etsy 9.2.1 Criticism of Propensity Score Methods a7 ‘Tiree Methods: Scenario 4 6 92.2 Crscom of Sensvny Analy (T) 38 “TableS8 Date Description and Estimated Ect by 9.2.3 Growp Randomized Trials 38 ‘Three Methods: Scenario 5 a ace ane ay % peer eae eee ete os TWhe<2 eof Sn eae Ou Sino mao es 16 store = 5 Sah nr 2sPROPENSITY SCORE AALS Tables bead bless ‘bleu ‘bear “Tble “ble 52 ‘bess The ‘ible ss bles ‘bles Toes Tibies ‘Tbles10 ‘blest ies12 Exhibit Stata reateg Output: Syatax to Check Saved Stati Sample Description for the Study Brausting he Impacts of Caregivers Rept of Substance Abuse Services on ‘Cad Devedopenentl Well-Being Diferece in Pelagia Outcomes Before aad After Adjustment of Sample Selection "esmate Teatment fet Model of Fith Grade’ {Change on ICST Socal Competence Scores on (CCC Proscia Behavior Sere Exhibit of Combined Analysis of Trestment Ect Model Based on Malple imputed Data Fes Estimating Overall Heatment Bet After Strtieston ahi of Stata pomateh2 Syntax and Outpat Running Greedy Matching nnd Mahalanobis Mets Dance Exhibit of Stata Bow Stax and Output Running Propensity Score Model Using GAM Ebi of R Syntax and Outpt Running Logie -egresion and Full Matching Sample Description and Lopate Regen Models reiting Propensity Sores (Example 53.1), Description of Matching Schemes and Resale Sie (amples) Results of Sensvty Analyses (Example 3.1) ‘Stats of Cid’ Use of AFDC by Status of Caregivers Use ‘of AEDC in Childhood (Example 5.9.2) Sample Description and Result of Repression Anais (amp 592) Reals of Optima Matching (Example 5.8.2) \Covariat Imbalance Before and After Matching by Matching Scheme (Example 59.2) ted AvergeTeatment Eton lever Word entiation Soren 1997 Win Hodges ehnann Aligned Rank Te (Mtg Schine Full Matching) (ampie533) ey us ms 156 in 189 181 wt 198 ‘ible S13, ‘bests ‘bless ‘ble s16 ‘hlest7 ‘blest ‘Tble 519 ‘bie 520 ‘Dest ‘Tablet ‘Teble 2 “Ribie 63 sober a erasing Dilfrence Score of Letter Word Idetifation ‘om Dilferense Sere of Coverite Aer Pe Matching (ample 59.4 CovarateImbalance After Propensity Score Weighing (ampie5.25) Regression Anais of Letter Weed entiation Seren 997 With Propensity Score Weighing (Gxampe 3.95), isuibuion of Dose Categorie (Example 598) Mulinomial Logit Mode Predicting Generalized Propensity (sample 59.5) Regresion Anas of the Inet of Dosage of (Chi AEDC Use onthe Leter Word detication ‘Score in 1997 With and Without Propensity Sze ‘Adjustment (sample 596) Comparisoa of Findings Across Mol Estimating the Impact of Poverty on Children’s Academie Aebierement (Example 9.) Comparison of Covaiate nblance Before and ‘After Matching Bete Rang and Sa’ boos (Pxample 538) Rogrsing Diffrence Snr of Outome (es Change of Aedemie Competence in Third Grade) ‘oa Difleence Scores of Covariates After Pair Matching (Comparison of Rvs Between Rand-ghm and Stas boost (sample 538) An Eumple of Simple Matching Wits One Observed Corn for Seen Observations An Example of Simple Matching With Tree Observed (Covariates for Seven Obseratons With Minimum Distance Determined by Vector Norm Using the ives, of Sample Variance Marie ‘An Frampe of Spl Matching With Tee Observed Covariates for Seven Obsereatons With Minimum Distance Detrined by Vectoe Norn Using the mers of Sample Vrianee-Covariance Matrix 9 200 201 om 208 ae 29 a‘il PROPENSITY SCORE ANALYSIS bees ‘bless ‘bless ‘ble 7 ‘bless ‘able72 ‘wher “Tables. ‘ble s2 ‘Tbleas ‘hlens Exhibit of Stata mmmateh ynax and Output Running Bia Corrected Matching Estinstors With Robust Standard Errore BI ‘mated Teatment Hts (Ef of Chil’ Use of AFDC) ‘on Pssage Comprehension Standard Score in 1997 Using Bias-Corected Matching With Robust Variance Estimators (eample 6a.) as timated Tetment ets Measured ab Change Score inthe Fourth and Fit Grades by Thee Estonstore (Gxamples2) 2 Sample Sie and Dissieton of Exporue Tine Progam Invention ("Dosage’) by Grade Example 642) mt czy Subset Anais Using Matching Bstinators Estimated Average Testment Bets forthe Tested (SATD) by Dose (Example 4.2) a “iit of Stata psmateh and be Syn and (Ouipur Running Matching With Nonperamezic Regresion 264 timated Average Teatment Bets fr the Tested CBCL Change Diferencsis-DierencesExtimation ‘by Loca Linear Repeson (Esample 7-12) 270 Estimated Tentment fect forthe eat (Chis Use ‘oF AFDC) on Pasage Comprehension Standard Score in 1997: Capri the Propesty Score Aalys With ‘Nonparametric Regression With Bia Cocesed Matching nd Robust Varianee xtimator (Example 7.42) m ey Asumpons end Ect by Comrestion Model 286, Ress of Monte Carlo Study Comparing Modes 256 Resa of Monte Calo Study Comparing Models Not Contig for Z Unde Stig as ‘aarp of Sensitivity Analysis: Blood Lead Leela of Children Whe Pret Ae Exposed to Lead at Their Pcs of Work Versus Children Whove Fuente Art ‘Unexpsed to Lea at Tht Paes of Werk 300 ‘bleas ‘Tbleas “bles ‘Tales ‘bless ‘ble 810 ‘bie 81 bies 2 ‘bteas hibit of Sep Take the Absolute Vale of Differences, ‘Sort the Data ia an Arcendng Order ofthe Absolute Dilferences, nd Create That Ranks the Asalute ‘Value of ifences and Adj for Tes sit of Se 2: Calculate the Wilco Signed-Rank Sate for the Diferene inthe Outcome Varabe Between Tata and Conta! Groupe ahi of Step 3: Caeuate Statistics Necessary for Obtaining the One Side Sgnicance Level for ‘he Standardined Deviate When T= hibit of Step 4 Caeulate Needed Statistics for ‘Obtaining the One Sided Skgnfcance Lees or the Stndardned Deviate i the Lower end ‘Upper Bound of pVae) When T= 2 Exhibit of tp Caleulae Needs Stasis for Cbsaining the One Sted Sgniicence Le or the Stundardiaed Deviates (ithe Lower and ‘Upper Bounds ofp Vue) When T= 25, Remake of the Snsity Analysis for ‘loo Lead Levels of Chen Range of ‘SigniteanoeLavels forthe Signed-Rank Statistic Exhibit of Stata rounds Syntax and Ourput (Grample 85.) Rass ofthe Sensitivity als forthe Study sf Chile's eter Word detiation Sore: Range of Sgifiance Levels forthe Siged-Rank Static Brample 8.5.2) east to den Bis in Four Observational Std, eet Tes a a 30 a 16 a 39List of Figures gure 2 Figure 3.1 gure 3.2 Figure 3.3 gure 34 Figure 35 Figure gure. Figure 32 Figure 53 Figures Figure 7.1 igre72 igue73 Pore Conemporancous Corelation Seatrpot of Duta Under Sceauio | Sésteplot of Dit Under Senario 2 Scateplot of Duta Under Scenario 3 Scterplot of Data Under Seno 4 Scteplot of Data Under Scoaaio $ Desiion Te for Eaton of Social Experiments Gear Proedre for Propensity Sore Matching, Ilsratin of Common Support Rion Using Hypothetical Data (43) = lg ~ 40) Survivor Functions: Percentage Remaining No Rereport (Gample5.81) Distribution of Estimated Propensity Scores (Bsample 592) Comparison of Estimated Propensity Scores Generated typ Rand.ghm and Those Generated by Stats boat (Gampies.9a) stration ofthe Nee for a Better Curve Smoothing Using Nonparametie Regesion The Tsk: Determining the Vale fra Foal Pint x, ‘ih Within the Spun Can Be Determine by she Ticube Kernel Eanction Py * » Ms 196 a a2s_ PROPENSITY SCORE ANALYSIS igre 7.4 Figure 78 Figure 76 igure 81 ‘The Value at the Foes Point sy ea Weighted Mens ‘The Nonparametric Regrsion Line Connects 1190 average Values “The Local Average Now I Predict by & Regrestion Line Instead ofa Line Parallel othe Axis Desig of the Monte Cal Stay “wo Setng of Section Bat a7 28 20 Acknowledgments Wistsesipr et hd in rei bok Hs ‘ethan i original dewoper often methods for sean sea ining ames Heck, Pl Rte, Dod Ra, Alber ‘adi Guo Unteny Heiko Ichimura, and Pera To, whose {ootrutons in developing the fur steal thd fr obser ‘Bima his bok pole Tie tank ihr Bat the ctcinchi of the bok, his aay ‘nao es and hela ring ole procs Whe epring tis bok we ied inate comments, gestions and et help or Thal Foeabeum, Bon Haren, Guido Inn, et Tod fob Fx Micha Foxe and wo snonois evens Sheayang Go thks iste Wa. Mason woe inortve aad igre rom in aii realy ape Ns Cartan choo quae meodlgy orale ea “te thakour ferme stone eo i Shea tect ser equ ede Vk Kaito Putin or tila preparing the book, "We thank many of our colette Unversity Nr Carin at ‘Chapt Hl sin the Bld of wil wok reach, Dean Jack Rian ‘shmof pometng ec igo wich was how by Ms upporto is oer moted to wit isbok Rr arth engaged i egal “oruon of design he oc and contd ele gens rs upto we sed secrets ely Barto dst plans heed mut In sod work reach Dine Wyant provided exelent ‘Storal ap rhe ene Boo Aan lls ped wih programa ia Ror ‘Soe lacng exp, ng Sok Le ped mange he SID and CDS {ist wer ployed sve hating examples Care Pes Dis Sd fn Ui heed hte earch fr computing procedures cae” valle aie fl, We tank Cyt Cte mig for stan: wih ‘pring non lec Prt of the Bancl apport wax provided by the John An Disoguhed Poesonhip hd Mac Poesil PROPENSITY SCORE ANALYSIS Finally, we thank our famies for their support, understanding, ad patience Specialy Shenyang Goo thanks his wife Sbenyan Li and chien an and Honse; and Mark Fraser thanks is wife Nary Fraser and chien ‘Ales end Katy Tis bok is diated fo ou all Introduction Peeps rin van dn tte ‘methods that bas proven us for evaluating treatment eles when ‘wing nonexpeimental or ebserrtonal data. Specially, propensity score stall oflers an epposch fo progam evaluation when randomized clinica Ul ar inal or une of when rereachers eed oases estent| ‘fice from sey dt, census daa, administrative dt, or other types of| Gate “colletod throug the observation of jes a hey operat in normal practie withost ay isterveatons implemented by randomized assignment Fules" (Rubin 1997, p. 757 ‘As such this book focuses on four cosy related but technically distinct model for estimating treatment effete (1) Heckmats sample slcion ‘adel (Heckman, 1976, 1978, 1979) and its revise version (Maal, 1989); (2) propensity score matching (Rosenbaum & Rubin, 1983) and seated ‘ods (3) matching estes (Abate bens, 202 206) and) propensity ‘oral wth Poop reson (Hedana, nurs, & Tod, 1997, 198). “hhough statisticians and econometrians hate not reached consensus ‘on the scope and content of propensiy score anal, the statincal models SGoctibed i ths book sare several silar characterises: ach hav the ‘bjectne of seing testment fects and contol for cova, each represen sat-oftheet analy in program enlurtion, and ech ean be “enployed to overcne various Kinds of challenges encountered in research. Although th randoized conti il is dese tobe the ol standard in esearch design, tue experimental digas ae not aways posible, practic, for cven desirable a human socal sclenee tesearch. Given tat sca src evech onnues to rly envily on quas-eximental resech design, ‘everches hee incesingly sought malbds of moved program evasion ‘Ove the past 390 yea methods of program evauation have undergone a signifiant change a tescorchers have Tcognzed the need to develop more2 PROPENSITY SCORE ANATYSIS sflcentepprouces for asessng treatment ft fom studies based on bteritional data and for evalations based onus expimestl designs ‘his rowing interest in sedking consent and ecient xian of progam cffectivenes ed to a surge in work focused an etimating average festment ffs under various sels of sumptions. Statticine (eq, Rosenbaum & Rubin, 193) and economeviiane (eg, Hes, 1978, 1979) have made sultant eontbatons by developing aad fining new approahcy for the esimation of causal efecs from observational data cote thee spproches are know as propensity sore ana Econometriians have integrated propensy score model ito ote ‘sonometric model (fe, instrumental vel, contol faction, dtrence initzens estimators to perform es expensive and let intr nonepet- ‘mental evshations of sci educational and eat progeas. Frchermores ‘ecencicsm and seormulatons ofthe classical experientl approach a _conometies ymbolzean important shit in eration methods. The signif cane ofthis movement was evidenced by the ston offs Heckman 5 ‘one ofthe 2000 Nobel Prize award winnes inthe eld of economics The Priv recngnize his developmen of theory ted mets for at analyisin elie samples, Repeeseing the intr in—and indeed perce uly ofthe new ‘methods th propensity score approach has been employ in vaiy of plines apd. profesions such ab educaon (Morgen, 2001), epidemiology (Normand eta, 201), medicine (eg, sl tl, 201; Gur, Thomas, Witanabe Blackstone & Laer, 201), puyhology (Jones D'Agostino, Gondal, {&Heket, 208, scl wor (arth, Gran, Gy Gree 2087; Barth Les Wild, Gu, 206; Gu, Bart & Gibbons, 2 Weiner, Barth 8 Ooo 2018), an sology (Smith, 1957). In scl were sad, conomsts and ‘ter used ropes sare methods in eatons ofthe National ob Ting, Patnehip Ac progam (Heckman, hint, & Tod 1987), the Nan Supported Work Demonstion (LaLonde, 885) andthe National Eaton of Weaareto-Work Strategies Stl (chalopuls, lor, & il, 2004. In dsrbing these new methods the preparaon and writing thisbook es guided by to primary objectives. The ft objecting was to ntodce ‘eadesto the orgns main etre, and debates entering onthe fu dels ‘of propensity sare gals We hope tis noduction wl he aecompsh ‘ou second objective of minting new ides concept and approche tat 4ocial behavioral rescrchrs ca apply to thee own fs to sive proses ‘hey might encounter in hei eeach efforts. In aon this book bas two ‘overarching oss Our primar goal oak the past thre decades of then ‘eal and technological advances in analtic methods acess and eae ‘na es tec and more pect fashion. The second gal is to promote Fr Inoduction 3 dscusions among socih haves retarchersrepeding the challenges ‘Rees and best metodo eimating cal et sng nonexperiesl smetbods. "The sim of hie chapter isto proride an overview ofthe propensity score approach. Scion 1. proseatdefintion of aberration stud. Scion 1.2 ferews the history and development of the methods. Section 13 Is an ‘review of the randomized experimental approach, whichis the gold Sondard developed by satin, and the model that sould serve as a foundation forthe nonexpersmentl approach, Seton 1. offers examples ‘dwn fom iteratare beyond the sof econometis and satis. These ‘eimples are intended o help readers determine the suatons in whi the propensity snee aproach may be appropriate. Section 1.5 reviews the ‘Computing sofvate packages that are crealyevanbe for propensity core fnulyiand the main fens ofthe pacage used inthe models presented throughout this Book Section 1 oie the organization ofthe book. 1.1 Observational Studies “The static methods wedincse maybe geez catered as methods for ‘dheratonal sue. According to Cochran (1863), an obseratonl sud i “Sn erpiil investigation whowe objective i to eit esl relationships (ies cause and ef), when is infasble to use contol experimentation no asignpatipents st random to ifsest procedures. In the general Iterature rested to progeam evavaton (Le, non- sais orentedIieratre), searchers se the term qua-eserimental ‘ote feguetl than obveretinal tus, withthe term defined ab exper- ‘mental studies that compre groups, Dut which lack the crt ment of| ‘Fandom signet. Indeed, quasiexperiments can be wed interchangeably ‘with observational tie as desertbed in the following quote frm Shit, (Cook, and Camptell (2002: ‘Qus-eeriment sare wil ter experiential purpose—to {ex dong cue! Iypotboce about anipubleeuinas well 3b try soca etch th regu! resect gon ae [peta mewn 0 spp» cuatertctal infeece oat what wuld “\ Five happened int svece of teen. Bu, by dein, gus) ‘Species lnk venom augnment sigan to condton by ‘low of election byw unt cone tevimet fer hems tres fest ny weer bates ng, Cheap pce or others decide which persons shoul et which wate fp 1-10)4 PROPENSITY SCORE AKALISS ‘often steppin eto on ent ice ham hen novi rs ope cn peta Stern sy hm po pal caning ae ee tn rn Smt od yor pret eee sh er pte br es Sond ban ep da om tne tinobrtin seeps ba Bancoecn ee ‘ices cv By non ee wero dash nr gm senegal prima anny ney, muss nae eee og 9), 1.2 History and Development ‘he tam popes eS speed 983 are by Rosh sad Babin wh dourbed he natn fa fas fom ose ata \ Heckman (975 1979) nok on ny endogenous rrr wing sivenousequion odsing sewed it mae hae of etnaog tmnt i wh angen mare: hw ema moe tors pnp ef ample econ Atongh Har werk on he danny eno vale bln empye feent emilee a the ne sprnch ve ena = Pre ants probity resving oe of two cnn Bek coal ap {2 te connate eaton of Henan and he mae tno esennumand fin) brads nen nunc on tec oe Sk although te tem prope ere ai coed by Roses snd in ised mot fee a er re feed tae lq eo cre rs sin sat aia The devopment of the Propensy score apron ge « sree tn ng tne se cn rls on auc eGion tng te ‘sical itn Ut primary rr on rndoniae apres (A inten abi 15 Hen 0, Theme a ae to Type Hari (14, 84), whe ring mth deapel ‘pmo ta ntanon ut taeda per er dtpendence song ostames dng bern Sng a cong on inp and pare ot sen es an sprouse ce (Tie ta of exiting mera stm ely dered and ed | br satis expr by scores teh fg \raroen det (ads, 18S hand 198 197). Hedena 97a Iesedatn 5 197) elo ose itor cee athe a piecing | (Zann aplaty moddig Be crf ceion oe frm of an | ‘Sihpno were As poly mentored, Hadar wok flowed tc omens td aed pb oxgh sacar eon) ode ntl qui din fom te conomets ado, be sail conn bed Yo oer 931 Ney (1929 ad Rabin {ore 17, Uae emenons al on srctar eaon ode he Sica edn stoma td 0 te od pene. The ‘mpl nna ron th yout oun non 8 Sendo i enfin Under ths ame he ce SGatoteamet on sape prpnt esy epost © weer) ae ‘pov cacy core opr spent xed ‘aera tin extn te otc amen nae opened ‘Eamon such ar oben tes witout nomion Tore ded dco ote vo din ered 4 specie he a oft er Siti ci 96S Se wch poss an teaing ga betwen tatcon a Soom Sgatant sole nthe Blinding, re ‘sm, oR and Roem —prted ins cin of peper ht wel ttrumeral aso ey cl ec parte? Beal eg testo (gt ey 1) 1.3 Randomized Experiments ‘Te sata emi of propa etn grounded ia ean of {erandoanedeaprimert Tet snr srg ptt in acion {fond baton in ebrond i o ti ey etes f Sh dom expenen sori Hobe 2025) hoy ef Chsantonal suas mut bw slave te ke of apdnizton it Cin huean cul den i themes fsb For ease ‘Sivas cng Rows prac ang da wih ‘Siler scents sppesh nde ae oft is tht wee {ciel prim er ned experienc When el rank and Hodge Lc eins Haves ges of cl ‘Sprint yeoman 6 Sanh, 1985) feqenly TS Socio of he conts de wich enon ina ay id te hing of ed bev sacar Tis spo Career pind nine eprint adomiaton tanh Clg thn ach of he plese in Sling stone(6 PROPENSITY SCORE ANANSI 1.3.1 FISHER'S RANDOMIZED EXPERIMENT The imeaton of the andi xin penal eed oS onl Pier, oe ofthe ore sia of he 20h cary Pars took, The Dep of pret ODI) noord eps ot ‘todoniaton, demoatalg than wh he owns cape eng Dish woman ings. Tseng ben ed ety ‘state te pve anda and hag of potas ang Gs ge acl & Delay 190 ety, 308, meat ‘ato, end sea a aon pre cos Ineabmied expres Tne 1981971) word he pelo s flows ‘lady ss ht by ting 4p of tx mae with ik he can di rin tte lo hee etn a ft de tthe ea Me ‘cone probe of dapingsnaperinen y meant wich ie Aerioecin eee (1) (Dig Hie ns he dominant practi a petition so [ son cores eonnding acts gt catenin etree ~ ec Thee, tat + pes tating aly se oe i Sictiminte tw made a ts ppt reac welcome tr tht coud ince hers sch ate temper he stag the teu bee spr an eamounta i cde ane {the myiadptel lence ht mag wr among ec of ‘tedinan experi As Me ad ey (90) pede ‘ell oleae pn ie Fiber cae ohne ‘yiepinent heal thee be sel“ be esa ake expt forthe independent arable bing spats Mae ree ti doe it gomnds Fr be ug at say tapes ade bath nthe expe and in aperineaton gel Seo et stg ht vn ere concenatete so cat lkene of ne ‘tay inpecpibi etic” on eros deen ef eta ‘wool it pric Be wo expense tan (0), Tasted of controling for every potential cosfunding fcr, Fer Proposed to contra for nothing nama. employ atnatod of randomization, ser (1995/1871) described his design as follows Os exerient consi ming it cp oe ari ome ay andor inthe hes ad preening tm othe et fr lpn se, cr he te abl san fh that i am ‘Ben hat he i eae oa gh ap ht he Sal be ‘stn a they sh be pret obama rr fn an der nt determined biz by human cies, bub he ta anion of he phys ppt ed nme of car a Tlie or more expel om pice oat of random ‘Sing sn poring 0 ge e al rls of wc an ‘Sidon Herta toda capi fo eof pone, ithe ent red) Imeducton 7 eo sng urbe ii culo note evel porta pois ead sng Perdis nh cal he i of ss nie (ike tht eate the presen of te ex cape tte ar Le = 8, Inch s wal of 8 cues cmp the sp). Sn, thes eet ‘Dignan proce a ape ue he ode of rein of ‘ir Ung venue's 22) notation sb random earl Zand sy ‘pei pronon ote cape to ttereareiation of Zor = ruin espn pret hte cp coats rcapth kde lid sr cos vith sed tw may re Pe Citioom, wae jt oe of many posite awignmest Ia ones nota, hepsi niger frm set OF fnlve @ Derniig heel under floes nf which Reverb ‘Dao isan importa foresee eg saa en ‘eacsomplae xing probity henry Tis pot wi be del in more Sal chee Thine anal ute whch the rea fhe tng tog cap off ae ger xc the ame or of Crane eat ret Le se etc ieee et ae ‘hp shoring te mile ft nd the ne our ops hr he en ‘ed tesy chun teams weld eer r= (101080) at he test csenl ans oder whee eter bd here Sy {lecsiminae the eK teow se mar et odamet ‘en y gening Tes nl pubs) under tng wuldbe SS ha ty dernintc a the ot ines cing aii Ciena ost he lipo a ren ipiae eed Balding on thes clots conte ih the ning tnd esc hn hie inplementad i andonae exe One port tee of tenomted expres tha In aden of inplesettn, the ence mast eae pts tome for ech dy is Fer {ios pose fretting a pons tomes een at sgn geen stone nt cy aan vn oe the tpprpriterss fy propendexperimenl din ays set ‘Bru all poe ral fhe epeiment and to hve dace without {hui hat ineptaton sil bpd apo xh oe then”) “he key of uch eaten toa the al ober of ees in the sof @ (eye ale of 1 nthe above empl we sen sae fThiry cml of este sgment 1000 ough macy othePROPENSITY SCOR ANALSSS| ‘weaiment assignments canbe ey gue out, chs aeratng caps often ith he mika at with he cups prepared by ang thee infusion St (Ge, 1101010), or presenting four cups with infusion added fst and then {ur cups with nik ade fit ie, DILL). Inti th counting rele, (ey permutations and combinations) infra at thatthe mabe of ft posible ways to preset the eight cup can be soed by finding out the ‘number of combiation of eight things aken four at ine oF Cy 38 nla 1)e—2) (n= rt) Dada AT ‘The sation 0 on problem ie ‘Therefore thre ar 70 posible way to presen the ts taster wih four cups wth mi added fist and for cps with tea added ist. We can beep ‘weg 1110000, 0701010, 0001111. uns we enka all 70 way. Hee, 170s the numberof tox elemets i te set of oral posits far ‘wcatment absigament ‘Toperform a aisles of Fs No ability her tured to the ask of looking into the posible outcomess Furthermore we defn the tastes toe bili to taste srimintly a ein all eg cups she Mette to match exactly what we presented tober, we cum then cleat the probity of hing the tre ootcome. The significance tet performed ere lnles ‘ejecting the null hypethes, andthe nal hypothe is exprsed as 0 bly” Fhe sed the logic of "fussing the outcome righ that ste taser ‘aso ability to discriminate Dut mikes her outcome correct by puesing, ‘Thus, what tthe probly of having the outcome that sential to te ‘ceatmentasignment 2? The outeome® soc hae ones of values om the 70posble eaten asigamens tht the ater old guess any outcome from 70 pose vices of 111000, 1010100, 0001111. Therefore he probably of gusing the right outome ie 7D = D124, which is very ow probabil Now, we an reject the nl brpothesi unde sal probably of ‘makinga Type error (ie, thee taser di have the aby. but we ero ‘ejected the“ ably” hypothesis) andthe chance indeed very low (O12, nother words, based on sts evidence (ie, an examination of all pone outcome), we con eet the "no abit” hyphal ‘signifcanc evel of 5, Ths, we may cde that under such a esgn, the taster may hee tue eating sity (p< 05) Inmeducton 9 Rosenbaum (2012) asd (2) to dente the ts static. nthe above est scenario, we requied perfect mteh—a tte of eight agreements been the teattet (Le he order of tes cups presented to the rentating pet) and the outcome (Le, the actual outcome denied bythe taste) Cherefore, the problem isto Bad out the probability (1,1) > 8). Tht robeblsy cn be mor formal exes in Rosenbaum notation a flows: Meo: ane n) x -orpob(ez8)28} = =0124. probe) 27) = However ifthe definition of resi eax wo allow for sic ect sgreoments rather than eight agreements (t,x caps inthe onder of ‘atcame match fo the onder of presentation), we cn lela te prob- {lity or signicance in texting the mall hypothesis of "ao aby” AS in ‘he eee compton thi calculation ivales te comparison of actual ‘ulcome vote testent ssgnment x, and she tea tasters outzome could be any ane of 70 possible outcomes. et us assume tat Ue aster ges het ‘outcome 25 x= (1111000), We now need to examin how many treatment Stsigaments (ie, number of 2) match ths outeme under the rezxed ‘etintion of tue ability” The amwer to this question is one perfect match (ies the match with cight agreements) pls 16 matches with six azeements (Rosenbaum, 20025, p30, for a ttl of 17 teatmentasigaments. To ‘hsteate, we peovide all 17 treatment segments hat atch to the tasers “outcome 1111000 ess mach 11110 nde lovin inc ogeemens 11100, 111000, 111916, 11010, 100, 1110, 141003, YoLL00e,soiere, 1o1o, 01100, 1111060, 11086, 11006, 00110006, where bod munis dite agreements? “hs the peobality of having sx exc agreement is 17770 Rosenbaum’ notation, he caution is [eeastzez 1) {22 ee peaioieele! «20 eob(t0.9)= 7) “Tay we define “tre abil” a corel deg sx out of ight ups of tte proabiy of having correct outcome increases to.24.The ‘ul pote caanot be rejected 305 level la other wards, under his ‘eaxeddefnion, we should be more conservative, of ought lo be more10 PROPENSITY SCORE ANALYSS ‘elactant, to dcare that he tea ater trae ability, With a sample of eight ‘ops in total and a relaxed definition of “sbi” the statin evidence i simply insufficient fr us to rj the mul hypothe and, therefore, he ‘experimental design ses signa in testing tae tating ait We ive described Fishers fanous example of rondo experiment in grat deta. Oar purpose of dong so ewofl. The fist i state Ue importance of understanding two procs in gneatng intervention data (1) the treatment asigment proces (Le, there ia random vrbleZ, and the total numberof pose ways Kis inevitably age) makes it poset koow in advance the probability of reeving tetment ine uniform ‘andomind experiment and 2) the proces of generating outcome dita (Le, ‘here is an oacome varie). This topic ie eit bth in Chapters 2 and 3 nthe dscusion ofthe so-called ignorble tetiment essgnment, ain {Chapter n the discussion of eleton bias and senstvty analy. The second purpose in providing a etd description of Fisher experiment was {ocallatnton to the covelement of rndomizl experiments According Rosebaum (20025), es, exeriners do ot regi, indo cat ronal eur tht fxperinenal unis be homognene, witout rarabley a Ut response... Second, experiments do ott indo ett, ‘enon rou ht expel anit sono sep fon Population of ols... Thi, ar ral nec ae te ec of etme 0 the ele nen experinent e ee ‘ha ersten be aoc ardor fo expres net ts ‘nay Ye bo etgeneos a Wh epones at sag Fom 2 apulaon, Four, probuiy enters te experinet oni vagy the Eins eigen often pce nly eens 1.3.2 TYPES OF RANDOMIZED EXPERIMENTS AND STATISTICAL TESTS Fibers framework laid the foundation for randomized experiments) design The method has Become gold standard in program evaluation and ‘continues to ean effective and obart mean for aseing eaten ets cal every fel of interest rors arcane, to compute cence to ma factoring t madicine and social wel, Purtheraoe, many sophisticated randomized designs have been developed to estimate virions kinds of treatment effets under various seting® of dats generation. For example, within the category of uniform randomized experiment ia ation to te ‘eatonal method of comply randomised experiment where station {absent (ie, $= 1 and S sands for numberof srt), escarches have toon 1 eveloed namin lock periment where wo or ore rata ate pemi- (Ble (en 522) and pried nndomizel experiments a which ny=2 (the ‘numberof study participants within stratum Ss aed a2), = 1 (the ‘nuber af paricponts receiving treatment within statu Sis eat 1, and ‘Seoul be reasonably large (Rosenbaum, 2002) "A moe important reson for toying randomised experiments i that satsial tet developed through randamied expeinents ay be performed ‘irtully without assumptions, whch isnot the case for nonrandomized “periments, The ca of ranomiton tex asreviewed and suramarzed by Rosenbaum (2002), indudes, 1, Tit fr Binary tn the Fs (19381971) cute the Mar Hao 1959) ss and he Maton) ot 2. Tt fran otcme ved conto sl amber of es ‘npreing mural cringe rere canary (eo ren ithe Met (163) xeon of thee ns 4. Tt for sgl tnt SL whe te oxtome sable may ae may met wean itera oavarile) th Wie (1988) ok 4 Tse foranommeyare thats and he nue fsa Sislange ompured wath plese Nth Hoe ad eon 92) eng ‘elon aie _As oppo ta drawing infos sing tert in randomized desis, caving inferences using thes ess in nonrandomized experiments eques ‘sumptions that are not atl innocoos” (Rosenbaum, 2002.27). 1.3.3 CRITIQUES OF SOCIAL EXPERIMENTATION Although the randomized experiment has proven useful In many pplication since Fathers seminal Wor, the past tue decades have wit ‘ese «chorus of challenge to the fundamental esumptions embedded in ‘he experimental approach. in parsclar eis have been quick to note the ‘complexities of plying randomiod alsin stadies conducted with humans rather than mechanial component ageclural fs, or cups of tea The ‘lemma presented in socal bebaviral stadis with human participants is ‘that ssigning pariipnt to acon condition means potently drying treatment or services to those participants in many settings, such denial of services would be unethical or Mga. Although the original rationale for ‘sings randomized experiment was the infeaslity of contoing covariates, ‘orevalston needs have retuned the plat where covariant contol rit ‘rials eg, matching) besos atractve. This particulary tue in socal ‘behavioral evalaons12 PROFENSITY SCORE ANALYSIS In a series of publications, Heckman and his colleagues (egy Heckman, 1978: Hedkman& Smith, 1985) dase the importance of ety moding | poses of sing tly parpans to tne condos by sng | tacts that intuencepartcpans’ decisions repsdig program patton. ‘eskman and his associates challenged thespian tat we can depend on -randomization to create groups in which the ested aad nonteted pate ants share the sume charset der te condition of nonteatmet. They ‘questioned the fundamental sumption embeded in the cls] exper ‘ment that ndomiation remover selon its. Heckman and Smith (1983 in partclar hed that socal behavioral vale ‘stons ned o explicitly adress four questions, none of which canbe han. ed sia by the sndomized experinnt (1) Whit are the effcs of tos ‘sich suis, averting, local shor marks, amy income, race and grader on program applction decisions 2) What are the eects of bute tate prfomancetndards Jc labormasket, and individ chet tieson administrative decisions wo ace splians and place them a peiic programs? (3) What are the effect offal bekground bile and hcl ‘kt condition oa decison to dropout o program and alenatvey on she ken f in gid compte por) Wha at theo cf ‘rarows atenative resent? 1.4 Why and When a Propensity Score Analysis Is Needed Deenag aaa incense in osetia sades oss without "andomaton' challenging. nd att Bs ote aay td ecnoetican to pre now antic eho Te bur tl (osha edicts in thon Gre fom ths wrk. Alou Be me Sb iron the spec mean epi or mode ain eos Se ig they et nga soni ee freronent flr wing nonrancomed ot - nde ose mltiensenal citer tw tone dimen ste resi Tove ae ya hen popes re bdr need recs foun ‘ocalbebvo dicplinsPropeniy cre amas i sulle te Stasis and ens inenc ral thes Mos of tse xa wile reed thouginr hsbc ‘ample I: Assing the Inpact of Catholic Vrs Pie Schoos on Learning ‘long-standing debate inedgeation is whether Cathie coos for private ‘schooksn general are mare eflecne than public choos in promoting ening Iooducion 13 ‘Obrint af selections are involved the formation of “eaten (Geretrance ino Cato shoo!) To name afew, sslcton is a process that those who choose wo sayin Catholc choos recie the ween ‘cholic ia proces that permits school to act only those students rho mee certain elements, prolly minimum academic sandards 10 {our into the eatent nancial sto prt that excludes rm he treatment those students whose falls cannot afford tion: and gopaphic ‘ston ins proces tht selects ut (Le excludes) students who Bre areas ‘ere no Catholic school exis, Uiately, the debate on Catholic shoo ‘Gates on the question of wheter difeense observed in outcome dat (Ges academic achievement or graduation rats) between Cathal and public ‘schools re triable othe intervention or atbutablet theft that he ‘thoi rchoole serve diferent population. In ther words f the dilferencs “reatibutable tthe erento, Hndigs set hat Cathal shoos ro ‘not Ising more este than publ choos whereas he direncss tne atsbatble othe population served by Catholic choos, Gndng would ‘how thi dens curently enrolled in Catholic schools would always ‘emonstrate beter academic outcomes regress of whee they attended private o public schools is infsible to condac randomized experiment to anor thee guetions ower, oburvaional data sacha the National aueatonal Longitudinal Survey (NELS) data are avaible to resechers interested inthis guest. Besse observational dita lick randomize’ assignment of pctiiants lato treatment conditions researchers mst eploy state procedures to ‘balance the daa before assessing tentent effets. Indeed, numerous pub led stages have wed the NELS data to addr the question of Catholic school efsivenens however, the findings have been coneaditory. For Instance, ang propensity sone matching and the NELS data Morgan (201) fours that the Cathoie school effet the strongest only among those Cathoi seo iden who according to thee oberved characteris, ae lest kel to aend Catholic achools. ower a stay tat wed the ne INELS data bot employed anew method that diel assed satya ‘Akon He, nd Taber (2005) found at attending a Caolc high School ‘sbtantally increased dent's proba of graduating fom high school sed more tentative atending calle ample 2 Assng the impact of Povey on Academie Aimer. Pear -ssarch ha sbown that exposure to poverty and participation is wellae p= ‘grams have song snpcts oa cid development. Ingenta growing uP Ia Poverty adversely fects a il’ fe prospects and the consequences become ‘ore severe with greater exposure to poverty (Duncan, Brooks-Gunn, Yeung ‘Smith, 1995; Foster & Parteners, 1998, 199; Sth & Yeung 1938). MostPROPENSITY SCORE ANARYSIS Prior inquires in his eld ve applied a amv ssass (eg, maple ‘egresion or ratesion-ype model) to samples of natonaly preentatve