Monographs on Statistics and Applied Probability 37 ‘Generalized Linear Models SECOND EDITION ‘ee -P. McCullagh and J.A. Nelder FRS eee A and HallGeneralized Linear Models SECOND EDITION P. MCCULLAGH Department of Statistics, University of Chicago and J.A. NELDER FRS Department of Mathematics, Imperial College of Science and Technology, London @ LONDON NEW YORK CHAPMAN AND HALLContents Preface to the first edition Preface 1 Introduction 11 12 1.3 1.4 1.5 Background 1.1.1 The problem of looking at data 1.1.2 Theory as pattern 1.1.3 Model fitting 1.1.4 What is a good model? The origins of generalized linear models 1.2.1 Terminology 1.2.2 Classical linear models 1.2.3 R.A. Fisher and the design of experiments 1.2.4 Dilution assay 1.2.5 Probit analysis 1.2.6 Logit models for proportions 1.2.7 Log-linear models for counts 1.2.8 Inverse polynomials 1.2.9 Survival data Scope of the rest of the book Bibliographic notes Further results and exercises 1 2 An outline of generalized linear models 2.1 Processes in model fitting 2.1.1 Model selection 2.1.2 Estimation 2.1.3 Prediction xvi xviii 21 21 21 23 25viii 2.2 2.3 2.4 2.5 2.6 2.7 CONTENTS The components of a generalized linear model 2.2.1 The generalization 2.2.2 Likelihood functions 2.2.3 Link functions 2.2.4 Sufficient statistics and canonical links Measuring the goodness of fit 2.3.1 The discrepancy of a fit 2.3.2. The analysis of deviance Residuals 2.4.1 Pearson residual 2.4.2 Anscombe residual 2.4.3 Deviance residual An algorithm for fitting generalized linear models 2.5.1 Justification of the fitting procedure Bibliographic notes Further results and exercises 2 3 Models for continuous data with constant variance 3.1 3.2 3.3 3.4 3.5 3.6 Introduction Error structure Systematic component (linear predictor) 3.3.1 Continuous covariates 3.3.2 Qualitative covariates 3.3.3 Dummy variates 3.3.4 Mixed terms Model formulae for linear predictors 3.4.1 Individual terms 3.4.2. The dot operator 3.4.3. The + operator 3.4.4 The crossing (+) and nesting (/) operators 3.4.5 Operators for the removal of terms 3.4.6 Exponential operator Aliasing 3.5.1 Intrinsic aliasing with factors 3.5.2 Aliasing in a two-way cross-classification 3.5.3 Extrinsic aliasing 3.5.4 Functional relations among covariates Estimation 3.6.1 The maximum-likelihood equations 3.6.2 Geometrical interpretation 26 27 28 30 32 33 33 35 37 37 38 39 40 41 43 44 48CONTENTS 3.7 3.8 3.9 3.6.3 Information 3.6.4 A model with two covariates 3.6.5 The information surface 3.6.6 Stability Tables as data 3.7.1 Empty cells 3.7.2 Fused cells Algorithms for least squares 3.8.1 Methods based on the information matrix 3.8.2 Direct decomposition methods 3.8.3 Extension to generalized linear models Selection of covariates 3.10 Bibliographic notes 3.11 Further results and exercises 3 4 Binary data 41 4.2 4.3 4.4 4.5 Introduction 4.1.1 Binary responses 4.1.2 Covariate classes 4.1.3 Contingency tables Binomial distribution 4.2.1 Genesis 4.2.2 Moments and cumulants 4.2.3 Normal limit 4.2.4 Poisson limit 4.2.5 Transformations Models for binary responses 4.3.1 Link functions 4.3.2 Parameter interpretation 4.3.3 Retrospective sampling Likelihood functions for binary data 4.4.1 Log likelihood for binomial data 4.4.2 Parameter estimation 4.4.3 Deviance function 4.4.4 Bias and precision of estimates 4.4.5 Sparseness 4.4.6 Extrapolation Over-dispersion 4.5.1 Genesis 4.5.2 Parameter estimation 98 98 98 100 101 101 102 103 105 105 107 107 110 111 114 114 115 118 119 120 122 124 124 12646 4.7 48 Example 4.6.1 Habitat preferences of lizards Bibliographic notes Further results and exercises 4 Models for polytomous data 5.1 5.2 5.3 5.4 5.5 5.6 5.7 5.8 Introduction Measurement scales 5.2.1 General points 5.2.2 Models for ordinal scales 5.2.3 Models for interval scales 5.2.4 Models for nominal scales 5.2.5 Nested or hierarchical response scales The multinomial distribution 5.3.1 Genesis 5.3.2. Moments and cumulants 5.3.3 Generalized inverse matrices 5.3.4 Quadratic forms 5.3.5 Marginal and conditional distributions Likelihood functions 5.4.1 Log likelihood for multinomial responses 5.4.2 Parameter estimation 5.4.3 Deviance function Over-dispersion Examples 5.6.1 A cheese-tasting experiment 5.6.2 Pneumoconiosis among coalminers Bibliographic notes Further results and exercises 5 Log-linear models 6.1 6.2 6.3 Introduction Likelihood functions 6.2.1 Poisson distribution 6.2.2 The Poisson log-likelihood function 6.2.3 Over-dispersion 6.2.4 Asymptotic theory Examples 6.3.1 A biological assay of tuberculins 6.3.2 A study of wave damage to cargo ships CONTENTS 128 128 135 135 149 149 150 150 151 155 159 160 164 164 165 168 169 170 171 171 172 174 174 175 175 178 182 184 193 193 194 194 197 198 200 200 200 204CONTENTS 6.4 6.5 6.6 6.7 6.8 Log-linear models and multinomial response models 6.4.1 Comparison of two or more Poisson means 6.4.2 Multinomial response models 6.4.3 Summary Multiple responses 6.5.1 Introduction 6.5.2 Independence and conditional independence 6.5.3 Canonical correlation models 6.5.4 Multivariate regression models 6.5.5 Multivariate model formulae 6.5.6 Log-linear regression models 6.5.7 Likelihood equations Example 6.6.1 Respiratory ailments of coalminers 6.6.2 Parameter interpretation Bibliographic notes Further results and exercises 6 7 Conditional likelihoods” 71 7.2 7.3 7A 7.5 76 1.7 Introduction Marginal and conditional likelihoods 7.2.1 Marginal likelihood 7.2.2 Conditional likelihood 7.2.3 Exponential-family models 7.2.4 Profile likelihood Hypergeometric distributions 7.3.1 Central hypergeometric distribution 7.3.2. Non-central hypergeometric distribution 7.3.3 Multivariate hypergeometric distribution 7.3.4 Multivariate non-central distribution Some applications involving binary data 7.4.1 Comparison of two binomial probabilities 7.4.2 Combination of information from 2x2 tables 7.4.3 Ille-et-Vilaine study of oesophageal cancer Some applications involving polytomous data 7.5.1 Matched pairs: nominal response 7.5.2 Ordinal responses 7.5.3 Example Bibliographic notes Further results and exercises 7 xi 209 209 211 213 214 214 215 217 219 222 223 225 229 229 233 235 236 245 245 246 246 248 252 254 255 255 257 260 261 262 262 265 267 270 270 273 276 277 279xii CONTENTS 8 Models with constant coefficient of variation 8.1 8.2 8.3 8.4 8.5 8.6 Introduction The gamma distribution Models with gamma-distributed observations 8.3.1 The variance function 8.3.2 The deviance 8.3.3 The canonical link 8.3.4 Multiplicative models: log link 8.3.5 Linear models: identity link 8.3.6 Estimation of the dispersion parameter Examples 8.4.1 Car insurance claims 8.4.2 Clotting times of blood 8.4.3 Modelling rainfall data using two generalized linear models 8.4.4 Developmental rate of Drosophila melanogaster Bibliographic notes Further results and exercises 8 9 Quasi-likelihood functions 9.1 9.2 9.3 9.4 9.5 9.6 9.7 9.8 Introduction Independent observations 9.2.1 Covariance functions 9.2.2 Construction of the quasi-likelihood function 9.2.3 Parameter estimation 9.2.4 Example: incidence of leaf-blotch on barley Dependent observations 9.3.1 Quasi-likelihood estimating equations 9.3.2 Quasi-likelihood function 9.3.3 Example: estimation of probabilities from marginal frequencies Optimal estimating functions 9.4.1 Introduction 9.4.2 Combination of estimating functions 9.4.3 Example: estimation for megalithic stone rings Optimality criteria Extended quasi-likelihood Bibliographic notes Further results and exercises 9 285 285 287 289 289 290 291 292 294 295 296 296 300 302 306 313 314 323 323 324 324 325 327 328 332 332 333 336 339 339 340 343 347 349 352 352CONTENTS 10 Joint modelling of mean and dispersion ll 12 10.1 10.2 10.3 10.4 10.5 10.6 10.7 10.8 10.9 Introduction Model specification Interaction between mean and dispersion effects Extended quasi-likelihood as a criterion Adjustments of the estimating equations 10.5.1 Adjustment for kurtosis 10.5.2 Adjustment for degrees of freedom 10.5.3 Summary of estimating equations for the dispersion model Joint optimum estimating equations Example: the production of leaf-springs for trucks Bibliographic notes Further results and exercises 10 Models with additional non-linear parameters 111 11.2 11.3 11.4 11.5 116 11.7 Introduction Parameters in the variance function Parameters in the link function 11.3.1 One link parameter 11.3.2 More than one link parameter 11.3.3 Transformation of data vs transformation of fitted values Non-linear parameters in the covariates Examples 11.5.1 The effects of fertilizers on coastal Bermuda grass 11.5.2 Assay of an insecticide with a synergist 11.5.3 Mixtures of drugs Bibliographic notes Further results and exercises 11 Model checking 12.1 12.2 12.3 12.4 12.5 Introduction Techniques in model checking Score tests for extra parameters Smoothing as an aid to informal checks The raw materials of model checking xiii 357 357 358 359 360 361 362 363 364 365 370 371 372 372 373 375 375 377 378 379 381 381 386 389 389 391 391 392 393 394 396