An Introduction to Regression: First Steps in Statistical Modelling

This book is intended as a first course in linear regression, which can be delivered at undergraduate or postgraduate level. The intention is to give a rigorous and complete account of linear regression together with real life examples using open datasets. One aim of the book is to be as self contained as possible, so any necessary theory on probability, statistics, matrices and linear algebra is given along the way. Beginning with an analysis of single predictor models, the concept of an objective function is introduced; two ostensibly different objective functions for regression are introduced (least squares and maximum likelihood) and it is shown that they result in the same line of best fit. Multi-parameter models are the next topic and this allows for an in depth discussion on the theory of estimation for ordinary linear regression using matrix notation and concepts from linear algebra. The book then moves in more practical directions, firstly discussing how we transform datasets, as they land on our desk, into design matrices (a crucial concept with wide applicability beyond the scope of this book). The remaining technical content covers the theory and practice of model checking/diagnostics, how to formally weight observations and a short review of some extensions that would lead on to future courses in statistical modelling. Following the chapter on weighted regression, a series of four case studies are presented, these are 'live' analyses of real datasets; the aim of this substantial chapter is to give the reader some feel as to how an analysis of a dataset might proceed and the troubles that one might encounter along the way (together with some suggested solutions!)