Modern Probability and Statistical Inference
Illustrated with R
Preface
Modern Probability and Statistical Inference has been developed to support the two-semester mathematical statistics course sequence for statistics students at Carnegie Mellon University (numbered 36-235 and 36-236). The primary difference between this textbook and others that are commonly assigned in mathematical statistics classes is that this one utilizes a spiral-learning framework (see, e.g., Bruner 1976), which simply means that concepts that might be covered just once elsewhere (such as point estimation) are repeatedly covered here. My primary motivation in adopting a spiral-learning framework is the observation that many students fixate on equations and coding and pay less attention to statistical concepts, with details that are seen once often being quickly forgotten. But in addition to helping enhance conceptual retention, a spiral-learning format can also help students build up knowledge of particular concepts bit-by-bit as opposed to all at once, leading to reduced cognitive overload.
In this book, I have radically revised the order (and way) in which topics are covered, relative to what one might see in a traditional mathematical statistics textbook. Such a textbook might have one chapter that illustrates the properties of discrete probability distributions and discusses the specifics of the binomial, geometric, and Poisson distributions, and then a subsequent chapter that illustrates the properties of continuous distributions such as, e.g., the normal and gamma distributions, with neither chapter actually demonstrating how these distributions are applied to carry out statistical inference. Here, I break the major distributions out into their own chapters (Chapters 1-5), and within each I revisit fundamental probability and inference concepts: probability mass and density functions, cumulative distribution functions, statistics, sampling distibutions, point estimation, interval estimation, and hypothesis testing, etc. As I move from chapter to chapter, I cover inferential concepts at greater depth. A concrete example of this is that when I discuss point estimation in Chapter 1, I introduce the concepts of bias, variance, and using the likelihood function to define estimators; then, when I return to point estimation in Chapter 2, I review these concepts (and derive the MLEs for normal distribution parameters), then add details about the concepts of consistency, Fisher information, the Cramer-Rao lower bound, and the asymptotic distribution of maximum likelihood estimates. In Chapter 3, I add in sufficient statistics and likelihood factorization, along with the minimum variance unbiased estimator. Etc.
Another important difference between this textbook and older, more established mathematical statistics textbooks is that I utilize R for coding visualizations, analyses, and simulations. It is expected that this will help students understand concepts more readily; it also allows me to broaden the “problem space” beyond typically used, analytically tractable distributions. (But I note that the newest generation of textbooks often employ enhanced computation…so it is really the spiral structure that makes this textbook fundamentally different.)
Mathematical Prerequisites
I expect students to be familiar with pre-calculus concepts (primarily exponential functions and logarithms, along with the use of the summation and product symbols) and with differential and integral calculus. In addition, students are expected to have basic familiarity with double integration and with matrices and matrix multiplication prior to engaging with the content on multivariate distributions, although experience has shown that instructors can cover these concepts themselves relatively quickly and as needed.
Adapting the Book Material
Below I suggest ways in which instructors can re-order the material of this book to fit with other course formats.
A Traditional One-Semester Probability Course
- Sections 1.1 - 1.9 (introductory material through cumulative distribution functions), with the
R-based sections 1.8 and 1.11 being optional - Sections 2.1 - 2.2 (the normal distribution), with selected results as appropriate in sections 2.3 - 2.10
- Section 2.17 (the exponential family)
- Sections 3.1 - 3.2 (the binomial and related distributions)
- Sections 4.1 - 4.2 (the Poisson and related distributions)
- Section 5.1 (the uniform and Pareto distributions)
- Chapter 6 (multivariate distributions)
A Traditional One-Semester Statistical Inference Course
- Sections 1.12 - 1.13 (statistics and the likelihood function)
- Selected results as appropriate in sections 2.3 - 2.10, plus sections 3.4 and 4.4 (derivation of sampling distributions, and sampling distributions related to the normal, binomial, and Poisson distributions)
- Section 3.5 (order statistics)
- Sections 1.14, 2.11, 3.6, and 4.5 (point estimation)
- Sections 1.15 - 1.16, 2.12, 3.7, and 4.6 (confidence interval estimation)
- Sections 1.17, 2.13, 3.8 - 3.9, and 4.7 (hypothesis testing)
- Sections 2.15 - 2.16, 3.10 - 3.11,and 4.8 (regression)
- Optional: sections 5.2 - 5.5 (inference with domain-specifying parameters)
A One-Semester Probability and Statistical Inference Course (for Data Scientists or Engineers)
- Chapter 1
- Sections 2.1 - 2.2 (the normal distribution), with formulae drawn as appropriate in sections 2.5 - 2.10
- Sections 3.1 - 3.2 (the binomial and related distributions)
- Sections 4.1 - 4.2 (the Poisson and related distributions)
- Optional: section 5.1 (the uniform and Pareto distributions) and chapter 6 (multivariate distributions)
In addition, one may wish to address numerical methods such as, e.g., root-finding for confidence interval estimation, optimization for maximum-likelihood estimation, simulations and the bootstrap, etc.
Acknowledgements
I would like to thank Eileen Xiao, who converted the original base-R graphics in the book to ggplot()-based graphics during the summer of 2023.
I would also like to thank Rebecca Nugent, without whose support the new mathematical statistics course sequence (and, by extension, this textbook) would never have been been developed, as well as my family (Sarah, Hugh, and Susannah), whose support helped see me through to the finish line.