Modern Probability and Statistical Inference
Illustrated with R
Preface
I have developed this textbook for a new mathematical statistics course sequence for statistics students at Carnegie Mellon University (numbered 36-235 and 36-236), first piloted in Fall 2022 and Spring 2023. The primary difference between it and others that are commonly used in math-stat classes is that I explicitly implement a spiral-learning framework, with concepts that are usually covered in relative isolation elsewhere being repeatedly covered here (such as, e.g., point estimation).
My use of a spiral-learning framework is motivated by the observation that many students fixate on mathematics and coding and pay less attention to statistical concepts, with the details of concepts that are seen once often quickly forgotten. The spiral approach is meant to result in enhanced conceptual retention.
To build the spiral structure, I have rather radically revised the order in which I cover topics, relative to the order one usually sees in a classic math-stat textbook. That textbook might have one chapter that illustrates the properties of discrete probability distributions using, e.g., the binomial, geometric, and Poisson distributions, then a second chapter that illustrates the properties of continuous distributions such as, e.g., the normal and gamma distributions, with neither chapter showing how these distributions are applied in statistical inference. Here, major distributions are broken out into their own chapters, and within each I revisit fundamental concepts: probability mass and density functions, cumulative distribution functions, statistics, sampling distibutions, point estimation, interval estimation, and hypothesis testing, etc. Then, as I move from chapter to chapter, I cover concepts at greater depth. As a concrete example: when discussing point estimation in Chapter 1, I introduce the concepts of bias, variance, and using the likelihood function to define estimators; when I return to point estimation in Chapter 2, I review these concepts (and derive the MLEs for normal distribution parameters), then add the concepts of consistency, Fisher information, the Cramer-Rao lower bound, and the asymptotic distribution of maximum likelihood estimates. Then in Chapter 3, I add in sufficient statistics and likelihood factorization, along with the minimum variance unbiased estimator. Etc.
Another important difference between this textbook and older, more established math-stat textbooks is that I utilize R for coding visualizations, analyses, and simulations. It is expected that this will help students understand concepts more readily; it also allows me to broaden the “problem space” beyond typically used, analytically tractable distributions. (But I note that the newest generation of textbooks often employ enhanced computation…so it is really the spiral structure that makes this textbook fundamentally different.)
Mathematical Prerequisites
We expect students to be familiar with both pre-calculus concepts (specifically exponential functions and logarithms) as well as differential and integral calculus. In addition, students are expected to have basic familiarity with double integration and with matrices and matrix multiplication prior to engaging with the content on multivariate distributions, although experience has shown that instructors can cover these concepts themselves relatively quickly and as needed, as they are not utilized at depth.
Adapting the Book Material
A Traditional One-Semester Probability Course
- Sections 1.1 - 1.9 (introductory material through cumulative distribution functions), with the
R-based sections 1.8 and 1.11 being optional - Sections 2.1 - 2.2 (the normal distribution), with selected results as appropriate in sections 2.3 - 2.10
- Section 2.17 (the exponential family)
- Sections 3.1 - 3.2 (the binomial and related distributions)
- Sections 4.1 - 4.2 (the Poisson and related distributions)
- Section 5.1 (the uniform and Pareto distributions)
- Chapter 6 (multivariate distributions)
A Traditional One-Semester Statistical Inference Course
- Sections 1.12 - 1.13 (statistics and the likelihood function)
- Selected results as appropriate in sections 2.3 - 2.10, plus sections 3.4 and 4.4 (derivation of sampling distributions, and sampling distributions related to the normal, binomial, and Poisson distributions)
- Section 3.5 (order statistics)
- Sections 1.14, 2.11, 3.6, and 4.5 (point estimation)
- Sections 1.15 - 1.16, 2.12, 3.7, and 4.6 (confidence interval estimation)
- Sections 1.17, 2.13, 3.8 - 3.9, and 4.7 (hypothesis testing)
- Sections 2.15 - 2.16, 3.10 - 3.11,and 4.8 (regression)
- Optional: sections 5.2 - 5.5 (inference with domain-specifying parameters)
A One-Semester Probability and Statistical Inference Course (for Data Scientists or Engineers)
- Chapter 1
- Sections 2.1 - 2.2 (the normal distribution), with formulae drawn as appropriate in sections 2.5 - 2.10
- Sections 3.1 - 3.2 (the binomial and related distributions)
- Sections 4.1 - 4.2 (the Poisson and related distributions)
- Optional: section 5.1 (the uniform and Pareto distributions) and chapter 6 (multivariate distributions)
In addition, one may wish to address numerical methods such as, e.g., root-finding for confidence interval estimation, optimization for maximum-likelihood estimation, simulations and the bootstrap, etc.
Acknowledgements
I would like to thank Eileen Xiao, who converted the original base-R graphics in the book to ggplot()-based graphics during the summer of 2023.
Peter Freeman
pfreeman@cmu.edu
August 2026