Chapter Exercises: Solutions

Chapter 3

Problem 1

The median of a \(\mathcal{N}(2,4)\) is \(\tilde{\mu} = 2\), thus \(P(X_i > 2) = 0.5\) by inspection. Now, let \(Y\) be the number of values \(> 2\). Then \[\begin{align*} P(Y=m) = \binom{n}{m} \left(\frac12\right)^m \left(\frac12\right)^{n-m} = \binom{n}{m} \left(\frac12\right)^n \,. \end{align*}\]

Problem 2

(a) We fix the number of successes to \(s=1\), with the random variable being the number of tickets we need to buy before buying the one that allows us to win the raffle. There are two correct answers here: \(F\) is sampled from the geometric distribution, with \(p=0.4\), or from the negative binomial distribution, with \(s=1\) and \(p=0.4\).

(b) We have that \(W = 2-F\), so \[\begin{align*} P(W > 0) = P(2-F > 0) &= P(F < 2) = p_X(0) + p_X(1) \\ &= (0.4)^1(1-0.4)^0 + (0.4)^1(1-0.4)^1 = 0.4 + 0.24 = 0.64 \,. \end{align*}\]

(c) Let \(X\) be the number of winning tickets. Then \[\begin{align*} P(X = 1 \vert X \geq 1) &= \frac{P(X = 1 \cap X \geq 1)}{P(X \geq 1)} = \frac{P(X=1)}{1-P(X=0)} \\ &= \frac{\binom{2}{1}(0.4)^1(0.6)^1}{1 - \binom{2}{0}(0.4)^0(0.6)^2} = \frac{2 \cdot 0.24}{1 - 0.36} = 0.48/0.64 = 0.75 \,. \end{align*}\]

Problem 3

The first step is to write down the probability mass function for the number of insured drivers, given that the number of insured drivers is odd: \[\begin{align*} P(X=1 \vert X=1 \cup X=3) &= \frac{p_X(1)}{p_X(1)+p_X(3)} = \frac{\binom{3}{1}(1/2)^1(1/2)^2}{\binom{3}{1}(1/2)^1(1/2)^2 + \binom{3}{3}(1/2)^3(1/2)^0} \\ &= \frac{3}{3 + 1} = \frac34 \\ P(X=3 \vert X=1 \cup X=3) &= 1 - P(X=1 \vert X=1 \cup X=3) = \frac14 \,. \end{align*}\] So the expected value is \[\begin{align*} E[X \vert X=1 \cup X=3] = \sum_{1,3} x p_X(x \vert x=1 \cup x=3) = 1 \cdot 3/4 + 3 \cdot 1/4 = 3/2 \,. \end{align*}\]

Problem 4

(a) We are conducting a negative binomial experiment with \(s = 2\). The random variable is the number of failures…here, \(X = 2\). So: \[\begin{align*} p_X(2) = \binom{2+2-1}{2} \left(\frac{1}{2}\right)^2 \left(\frac{1}{2}\right)^2 = \frac{3!}{1!2!} \frac{1}{16} = \frac{3}{16} \,. \end{align*}\]

(b) The sum of the data is negatively binomially distributed for \(s=4\) successes and probability of success \(p=1/2\). The overall number of failures here is \(X = 1\). So \[\begin{align*} p_X(1) = \binom{1+4-1}{1} \left(\frac{1}{2}\right)^4 \left(\frac{1}{2}\right)^1 = \frac{4!}{1!3!} \frac{1}{32} = \frac{4}{32} = \frac18 \,. \end{align*}\]

(c) This is a negative binomial experiment, so \(E[X] = s(1-p)/p\), which decreases as \(p\) increases. Referring to the confidence interval reference table, we see that \(q = 1-\alpha = 0.9\).

Problem 5

(a) We have that \(f_X(x) = 3x^2\) and thus that \(F_X(x) = x^3\). Plugging these into the formula for \(f_{(j)}(x)\) (along with \(j=1\)) yields \[\begin{align*} f_{(1)}(x) = \frac{4!}{3!0!}(1-x^3)^3 3 x^2 = 12(1-x^3)^3x^2 \,, \end{align*}\] for \(x \in [0,1]\).

(b) We have that \[\begin{align*} E[X_{(4)}] = \int_0^1 x 12x^{11} = \int_0^1 12x^{12} = \left.\frac{12}{13}x^{13}\right|_0^1 = \frac{12}{13} \,. \end{align*}\]

Problem 6

(a) The cdf is \(F_X(x) = \int_0^x y dy = \frac{x^2}{2}\). Thus \[\begin{align*} f_{(3)} = 3x\left[F_X(x)\right]^{2} = 3x\frac{x^4}{4} = \frac{3}{4} x^5 \end{align*}\] for \(x \in [0,\sqrt{2}]\).

(b) The variance is \(V[X_{(3)}] = E[X_{(3)}^2] - (E[X_{(3)}])^2\), where \[\begin{align*} E[X_{(3)}] &= \int_0^{\sqrt{2}} x \left(\frac{3}{4} x^5\right)dx =\frac{3}{4} \frac{x^7}{7}\bigg|_0^{\sqrt{2}} = \frac{3\cdot 2^3}{28} \sqrt{2} = \frac{6}{7}\sqrt{2} = 1.212 \,, \end{align*}\] and \[\begin{align*} E[X_{(3)}^2] &= \int_0^{\sqrt{2}} x^2 \left(\frac{3}{4} x^5\right)dx =\frac{3}{4} \frac{x^8}{8}\bigg|_0^{\sqrt{2}} = \frac{3\cdot2^4}{32} = \frac{3}{2} \,. \end{align*}\] Thus \[\begin{align*} V[X_{(3)}] = \frac{3}{2} - \frac{36\cdot 2}{49} = \frac{147 - 144}{98} = \frac{3}{98} = 0.031 \,. \end{align*}\]

Problem 7

We have that \(f_X(x) = 1\), \(F_X(x) = x\), and \(j = \frac{n+1}{2} = 2\), so \[\begin{align*} f_{(2)}(x) = \frac{3!}{1!1!}x^1(1-x)^1(1) = 6x(1-x) \end{align*}\] for \(x \in [0,1]\). Therefore \[\begin{align*} P\left(\frac{1}{3} \leq X_{(2)} \leq \frac{2}{3}\right) &= \int_{1/3}^{2/3} 6x(1-x) dx = 6\left[ \frac{x^2}{2}\bigg|_{1/3}^{2/3} - \frac{x^3}{3}\bigg|_{1/3}^{2/3}\right]\\ &= 6\left[ \frac{1}{2} \left( \frac{4}{9} - \frac{1}{9}\right) - \frac{1}{3} \left( \frac{8}{27} - \frac{1}{27}\right) \right]\\ &= 6\left[ \frac{3}{18} - \frac{7}{81} \right] = 6\left[ \frac{27}{162} - \frac{14}{162} \right] = \frac{13}{27} = 0.481 \,. \end{align*}\]

Problem 8

We have that \(f_X(x) = e^{-x}\) for \(x \geq 0\) and thus that \(F_X(x) = 1-e^{-x}\) over the same domain. Thus \[\begin{align*} f_{(2)}(x) &= \frac{3!}{1!1!}(1 - e^{-x})^1\left[1 - (1- e^{-x}) \right]^1 e^{-x} = 6(1 - e^{-x})e^{-x}e^{-x} \\ &= 6(1 - e^{-x})e^{-2x} = 6e^{-2x} - 6e^{-3x} \,, \end{align*}\] and \[\begin{align*} E[X_{(2)}] &= \underbrace{\int_0^{\infty} 6xe^{-2x} dx}_{\text{by } y=2x, \, dy/2 = dx} - \underbrace{\int_0^{\infty} 6xe^{-3x} dx}_{\text{by } y=3x, \, dy/3 = dx} \\ &= \int_0^{\infty} \frac{3}{2}y e^{-y} dy - \int_0^{\infty} \frac{2}{3}y e^{-y} dy\\ &= \frac{3}{2} \Gamma(2) - \frac{2}{3}\Gamma(2) = \frac{3}{2} - \frac{2}{3} =\frac{5}{6} \,. \end{align*}\]

Problem 9

(a) A probability density function is the derivative of its associated cumulative distribution function, so \[\begin{align*} f_X(x) = \frac{d}{dx} x^3 = 3x^2 \,. \end{align*}\]

(b) The maximum order statistic has pdf \[\begin{align*} f_{(n)}(x) &= n f_X(x) [F_X(x)]^{n-1} \\ &= n (3x^2) [x^3]^{n-1} = 3n x^2 x^{3n-3} = 3n x^{3n-1} \,. \end{align*}\]

(c) We have that \[\begin{align*} F_{(n)}(x) = [F_X(x)]^n ~~ \Rightarrow ~~ F_{(n)}(x) = x^{3n} \,. \end{align*}\] We can also show this via integration: \[\begin{align*} F_{(n)}(x) = \int_0^x f_{(n)}(y) dy = \int_0^x 3n y^{3n-1} dy = \left. y^{3n}\right|_0^x = x^{3n} \,. \end{align*}\]

(d) The expected value is \[\begin{align*} E[X_{(n)}] = \int_0^1 x f_{(n)}(x) dx = \int_0^1 3n x^{3n} dx = \left. \frac{3n}{3n+1} x^{3n+1} \right|_0^1 = \frac{3n}{3n+1} \,. \end{align*}\]

Problem 10

(a) The cdf within the domain is \[\begin{align*} F_X(x) = \int_0^x \frac12 y dy = \left. \frac14 y^2 \right|_0^x = \frac{x^2}{4} \,. \end{align*}\]

(b) We plug \(n\), \(f_X(x)\), and \(F_X(x)\) into the order statistic pdf equation (where \(j = n = 2\)): \[\begin{align*} f_{(2)}(x) = 2 f_X(x) \left[ F_X(x) \right]^{2-1} \left[ 1 - F_X(x) \right]^{2-2} = 2 \left( \frac12 \right) x \left( \frac14 \right) x^2 = \frac14 x^3 \,. \end{align*}\]

(c) The expected value is \[\begin{align*} E[X_{(2)}] = \int_0^2 x f_{(2)}(x) dx = \int_0^2 \frac14 x^4 dx = \frac{1}{20} \left. x^5 \right|_0^2 = \frac{32}{20} = \frac85 = 1.6 \,. \end{align*}\]

(d) They cannot be independent: given the value of one, the other has to be either smaller (\(X_{(1)}\)) or larger (\(X_{(2)}\)).

Problem 11

Let \(X_1, \ldots, X_n\) denote the samples from the Bernoulli distribution. The log-likelihood is \[\begin{align*} \ell(X_1, \ldots, X_n | p) = \sum_{i = 1}^n [X_i \log(p) + (1 - X_i) \log (1-p)] \,. \end{align*}\] The first two derivatives are \[\begin{align*} \frac{d}{dp} \ell(X_1, \ldots, X_n | p) &= \sum_{i = 1}^n\bigg[ \frac{X_i}{p} - \frac{(1 - X_i)}{1-p}\bigg] \\ \frac{d^2}{dp^2} \ell(X_1,\ldots, X_n | p ) &= \sum_{i = 1}^n \bigg[-\frac{X_i}{p^2}- \frac{(1- X_i)}{(1-p)^2}\bigg] \,. \end{align*}\] The Fisher information is thus \[\begin{align*} I_n(p) = E\bigg[-\frac{d^2}{dp^2} \ell(X_1,\ldots, X_n | p )\bigg] &= \sum_{i=1}^n \bigg[\frac{E[X_i]}{p^2} + \frac{E[(1- X_i)]}{(1-p)^2}\bigg] \\ &= \sum_{i=1}^n \bigg[\frac{p}{p^2} + \frac{1-p}{(1-p)^2}\bigg] \\ &= \sum_{i=1}^n \bigg[\frac{1}{p} + \frac{1}{1-p}\bigg] \\ &= \sum_{i=1}^n \bigg[\frac{1-p}{p(1-p)} + \frac{p}{p(1-p)}\bigg] \\ &= \frac{n}{p(1-p)} \,, \end{align*}\] and the asymptotic distribution of the MLE is \(N(p,\frac{p(1-p)}{n})\).

Problem 12

(a) The log-likelihood and its derivative are \[\begin{align*} \ell(p \vert \mathbf{x}) &= \log (1-p) \sum_{i=1}^n (x_i - 1) + n \log p\\ \ell'(p \vert \mathbf{x}) &= -\frac{\sum_{i=1}^n x_i - n}{1 - p} + \frac{n}{p} \,. \end{align*}\] Setting the derivative to zero, we find that \[\begin{align*} \frac{\sum_{i=1}^n x_i - n}{1 - p} &= \frac{n}{p} \\ \Rightarrow ~~~ \left(\sum_{i=1}^n x_i - n\right) p &= n (1 - p) \\ \Rightarrow ~~~ p \,\sum_{i=1}^n x_i &= n \\ \Rightarrow ~~~ \hat{p} &= \frac{1}{\bar{X}} \,. \end{align*}\] Using the invariance property of the MLE, we find that \(\widehat{1/p}_{MLE} = \bar{X}\).

(b) The variance of this estimator is \[\begin{align*} V\left[\widehat{1/p}_{MLE}\right] = V \left[ \frac{\sum_{i=1}^n X_i}{n} \right] = \frac{V[X]}{n} = \frac{1 - p}{np^2} \,. \end{align*}\]

Problem 13

The likelihood for \(p\) is \[\begin{align*} \mathcal{L}(p \vert \mathbf{x}) = \prod_{i=1}^n p_X(x_i \vert p) = \prod_{i=1}^n -\frac{1}{\log(1-p)} \frac{p^{x_i}}{x_i} = \underbrace{- \prod_{i=1}^n \frac{1}{x_i}}_{h(\mathbf{x})} \times \underbrace{\frac{1}{[\log(1-p)]^n} p^{\sum_{i=1}^n x_i}}_{g(p,\mathbf{x})} \,. \end{align*}\]
Given the expression for \(g(\cdot)\), we can see that a sufficient statistic for \(p\) is \(Y = \sum_{i=1}^n X_i\).

Problem 14

(a) The likelihood is \[\begin{align*} \mathcal{L}(a,b \vert \mathbf{x}) &= \prod_{i=1}^n a b x_i^{a-1} (1-x_i^a)^{b-1}\\ &= a^n b^n \left(\prod_{i=1}^n x_i\right)^{a-1} \left(\prod_{i=1}^n (1-x_i^a)\right)^{b-1} \end{align*}\] At first glance, it seems that we can take \[\begin{align*} \mathbf{Y} = \left\{ \prod_{i=1}^n x_i, \prod_{i=1}^n (1-x_i^a) \right\} \end{align*}\] as the joint sufficient statistics. However, note that the parameter \(a\) occurs in the second statistic. Because this second statistic includes a parameter value, it cannot be a sufficient statistic…and thus we conclude that we cannot identify joint sufficient statistics for \(a\) and \(b\).

(b) With \(a=1\) the density function becomes \(f_X(x) = b \cdot (1-x)^{b-1}\), with resulting likelihood \[\begin{align*} \mathcal{L}(b \vert \mathbf{x}) &= \prod_{i=1}^n b \cdot (1-x_i)^{b-1}\\ &= b^n \left(\prod_{i=1}^n (1-x_i)\right)^{b-1}\\ &= h(\mathbf{x}) g(b,\mathbf{x}) \,. \end{align*}\] Hence, a sufficient statistic for \(b\) is \(Y = \prod_{i=1}^n (1-X_i)\).

Problem 15

(a) The likelihood is \[\begin{align*} \mathcal{L}(\beta \vert \mathbf{x}) = \prod_{i=1}^n f_X(x_i \vert \beta) &= \prod_{i=1}^n \frac{x_i}{\beta^2} \exp\left(-\frac{x}{\beta}\right) \\ &= \underbrace{\prod_{i=1}^n x_i}_{h(\mathbf{x})} \times \underbrace{\frac{1}{\beta^{2n}} \exp\left(-\frac{1}{\beta}\sum_{i=1}^n x_i\right)}_{g(\beta,\mathbf{x})} \,. \end{align*}\] We can examine \(g(\cdot)\) and immediately identify that a sufficient statistic for \(\beta\) is \(Y = \sum_{i=1}^n X_i\).

(b) We have that \[\begin{align*} E[Y] = E[\sum_{i=1}^n X_i] = \sum_{i=1}^n E[X_i] = \sum_{i=1}^n 2\beta = 2n\beta \,. \end{align*}\] Hence \[\begin{align*} E\left[\frac{Y}{2n}\right] = \beta \end{align*}\] and \(\hat{\beta}_{MVUE} = Y/2n = \bar{X}/2\).

(c) Utilizing the general rule from 235: \[\begin{align*} V[\hat{\beta}_{MVUE}] = V\left[\frac{\bar{X}}{2}\right] = \frac{V[\bar{X}]}{4} = \frac{V[X]}{4n} = \frac{2\beta^2}{4n} = \frac{\beta^2}{2n} \,. \end{align*}\]

(d) The first step is to write down the log-likelihood for one datum: \[\begin{align*} \ell(\beta \vert x) = \log f_X(x \vert \beta) = \log x - \frac{x}{\beta} - 2\log\beta \,. \end{align*}\] We take the first two derivatives: \[\begin{align*} \frac{d\ell}{d\beta} &= \frac{x}{\beta^2} - \frac{2}{\beta} \\ \frac{d^2\ell}{d\beta^2} &= -\frac{2x}{\beta^3} + \frac{2}{\beta^2} \,, \end{align*}\] and then compute the expected value: \[\begin{align*} I(\beta) = E\left[ \frac{2X}{\beta^3} - \frac{2}{\beta^2} \right] = \frac{2}{\beta^3}E[X] - \frac{2}{\beta^2} = \frac{2}{\beta^3}(2\beta) - \frac{2}{\beta^2} = \frac{2}{\beta^2} \,. \end{align*}\] Thus \(I_n(\beta) = (2n)/\beta^2\) and the CRLB is \(1/I_n(\beta) = \beta^2/(2n)\). The MVUE achieves the CRLB.

Problem 16

(a) We can factorize the likelihood as follows: \[\begin{align*} \mathcal{L}(\theta \vert \mathbf{x}) = \prod_{i=1}^n \frac{1}{\theta} e^{x_i} e^{-e^{x_i}/\theta} = e^{\sum_{i=1}^n x_i} \theta^{-n} e^{-(\sum_{i=1}^n e^{x_i})/\theta} \,. \end{align*}\] The first term does not contain \(\theta\) and thus can be ignored. Thus we identify \(Y = \sum_{i=1}^n e^{X_i}\) as a sufficient statistic.

(b) We can determine \(E[Y]\) by noticing that \(Y \sim\) Gamma\((n,\theta)\), as stated in the question…so \(E[Y] = n\theta\) and \(E[Y/n] = \theta\). Thus the MVUE for \(\theta\) is \((\sum_{i=1}^n e^{X_i})/n\).

(c) The MVUE will be a function of the sufficient statistic for \(\theta\), so let’s try \((\sum_{i=1}^n e^{X_i})^2\): \[\begin{align*} E\left[\left(\sum_{i=1}^n e^{X_i}\right)^2\right] = V\left[\left(\sum_{i=1}^n e^{X_i}\right)\right] + E\left[\sum_{i=1}^n e^{X_i}\right]^2 = n \theta^2 + (n\theta)^2 = n(n+1)\theta^2 \,. \end{align*}\] Therefore \((\sum_{i=1}^n e^{X_i})^2/(n(n+1))\) is the MVUE for \(\theta^2\).

Problem 17

(a) We factorize the likelihood: \[\begin{align*} \mathcal{L}(a \vert \mathbf{x}) = \prod_{i=1}^n \sqrt{\frac{2}{\pi}} \frac{x_i^2}{a^3} e^{-x_i^2/(2a^2)} = \left[ \left(\frac{2}{\pi}\right)^{n/2} \left( \prod_{i=1}^n x_i^2 \right) \right] \times \left[ \frac{1}{a^{3n}} e^{-(\sum_{i=1}^n x_i^2)/(2a^2)} \right] = h(\mathbf{x}) \times g(a,\mathbf{x}) \,. \end{align*}\] We can read off from the \(g(\cdot)\) function term that \(Y = \sum_{i=1}^n X_i^2\). (Including the minus sign, for instance, is fine because a function of a sufficient statistic is itself sufficent and we will get to the same MVUE in the end.)

(b) We utilize the shortcut formula: \[\begin{align*} E[X^2] = V[X] + (E[X])^2 = a^2 \frac{(3 \pi - 8)}{\pi} + (2a)^2 \frac{2}{\pi} = 3 a^2 + \frac{8 a^2}{\pi} - \frac{8 a^2}{\pi} = 3 a^2 \,. \end{align*}\]

(c) We compute the expected value for \(Y\): \[\begin{align*} E[Y] = E\left[\sum_{i=1}^n X_i^2\right] = \sum_{i=1}^n E[X_i^2] = n E[X^2] = 3 n a^2 \,. \end{align*}\] Thus the expected value for \(Y/(3n)\) is \(a^2\): \[\begin{align*} \widehat{a^2}_{MVUE} = \frac{1}{3n} \sum_{i=1}^n X_i^2 \,. \end{align*}\]

(d) There is no invariance principle for the MVUE. Maybe the desired result holds and maybe it doesn’t, but we cannot simply state that it does.

Problem 18

We are constructing an upper-tail test where the test statistic is trivially \(Y = X\). (So the NP Lemma does not really come into play here, given the lack of choices for the test statistic.) The expected value of \(Y\) is \[\begin{align*} E[Y] = \int_0^2 y f_Y(y) dy = \int_0^2 \frac{\theta}{2^\theta} y^{\theta} dy = \left. \frac{\theta}{2^\theta} \frac{y^{\theta+1}}{\theta+1} \right|_0^2 = \frac{2\theta}{\theta+1} \,. \end{align*}\] \(E[Y]\) increases as \(\theta\) increases, so we will be on the “yes” line of the hypothesis test reference table. Hence the rejection region will be of the form \[\begin{align*} y_{\rm obs} > F_Y^{-1}(1-\alpha \vert \theta_o) \,. \end{align*}\] The cdf \(F_Y(y)\) is \[\begin{align*} F_Y(y) = \int_0^y \frac{\theta}{2^\theta} u^{\theta-1} du = \left. \frac{u^\theta}{2^\theta}\right|_0^y = \left(\frac{y}{2}\right)^\theta \,, \end{align*}\] and the inverse cdf \(F_Y^{-1}(q)\) is \(y = 2q^{1/\theta}\). Hence the test we seek rejects the null hypothesis if \[\begin{align*} y_{\rm obs} > 2(1-\alpha)^{1/\theta_o} \,. \end{align*}\] The rejection-region boundary does not depend on \(\theta_a\), so we know that the test is the most powerful one for all alternative values \(\theta_a > \theta_o\)…thus it is a uniformly most powerful test.

Problem 19

(a) The likelihood is \[\begin{align*} \mathcal{L}(\beta \vert \mathbf{x}) = \prod_{i=1}^n \frac{\theta}{\beta}x_i^{\theta-1}\exp\left(-\frac{x_i^\theta}{\beta}\right) = \left[ \theta x_i^{\theta-1} \right] \times \left[ \frac{1}{\beta} \exp\left(-\frac{1}{\beta} x_i^\theta \right) \right] = h(\mathbf{x}) \times g(\beta,\mathbf{x}) \,, \end{align*}\] thus a sufficient statistic for \(\beta\) is \(Y = \sum_{i=1}^n X_i^\theta\).

(b) We are given that \(X^\theta \sim\) Exp(\(\beta\)). The mgf for an exponential distribution is \[\begin{align*} m_X(t) = (1 - \theta t)^{-1} \,, \end{align*}\] and hence the mgf for \(Y = \sum_{i=1}^n X_i\) will be \[\begin{align*} m_Y(t) = \prod_{i=1}^n (1 - \theta t)^{-1} = \left[ (1 - \theta t)^{-1} \right]^n = (1 - \theta t)^{-n} \,. \end{align*}\] Following the hint given in the question, we find that \(Y\) is a gamma-distributed random variable with “shape” parameter \(n\) and “scale” parameter \(\theta\). (There are two common parameterizations of the gamma distribution\(-\)shape/scale and shape/rate\(-\)and it is imperative to determine the correct one! This will impact the answer to part (c).)

(c) The statistic \(Y\) has expected value \(E[Y] = n\theta\), which increases with \(\theta\). Hence we utilize the upper-tail/yes line of the hypothesis test reference table: \(y_{\rm RR} = F_Y^{-1}(1 - \alpha \vert \theta_o)\), or, in code,

y.rr <- qgamma(1-alpha,shape=n,scale=theta)

Problem 20

(a) The moment-generating function for the random variable \(X\) is \[\begin{align*} m_X(t) = E\left[e^{tX}\right] &= \int_b^\infty e^{tx} \frac{1}{\theta} e^{-(x-b)/\theta} dx \\ &= e^{b/\theta} \frac{1}{\theta} \int_b^\infty e^{-x(1/\theta - t)} dx \\ &= e^{b/\theta} \frac{1}{\theta} \frac{e^{-b(1/\theta - t)}}{(1/\theta-t)} \\ &= e^{bt} (1-t\theta)^{-1} \,. \end{align*}\] (Here we make the implicit assumption that \(t < 1/\theta\), so that the integral evaluated at \(\infty\) is zero.) This is the final answer, but recall that when \(X = U+b\), \(m_X(t) = e^{bt} m_U(t)\). Since we recognize that \((1-t\theta)^{-1}\) is the mgf for an exponential distribution, we can state that \(U = X-b\) is an exponentially distributed random variable.

(b) The mgf for \(Y = \sum_{i=1}^n X_i\) is \[\begin{align*} m_Y(t) = \prod_{i=1}^n e^{bt} (1-t\theta)^{-1} = \left[ e^{bt} (1-t\theta)^{-1} \right]^n = e^{nbt} (1-t\theta)^{-n} \,. \end{align*}\]

(c) Going back to our answer for (a) (and our answer for the previous problem), we recognize that the mgf for \(\sum_{i=1}^n U_i\) is \((1-t\theta)^{-n}\), which is the mgf for a gamma distribution with shape parameter \(n\) and scale parameter \(\theta\). Hence \(Y' = Y - nb \sim \text{Gamma}(n,\theta)\).

(d) We are on the lower-tail/yes line of the hypothesis test reference table: \(y_{\rm RR}' = F_Y^{-1}(\alpha \vert \theta_o)\), or, in code,

y.rr.prime <- qgamma(alpha,shape=n,scale=theta)

We would reject the null hypothesis if \(y_{\rm obs} - nb < y_{\rm RR}'\). Because this test is constructed using a sufficient statistic and because no value of the alternative hypothesis appears in the definition of the rejection region, we indeed have defined a uniformly most powerful test of \(H_o : \theta = \theta_o\) versus \(H_a : \theta < \theta_o\).

Problem 21

(a) Let’s first find a sufficient statistic: \[\begin{align*} \mathcal{L}(\theta \vert \mathbf{x}) = \prod_{i=1}^n \theta e^{-\theta x_i} = \theta^n e^{-\theta \sum_{i=1}^n x_i} \,. \end{align*}\] A sufficient statistic is \(Y = \sum_{i=1}^n X_i\). We are conducting a lower-tail test, and since \(E[X] = 1/\theta\) decreases as \(\theta\) increases, we are on the “no” line of the reference table. We reject the null if \(y_{\rm obs} = \sum_{i=1}^n x_i > y_{\rm RR}\).

(b) \(\theta_o\) is plugged in to compute the rejection-region boundary, but \(\theta_a\) does not appear at all. Hence the defined test is uniformly most powerful, since it is most powerful for any value of \(\theta_a < \theta_o\).

Problem 22

(a) The sampling distribution is Binom(\(nk,p\)). We can determine this using the method of moment-generating functions, if necessary.

(b) \(E[Y] = nkp\) increases with \(p\), so we are on the upper-tail/“yes” line of the hypothesis test reference tables. The rejection-region boundary is given by \(F_Y^{-1}(1-\alpha \vert \theta_o)\), or, in code, with \(p_o\) in place of \(\theta_o\),

qbinom(1-alpha,n*k,p.o)

(c) For an upper-tail/yes test, the \(p\)-value is \(1 - F_Y(y_{\rm obs} \vert \theta_o)\). In code, with \(p_o\) in place of \(\theta_o\), the \(p\)-value is

1 - pbinom(y.obs,n*k,p.o)

However, we have to apply a discreteness correction, because otherwise we will not be summing over the correct range of \(y\) values, i.e., our \(p\)-value will be wrong. Here, that factor is \(-1\), applied to the input. So…

1 - pbinom(y.obs-1,n*k,p.o)

is the final answer.

Problem 23

This is straightforward if we remember to set the link function to the equation for the line: \[\begin{align*} -(Y \vert x)^{-1} = \beta_0 + \beta_1 x ~~~ \Rightarrow ~~~ (Y \vert x)^{-1} = -\beta_0 - \beta_1 x ~~~ \Rightarrow ~~~ Y \vert x = (-\beta_0 - \beta_1 x)^{-1} \,. \end{align*}\]

Problem 24

(a) The degrees of freedom for the residual deviance is \(n-p\), where \(p\) is the number of parameters (here, two: \(\beta_0\) and \(\beta_1\)). Hence \(n = 32\).

(b) \(\beta_1\) is set to zero to compute the null deviance. So \(-2\log\mathcal{L}_{\rm max} = 43.230\).

(c) The odds are \(O(x) = \exp(\hat{\beta}_0 + \hat{\beta}_1 x)\) or just \(\exp(\hat{\beta}_0)\) for \(x = 0\), meaning thet \(O(x=0) = \exp(12.040)\).

(d) The estimated slope \(\hat{\beta}_1\) is negative, and we know that \(O(x+1) = O(x) \exp(\hat{\beta}_1)\), so we know that \(O(x+1) < O(x)\)…the odds decrease as \(x\) increases.

Problem 25

(a) We have that \[\begin{align*} O(x) = \frac{p \vert x}{1 - p \vert x} = \frac{0.1}{1-0.1} = \frac19 = 0.111 \,. \end{align*}\]

(b) The new odds are \[\begin{align*} O(589+100) = \exp(\hat{\beta}_0 + \hat{\beta_1}(589+100)) = O(589) \exp(100\hat{\beta}_1) = \frac19 \exp(0.14684) = 0.129 \,. \end{align*}\]

(c) We have that \(Y_1 = 0\) and \(\hat{Y}_i = 0.07\), so \[\begin{align*} d_1 &= \mbox{sign}(Y_1-\hat{Y}_1)\sqrt{-2[Y_1\log\hat{Y}_1+(1-Y_1)\log(1-\hat{Y}_1)]} = \mbox{sign}(-0.07) \sqrt{-2\log(0.93)} \\ &= -\sqrt{-2\log(0.93)} = 0.381 \,. \end{align*}\]

(d) The null deviance is computed assuming \(\beta_1 = 0\). This model lies “farther” from the observed data than the model with \(\hat{\beta}_1 = 0.00147\), meaning it deviates more from the data, meaning that the deviance would be higher.

Problem 26

Let’s start by collecting the basic pieces of information that we would combine in a Naive Bayes regression model: \[\begin{align*} p(0) = 3/5 ~~\mbox{and}~~ p(1) = 2/5 \,, \end{align*}\] where \(0\) and \(1\) are the two response (i.e., \(Y\)) values. Next up, the conditionals: \[\begin{align*} p(x1 = N \vert 0) = 2/3 ~~ &\mbox{and}& ~~ p(x1 = Y \vert 0) = 1/3 \\ p(N \vert 1) = 1/2 ~~ &\mbox{and}& ~~ P(Y \vert 1) = 1/2 \\ \\ p(x2 = T \vert 0) = 2/3 ~~ &\mbox{and}& ~~ p(x2 = F \vert 0) = 1/3 \\ p(T \vert 1) = 1/2 ~~ &\mbox{and}& ~~ P(F \vert 1) = 1/2 \,. \end{align*}\] The estimated probability of observing a datum of Class 0 given Y and F is thus \[\begin{align*} p(0 \vert Y,F) &= \frac{p(Y \vert 0) p(F \vert 0) p(0)}{p(Y \vert 0) p(F \vert 0) p(0) + p(Y \vert 1) p(F \vert 1) p(1)} \\ &= \frac{1/3 \times 1/3 \times 3/5}{1/3 \times 1/3 \times 3/5 + 1/2 \times 1/2 \times 2/5} \\ &= \frac{1/15}{1/15 + 1/10} = \frac{2/30}{2/30+3/30} = \frac{2}{5} \,. \end{align*}\]

Problem 27

(a) The pdf is of the form \[\begin{align*} k x^{\alpha-1} (1-x)^{\beta-1} \,, \end{align*}\] with \(0 \leq x \leq 1\), so what we have is a beta distribution: \(X \sim\) Beta\((1,2)\).

(b) We have that \[\begin{align*} E[X] = \frac{\alpha}{\alpha+\beta} = \frac{1}{3} \end{align*}\] and \[\begin{align*} E[X^2] = V[X] + (E[X])^2 = \frac{\alpha\beta}{(\alpha+\beta)^2(\alpha+\beta+1)} + \left(\frac{1}{3}\right)^2 = \frac{2}{36} + \frac{4}{36} = \frac{1}{6} \,. \end{align*}\]

(c) These expressions are straightforward to evaluate: \[\begin{align*} E[C] = E[10X] = 10E[X] = \frac{10}{3} = 3.333 \end{align*}\] and \[\begin{align*} V[C] = E[C^2] - (E[C])^2 = E[100X^2] - \frac{100}{9} = 100E[X^2] - \frac{100}{9} = \frac{150}{9} - \frac{100}{9} = \frac{50}{9} = 5.556 \,. \end{align*}\]

Problem 28

(a) \(f_X(x) = 12x - 24x^2 + 12x^3 = 12x(1-x)^2\) for \(x \in [0,1]\)…so this is a Beta(2,3) distribution.

(b) \(X \sim {\rm Beta}(2,3) \Rightarrow E[X] = \alpha/(\alpha+\beta) = 2/(2+3) = 2/5 = 0.4\) ,.

Problem 29

One way to solve this problem is to utilize the shortcut formula: \(E[X^2] = V[X] + E[X]^2\). With this in hand: \[\begin{align*} V[X] = \frac{\alpha \beta}{(\alpha + \beta)^2(\alpha + \beta + 1)} = \frac{6}{25 \cdot 6} = \frac{1}{25} \end{align*}\] and \[\begin{align*} E[X] = \left(\frac{\alpha}{\alpha + \beta}\right)^2 = \left(\frac{2}{5}\right)^2 = \frac{4}{25} \,. \end{align*}\] Therefore, \[\begin{align*} E[X^2] = \frac{1}{25} + \frac{4}{25} = \frac{1}{5} \,. \end{align*}\] A second way to solve this problem is by brute-force integration: \[\begin{align*} E[X^2] &= \int_0^1 x^2 \frac{x(1-x)^2}{B(2,3)}dx = \int_0^1 \frac{x^3(1-x)^2}{B(2,3)}dx = \int_0^1 \frac{x^3(1-x)^2}{B(2,3)} \frac{B(4,3)}{B(4,3)}dx \\ &= \frac{B(4,3)}{B(2,3)}\underbrace{\int_0^1 \frac{x^3(1-x)^2}{B(4,3)}dx}_{=1} \\ &= \frac{B(4,3)}{B(2,3)} = \frac{\Gamma(4) \Gamma(3)}{\Gamma(7)}\frac{\Gamma(5)}{\Gamma(2)\Gamma(3)}\ = \frac{\Gamma(4) \Gamma(5)}{\Gamma(2)\Gamma(7)} = \frac{3! 4!}{1!6!} = \frac{1}{5} \,. \end{align*}\]

Problem 30

(a) The median is the second sampled datum. The pdf for \(X\) is \(f_X(x) = 3x^2\) and \(F_X(x) = x^3\), both for \(x \in [0,1]\). Thus \[\begin{align*} f_{(2)}(x) = \frac{3!}{1!1!} [x^3]^1 [1 - x^3]^1 3x^2 = 18 x^5 (1 - x^3) \,, \end{align*}\] for \(x \in [0,1]\), and \[\begin{align*} E[X_{(2)}] &= \int_0^1 x 18 x^5 (1-x^3) dx = 18 \int_0^1 (x^6 - x^9) dx \\ &= 18 \left( \left.\frac{x^7}{7}\right|_0^1 - \left.\frac{x^{10}}{10}\right|_0^1 \right) = 18 \left( \frac{1}{7}-\frac{1}{10} \right) = \frac{18 \cdot 3}{70} = \frac{27}{35} \,. \end{align*}\]

Problem 31

(a) This is a Beta(2,2) distribution.

(b) We can determine \(c\) via brute-force integration: \[\begin{align*} c \int_0^1 x(1-x) dx = c\left[ \int_0^1 x dx - \int_0^1 x^2 dx \right] &= c \left[ \left.\frac{x^2}{2}\right|_0^1 - \left.\frac{x^3}{3}\right|_0^1 \right] \\ &= c \left[ \frac{1}{2} - \frac{1}{3} \right] \\ &= c \frac{1}{6} = 1 ~~\Rightarrow~~ c = 6 \,. \end{align*}\] Alternatively, we can recognize that \[\begin{align*} c &= \frac{1}{B(2,2)} = \frac{\Gamma(4)}{\Gamma(2) \Gamma(2)} = \frac{3!}{1! 1!} = 6 \,. \end{align*}\]

(d) We have that \[\begin{align*} P(X \leq 1/4 \vert X \leq 1/2) &= \frac{P(X \leq 1/4 \cap X \leq 1/2)}{P(X \leq 1/2)} = \frac{P(X \leq 1/4)}{P(X \leq 1/2)} = \frac{P(X \leq 1/4)}{1/2} \\ &= 2P(X \leq 1/4) = 2 \int_0^{1/4} 6 x (1-x) dx = 12 \int_0^{1/4} x (1-x) dx \\ &= 12 \left[ \left.\frac{x^2}{2}\right|_0^{1/4} - \left.\frac{x^3}{3}\right|_0^{1/4} \right] = 12 \left[ \frac{1}{32} - \frac{1}{192} \right] = 12 \frac{5}{192} = \frac{60}{192} = \frac{30}{96} = \frac{5}{16} \,. \end{align*}\]

Problem 32

(a) We carry out a chi-square goodness-of-fit test: \[\begin{align*} W = \sum_{i=1}^n \frac{(X_i - kp_i)^2}{kp_i} = \frac{1}{7}[(10-7)^2+(5-7)^2+(6-7)^2] = 2 \,. \end{align*}\]

(b) There are \(m=3\) outcomes, but we lose one degree of freedom because of the constraint that \(\sum_{i=1}^m X_i = 21\), so the number of degrees of freedom is 2.

Problem 33

(a) The correct answer is homogeneity, since the researcher is splitting the respondents into groups by giving them the different types of leaflets to read, and the goal is to determine the willingness to spend government funding is homogenous across the type of pamphlets.

(b) Since chi-square statistics involving summing squared differences over all combinations of leaflets and spending opinions, there are \(12 = 3 \times 4\) terms.

(c) Since the all the information but one for each factor is sufficent, the test statistics will follow a chi-square distribution with \(6 = (3-1) \times (4-1)\) degrees of freedom.

(d) The correct answer is (i) since \(p\)-value is defined as a probability of events at least as extreme as the what actually observed under the null hypothesis, and larger values of the test statistic here correspond to larger differences between what we would expect under the null and what we observe.

Problem 34

(a) The exponential distribution with mean 1 is \(e^{-x}\), for \(x \geq 0\). Therefore, the probabilities for arriving between 1 and 2 minutes after the previous person is given by \[\begin{align*} \int_1^2 e^{-x} dx = -\left.e^{-x}\right|_1^2 = 0.233 \,. \end{align*}\]

(b) The probabilities for the other two “bins” are \[\begin{align*} \int_0^1 e^{-x} dx &= -\left.e^{-x}\right|_0^1 = 0.632 \\ \int_2^\infty e^{-x} dx &= 1 - 0.632 - 0.233 = 0.135 \,. \end{align*}\] We can now carry out a chi-square goodness-of-fit test: \[\begin{align*} W = \frac{(52-63.2)^2}{63.2} + \frac{(23-23.3)^2}{23.3} + \frac{(25-13.5)^2}{13.5} = 1.985 + 0.004 + 9.796 = 11.785 \,, \end{align*}\] and \(W \sim \chi_2^2\). The rejection region is \(W > w_{\rm RR} = 5.991\) (i.e., qchisq(0.95,2)), and the \(p\)-value is 0.0028 (i.e., 1-pchisq(11.785,2)). We have sufficient evidence to reject the null hypothesis and conclude that the time elapsed between people walking through a particular door is not exponentially distributed with mean 1.

(c) If \(n = 10\) and there are 3 bins, then there is no way that \(np_i \geq 5\) for all bins. Thus the chi-square goodness-of-fit test should not be applied.

Problem 35

(a) We have that \(p_{\rm out} = 8/9\) and \(p_{\rm in} = 1/9\), so \(kp_{\rm out} = 180 (8/9) = 160\) and \(kp_{\rm in} = 180 (1/9) = 20\).

(b) We perform a chi-square goodness-of-fit test: \[\begin{align*} W = \frac{(150-160)^2}{160} + \frac{(30-20)^2}{20} = \frac{100}{160} + \frac{100}{20} = \frac58 + 5 = 5.625~(\mbox{or}~5~5/8) \,. \end{align*}\]

(d) The rejection region for a chi-square GoF test is \(W > w_{\rm RR}\), so, since 5.625 is greater than 3.841, we would reject the null hypothesis.

Chapter 4

Problem 1

(a) This is a Poisson problem: \(X \sim\) Poisson(\(\lambda = 2 \cdot 1/4 = 1/2\)). So \[\begin{align*} \mu = E[X] = \lambda = 1/2 ~~\text{and}~~ \sigma = \sqrt{V[X]} = \sqrt{\lambda} = \sqrt{1/2} \approx 0.707 \,. \end{align*}\] Thus \[\begin{align*} P(1/2 \leq X \leq 1/2+1.414) = p(1) = \frac{\lambda^1}{1!} e^{-\lambda} = \frac12 e^{-1/2} = 0.303 \,. \end{align*}\]

(b) We have that \(X \sim\) Exp(\(\beta = 1/2\)) (since there is a half-hour on average between calls). By the memorylessness property, \(P(X > 1/2 \vert X > 1/4) = P(X > 1/4)\). Thus \[\begin{align*} P(X > 1/4) = \int_{1/4}^\infty \frac{1}{\beta} e^{-x/\beta} dx = \int_{1/4}^\infty 2 e^{-2x} dx = \left.-e^{-2x}\right|_{1/4}^\infty = e^{-1/2} = 0.607 \,. \end{align*}\]

(c) The overall time \(T\) is \(10X + (10-X)\), where \(X\), the number of calls from the friend, is sampled from a binomial distribution with \(k = 10\) and \(p = 0.2\). Thus the average total number of minutes is \[\begin{align*} E[T] = E[10X + (10-X)] = E[9X+10] = 9E[X] + 10 = 9kp + 10 = 28 \,. \end{align*}\]

Problem 2

(a) The number of (successful) shots can be infinite; only on average is the number of shots in eight minutes going to be four. So we are working with a Poisson distribution whose parameter \(\lambda\) (the expected number of successful shots) is \[\begin{align*} \lambda = (1 \quad \frac{\text{shot}}{2\text{min}})(\frac{1}{2} \frac{\text{success}}{\text{shot}})(4 \quad 2\text{min}) = 2 \,. \end{align*}\] Let \(X\) be the number of successful shots. Then \[\begin{align*} P(X \leq 1) = \frac{\lambda^0}{0!}e^{-\lambda} + \frac{\lambda^1}{1!}e^{-\lambda} = e^{-2}(1+2) = 3e^{-2} \,. \end{align*}\]

(b) We know that \(E[X] = \lambda = 2\), \(V[X] = \lambda = 2\), and \(\sigma = \sqrt{\lambda} = \sqrt{2}\). Thus \[\begin{align*} P(2 - \sqrt{2} < X < 2 + \sqrt{2}) &= p_X(1) + p_X(2) + p_X(3) = e^{\lambda}\left(\lambda + \frac{\lambda^2}{2} + \frac{\lambda^3}{6} \right) \\ &= e^{2}\left(2 + 2+ \frac{8}{6} \right) = \frac{16}{3} e^{-2} = 0.722 \,. \end{align*}\]

Problem 3

The particular event in question happens 8 times per year in the three states, and thus the expected number of events in a two-year window is 16. The appropriate distribution in this case is the Poisson distribution. If the total number of observed events is denoted \(X\), then \(X \sim\) Poisson(\(\lambda\) = 16), and \(E[X] = V[X] = 16\).

Problem 4

(a) We utilize the shortcut formula: \[\begin{align*} E[X^2] = V[X] + (E[X])^2 = 2\sigma^2 - \frac{\pi}{2}\sigma^2 + \frac{\pi}{2}\sigma^2 = 2\sigma^2 \,. \end{align*}\]

(b) The first population moment is \(\mu_1' = E[X] = \sigma\sqrt{\pi/2}\) and the first sample moment is \(m_1' = (1/n)\sum_{i=1}^n X_i = \bar{X}\). We set these equal and determine that \[\begin{align*} \hat{\sigma}_{MoM} = \sqrt{\frac{2}{\pi}} \bar{X} \,. \end{align*}\]

(c) The bias is \(E[\hat{\theta}-\theta] = E[\hat{\theta}] - \theta\), or \[\begin{align*} B[\hat{\theta}_{MoM}] &= E\left[\sqrt{\frac{2}{\pi}} \bar{X}\right] - \sigma = \sqrt{\frac{2}{\pi}} E\left[\bar{X}\right] - \sigma = \sqrt{\frac{2}{\pi}} E\left[X\right] - \sigma = \sqrt{\frac{2}{\pi}} \sqrt{\frac{\pi}{2}} \sigma - \sigma = 0 \,. \end{align*}\]

(d) The variance is \[\begin{align*} V[\hat{\theta}_{MoM}] &= V\left[\sqrt{\frac{2}{\pi}} \bar{X}\right] = \frac{2}{\pi} V\left[\bar{X}\right] = \frac{2}{\pi} \frac{V\left[X\right]}{n} = \frac{2}{n\pi} \frac{(4-\pi)\sigma^2}{2} = \frac{(4-\pi)\sigma^2}{n\pi} \,. \end{align*}\]

(e) The second population moment is \(\mu_2' = E[X^2] = 2\sigma^2\) (from part a) and the second sample moment is \(m_2' = (1/n)\sum_{i=1}^n X_i^2 = \overline{X^2}\). We set these equal and determine that \[\begin{align*} \widehat{\sigma^2}_{MoM} = \frac{\overline{X^2}}{2} \,. \end{align*}\]

Problem 5

(a) We have that \(\mu_1' = \frac{1}{p}\) and \(m_1' = X\). It follows from moment equation \(\mu_1' = m_1'\) that \(\frac{1}{p} = X\), so \(\hat{p}_{MoM} = \frac{1}{X}\).

(b) We have that \(\mu_2' = \frac{1 - p}{p^2} + \frac{1}{p^2} = \frac{2 - p}{p^2}\) and \(m_2' = X^2\). It follow from the second moment equation \(\mu_2' = m_2'\) that \[\begin{align*} \frac{2-p}{p^2} & = X^2 \\ \Rightarrow ~~~ 2-p & = p^2 X^2 \\ \Rightarrow ~~~ p^2 X^2 +p -2 &= 0 \\ \Rightarrow ~~~ \hat{p}_{MoM} & = \frac{-1 + \sqrt{1 + 8 X^2}}{2X^2} \,. \end{align*}\]

Problem 6

(a) The expected value for a beta distribution is \(E[X] = \alpha/(\alpha+\beta)\), so, using the first sample moment, we get that \[\begin{align*} E[X] &= \bar{X} \\ \Rightarrow ~~~ \frac{\alpha}{\alpha+\beta} &= \bar{X} \\ \Rightarrow ~~~ \frac{\alpha+\beta}{\alpha} = 1 + \frac{\beta}{\alpha} &= \frac{1}{\bar{X}} \\ \Rightarrow ~~~ \frac{\beta}{\alpha} &= \frac{1}{\bar{X}}-1 \\ \Rightarrow ~~~ \hat{\beta}_{MoM} &= \alpha\left(\frac{1}{\bar{X}}-1\right) \,. \end{align*}\]

(b) There is no invariance property for the method-of-moments estimator, so the answer is no.

Problem 7

(a) The argument list is fine.

(b) We need to change mean(X) to sum(X) and lambda to n*lambda.

(c) The reference table tells us that a one-sided lower bound where \(E[U]\) increases with \(\lambda\) will have a value of \(q\) equal to \(1-\alpha\). So we plug in 0.95.

(d) Confidence intervals derived from discrete sampling distributions will have coverages \(\geq 1-\alpha\), so, here, we would say “greater than or equal to 95%.”

Problem 8

(a) The likelihood ratio test statistic is \[\begin{align*} \lambda_{LR} = \frac{\mbox{sup}_{\theta \in \Theta_o} \mathcal{L}(\theta \vert \mathbf{x})}{\mbox{sup}_{\theta \in \Theta} \mathcal{L}(\theta \vert \mathbf{x})} \end{align*}\] Here, that becomes \[\begin{align*} \lambda_{LR} &= \frac{\mathcal{L}(p_o \vert \mathbf{x})}{\mathcal{L}(\hat{p}_{MLE} \vert \mathbf{x})} = \frac{p_o^{\sum_{i=1}^n X_i}(1-p_o)^{n-\sum_{i=1}^n X_i}}{\bar{X}^{\sum_{i=1}^n X_i}(1-\bar{X})^{n-\sum_{i=1}^n X_i}} = \frac{p_o^U(1-p_o)^{n-U}}{\bar{X}^U(1-\bar{X})^{n-U}} \,. \end{align*}\]

(b) The test is a two-sided test, so we cannot proclaim it to be uniformly most powerful. However, it very well may be…we just cannot say with the information we have at hand. So: “maybe.”

Problem 9

(a) The maximum-likelihood estimate is \[\begin{align*} \ell(\lambda \vert x) &= x \log \lambda - \log x! - \lambda \\ \Rightarrow ~~~ \ell'(\lambda \vert x) &= \frac{x}{\lambda} - 1 = 0\\ \Rightarrow ~~~ \hat{\lambda}_{MLE} &= X \,. \end{align*}\] which here takes on the value \(x_{\rm obs}\).

(b) The likelihood-ratio test statistic is \[\begin{align*} \frac{\mbox{sup}_{\theta \in \Theta_o}\mathcal{L}(\theta \vert \mathbf{x})}{\mbox{sup}_{\theta \in \Theta}\mathcal{L}(\theta \vert \mathbf{x})} \,. \end{align*}\] Here, that means that for the numerator, we insert the Poisson pmf (remember: one datum) with \(\lambda_o\) plugged in, i.e., \[\begin{align*} \frac{\lambda_o^{x_{\rm obs}}}{x_{\rm obs}!} e^{-\lambda_o} \,, \end{align*}\] while for the denominator, we plug in the MLE for \(\lambda\), i.e., \[\begin{align*} \frac{x_{\rm obs}^{x_{\rm obs}}}{x_{\rm obs}!} e^{-x_{\rm obs}} \,. \end{align*}\] So the ratio is \[\begin{align*} \left( \frac{\lambda_o}{x_{\rm obs}} \right)^{x_{\rm obs}} e^{-(\lambda_o-x_{\rm obs})} \,. \end{align*}\]

(c) The expression is \[\begin{align*} W = -2 \log \lambda_{LR} = -2 x_{\rm obs} \log \left( \frac{\lambda_o}{x_{\rm obs}} \right) + 2 (\lambda_o-x_{\rm obs}) \,. \end{align*}\]

(d) \(W\) is sampled from a chi-square distribution for 1 degree of freedom.

Problem 10

(a) The factorized likelihood is \[\begin{align*} \mathcal{L}(\lambda \vert \mathbf{x}) = \prod_{i=1}^n \frac{\lambda^{x_i}}{x_i!}e^{-\lambda} = \left(\frac{1}{\prod_{i=1}^n x_i!}\right) \times \lambda^{\sum_{i=1}^n x_i} e^{-n\lambda} = h(\mathbf{x}) \times g(\lambda,\mathbf{x}) \,. \end{align*}\] The sufficient statistic is \(Y = \sum_{i=1}^n X_i\).

(b) The sum of \(n\) iid Poisson random variables is a Poisson random variable with parameter \(n\lambda\). The moment-generating function for a Poisson random variable is \(m_X(t) = \exp(\lambda(e^t-1))\), so the mgf for \(Y\) is \[\begin{align*} m_Y(t) = \prod_{i=1}^n m_{X_i}(t) = \left[ m_{X_i}(t) \right]^n = \exp(n\lambda(e^t-1)) \,. \end{align*}\] This is the mgf for a Poisson(\(n\lambda\)) distribution.

(c) Recall that in an LRT context, \(\Theta = \Theta_o \cup \Theta_a\); in other words, the null must contain all possible values of \(\theta\) that are not in the alternative. Hence: \(H_o : \theta \geq \theta_o\). This inequality does not actually change how the test is constructed, but does change how we interpret it: the true \(\alpha\) for this test will be less than or equal to the stated \(\alpha\).

(d) We are performing a lower-tail test, and we are on the “yes” line (since \(E[Y] = n\lambda\) increases with \(\lambda\)). From the reference table, that means \(y_{\rm RR}\) is equal to \(F_Y^{-1}(\alpha \vert n\lambda_o)\), which in code is qpois(0.05,n*lambda.o). (Recall that there are no discreteness corrections in rejection-region boundary computations.)

Problem 11

(a) The likelihood function for the sample is: \[\begin{align*} \mathcal{L}(\beta \vert \mathbf{x}) = \prod_{i=1}^{n} \frac{1}{\beta}e^{-x/\beta} = \frac{1}{\beta^n} e^{-\frac{\sum_{i=1}^{n}x_i}{\beta}} \,. \end{align*}\] Under the null \(H_0 : \beta = 1\), the likelihood function becomes \[\begin{align*} \mathcal{L}(\beta_0 \vert \mathbf{x}) = e^{-\sum_{i=1}^{n}x_i} = e^{-n\bar x} \,, \end{align*}\] while under the alternative, \[\begin{align*} \sup_{\beta > 0} \mathcal{L}(\beta \vert \mathbf{x}) &= \mathcal{L}(\hat{\beta}_{MLE} \vert \mathbf{x})\\ & = \frac{1}{\bar x^n} e^{-\frac{\sum_{i=1}^{n}x_i}{\bar x}} = \frac{1}{\bar x^n} e^{-n} \,. \end{align*}\] So the likelihood ratio test statistic is \[\begin{align*} \lambda_{LR} &= \frac{\mathcal{L}(\beta_o\vert \mathbf{x})}{\sup_{\beta > 0} \mathcal{L}(\beta \vert \mathbf{x})}\\ &= \frac{e^{-n\bar x}}{\frac{1}{\bar x^n} e^{-n}} \\ &= \bar x^ne^{-n(\bar x-1)} \,. \end{align*}\]

(b) Under the null hypothesis, the number of degrees of freedom is \(r_o = 0\), because \(\beta = 1\) is set to a constant. Under the alternative hypothesis, the number of degrees of freedom is \(r = 1\), because we have one free parameter: \(\beta\). Therefore, according to Wilks’ theorem, the degree of freedom of the \(\chi^2\) distribution is \(r - r_o = 1-0=1\). Under the large-\(n\) approximation, \(-2\log(\lambda_{LR}) \sim \chi^2(1)\). Therefore, the rejection region corresponds to: \[\begin{align*} -2\log(\lambda) &> \chi^2_{0.95, 1}\\ \Rightarrow ~~~ -2\log\left(\bar x^ne^{-n(\bar x-1)}\right) &> \chi^2_{0.95,1} = 3.84 \\ \Rightarrow ~~~ n\left(\log(\bar x) - \bar x+1\right) &< \frac{3.84}{2} \,. \end{align*}\]

Problem 12

(a) \(H_0: p = p_0 = 0.5\) and \(H_a: p \neq 0.5\)

(b) \(\Theta_0 = \{p_0\}\) and \(\Theta_a = \{p\, \vert\, p \in [0,1] ~~\text{and}~~ p \neq p_0\}\)

(d) The likelihood ratio test statistic is \[\begin{align*} \lambda = \frac{\mathcal{L}(p_0 \vert x)}{\mathcal{L}(\hat{p}_{MLE} \vert x)} = \frac{\frac{1000!}{550!450!} 0.5^{550} (1-0.5)^{450} }{ \frac{1000!}{550!450!} 0.55^{550} (1-0.55)^{450}} = \frac{0.5^{1000}}{0.55^{550} \cdot 0.45^{450}} = 0.00668 \,, \end{align*}\] where we make use of the fact that \(\hat{p}_{MLE} = x/n = 0.55\).

(e) We have that \(W_{\rm obs} = -2 \log(\lambda_{LR}) = 10.017\). According to Wilk’s theorem, the \(p\)-value is \[\begin{align*} \int_{W_{\rm obs}}^\infty f_W(w) dw \,, \end{align*}\] for 1 degree of freedom, or 1 - pchisq(10.017,1) (= 0.00155).

(f) We have sufficient evidence to reject the null hypothesis and thus to conclude that the coin is not a fair one.

Problem 13

By inspection, \(X \sim\) Gamma(3,2/3). Thus \(E[X] = \alpha \beta = 2\) and \(V[X] = \alpha \beta^2 = 3 (2/3)^2 = 4/3\).

Problem 14

(a) \(E[X] = \alpha \beta\) and \(V[X] = \alpha \beta^2\), so \(V[X]/E[X] = \beta = 10/5 = 2\), and \(\alpha = 5/2 = 2.5\).

(b) \(\beta = 2\) and \(\alpha = 2.5\) \(\Rightarrow\) chi-square distribution (for 5 degrees of freedom).

Problem 15

We have that \[\begin{align*} E[X^{-1}] &= \int_0^\infty \frac1x f_X(x) dx = \int_0^\infty \frac1x \frac{x^{\nu/2-1}}{2^{\nu/2}} \frac{e^{-x/2}}{\Gamma(\nu/2)} dx \\ &= \int_0^\infty \frac{x^{\nu/2-2}}{2^{\nu/2}} \frac{e^{-x/2}}{\Gamma(\nu/2)} dx = \int_0^\infty \frac{x^{\nu/2-2}}{2^{\nu/2}} \frac{2^{-1}}{2^{-1}} \frac{e^{-x/2}}{\Gamma(\nu/2)} \frac{\Gamma(\nu/2-1)}{\Gamma(\nu/2-1)} dx \\ &= 2^{-1} \frac{\Gamma(\nu/2-1)}{\Gamma(\nu/2)} \int_0^\infty \frac{x^{\nu/2-2}}{2^{\nu/2-1}} \frac{e^{-x/2}}{\Gamma(\nu/2-1)} dx = 2^{-1} \frac{\Gamma(\nu/2-1)}{\Gamma(\nu/2)} \\ &= \frac12 \frac{\Gamma(\nu/2-1)}{(\nu/2-1)\Gamma(\nu/2-1)} = \frac{1}{2(\nu/2-1)} = \frac{1}{\nu-2} \,. \end{align*}\]

Problem 16

We have that \[\begin{align*} E[X] = \int_0^\infty x f_X(x) dx &= \int_0^\infty x \frac{\beta^\alpha}{\Gamma(\alpha)} \frac{1}{x^{\alpha+1}} e^{-\beta/x} dx \\ &= \int_0^\infty \frac{\beta^\alpha}{\Gamma(\alpha)} \frac{1}{x^{\alpha}} e^{-\beta/x} dx \\ &= \frac{\beta^{\alpha}}{\beta{\alpha-1}} \frac{\Gamma(\alpha-1)}{\Gamma(\alpha)} \int_0^\infty \frac{\beta^{\alpha-1}}{\Gamma(\alpha-1)} \frac{1}{x^{\alpha}} e^{-\beta/x} dx \\ &= \frac{\beta^{\alpha}}{\beta{\alpha-1}} \frac{\Gamma(\alpha-1)}{\Gamma(\alpha)} \times 1 \\ &= \beta \frac{\Gamma(\alpha-1)}{(\alpha-1)\Gamma(\alpha-1)} \\ &= \frac{\beta}{\alpha-1} \,. \end{align*}\]

Problem 17

(a) If \(\beta = 2\) and \(\alpha\) is a half-integer or an integer, then \(X\) is sampled from a “chi-square” distribution. (Note that because \(\alpha\) is a half-integer here, we cannot answer “Erlang” or “exponential.”)

(b) The gamma pdf with \(\alpha = 3/2\) is \[\begin{align*} f_X(x) = \frac{x^{1/2}}{\beta^{3/2}} \frac{e^{-x/\beta}}{\Gamma(3/2)} \,, \end{align*}\] so the likelihood function is \[\begin{align*} \mathcal{L}(\beta \vert \mathbf{x} = \prod_{i=1}^n \frac{x_i^{1/2}}{\beta^{3/2}} \frac{e^{-x_i/\beta}}{\Gamma(3/2)} = \frac{\sqrt{\prod_{i=1}^n x_i}}{[\Gamma(3/2)]^n} \times \frac{e^{-(\sum_{i=1}^n x_i)/\beta}}{\beta^{3/2}} = h(\mathbf{x}) \times g(\beta,\mathbf{x}) \,. \end{align*}\] We can read off of the \(g(\cdot)\) function that \(Y = \sum_{i=1}^n X_i\) is a sufficient statistic for \(\beta\).

(c) We start by computing \[\begin{align*} E[Y] = E\left[ \sum_{i=1}^n X_i \right] = \sum_{i=1}^n E[X_i] = nE[X] = n \alpha \beta = \frac{3}{2}n\beta \,. \end{align*}\] Thus \[\begin{align*} E\left[\frac{2Y}{3n}\right] = \beta \end{align*}\] and \[\begin{align*} \hat{\beta}_{MVUE} = \frac{2Y}{3n} = \frac{2}{3}\bar{X} \,. \end{align*}\]

(d) The first population moment is \(\mu_1' = E[X] = (3/2)\beta\) and the first sample moment is \(m_1' = (1/n)\sum_{i=1}^n X_i = \bar{X}\). We set these equal and find that \[\begin{align*} \hat{\beta}_{MoM} = \frac{2}{3}\bar{X} \,. \end{align*}\]

(e) Because the MoM is equivalent to the MVUE, we know immediately that the bias of the MoM is 0.

Problem 18

We have that \[\begin{align*} E[X^{1/2}] = \int_0^\infty x^{1/2} f_X(x) dx &= \int_0^\infty x^{1/2} \frac{x^{\alpha-1}}{\beta^\alpha}\frac{e^{-x/\beta}}{\Gamma(\alpha)} dx \\ &= \int_0^\infty \frac{x^{\alpha-1/2}}{\beta^\alpha}\frac{e^{-x/\beta}}{\Gamma(\alpha)} dx \\ &= \int_0^\infty \frac{x^{\alpha-1/2}}{\beta^\alpha}\frac{e^{-x/\beta}}{\Gamma(\alpha)} \frac{\beta^{\alpha+1/2}}{\beta^{\alpha+1/2}} \frac{\Gamma(\alpha+1/2)}{\Gamma(\alpha+1/2)} dx \\ &= \frac{\beta^{\alpha+1/2}}{\beta^{\alpha}} \frac{\Gamma(\alpha+1/2)}{\Gamma(\alpha)} \int_0^\infty \frac{x^{\alpha-1/2}}{\beta^{\alpha+1/2}}\frac{e^{-x/\beta}}{\Gamma(\alpha+1/2)} dx \\ &= \frac{\beta^{\alpha+1/2}}{\beta^{\alpha}} \frac{\Gamma(\alpha+1/2)}{\Gamma(\alpha)} \\ &= \sqrt{\beta} \frac{\Gamma(\alpha+1/2)}{\Gamma(\alpha)} \,. \end{align*}\] We note that in general, \[\begin{align*} E[X^k] = \beta^k \frac{\Gamma(\alpha+k)}{\Gamma(\alpha)} \,. \end{align*}\]

Problem 19

(a) The overdispersion parameter in a negative binomial regression is dubbed “Theta” and is thus 214,488.

(b) If the overdispersion parameter is \(\infty\), then Poisson regression is recovered. The value here is sufficiently large that we can say with confidence that there is no overdispersion. (Backing up this conclusion are the nearly identical results since when learning both regression models.)

(c) The answer is the Likelihood Ratio test. The statistic is the difference in the deviance values, which we assume under the null is chi-square distributed for the difference in the numbers of degree of freedom.

(d) The null hypothesis in the LRT is \(\beta_1 = 0\) and the alternative is \(\beta_1 \neq 0\). The test statistic is so large (\(\approx\) 165), especially considering that the expected value for a chi-square distribution is 1 for 1 degree of freedom, that we can safely conclude that \(\beta_1 \neq 0\).

Chapter 5

Problem 1

Here, the first population moment is \(\mu_1' = E[X] = \frac{3}{2} \theta\) and the first sample moment is \(m_1' = \frac{1}{n} \sum X_i = \bar{X}\). So the MoM estimator for \(\theta\), following from setting \(\mu_1' = m_1'\), is \(\hat{\theta}_{MoM} = \frac{2}{3} \bar{X}\).

Problem 2

We have that \[\begin{align*} P(X > a+b \vert X > b) = \frac{P(X > a+b \cap X > b)}{P(X > b)} = \frac{P(X > a+b)}{P(X > b)} \,, \end{align*}\] and \[\begin{align*} P(X > a+b) = \int_{a+b}^1 dx = 1-(a+b) ~~~ P(X > b) = \int_b^1 dx = 1-b \,. \end{align*}\] So \[\begin{align*} \frac{P(X > a+b)}{P(X > b)} = \frac{1 - (a+b)}{1-b} \,. \end{align*}\] The uniform distribution does not exhibit the memoryless property, as the ratio above does not depend just on \(a\). (If \(b\) cancelled out top and bottom, then the distribution would exhibit memorylessness.)

Problem 3

(a) It doesn’t matter when she arrives: \[\begin{align*} P(x_0 \leq X \leq x_0 + 10) = \int_{x_0}^{x_0 + 10} \frac{1}{70} dx = \frac{x_0 + 10}{70} - \frac{x_0}{70} = \frac{1}{7} \,. \end{align*}\]

(b) We have that \(Y =\) Binomial\((n=5,p=1/7)\), so \[\begin{align*} P(Y\geq 1) = 1 - P(Y = 0) = 1 - {5 \choose 0}\frac{1}{7}^0\left(1 - \frac{1}{7}\right) = 1 - \left( \frac{6}{7}\right)^5. \end{align*}\]

Problem 4

We have that \[\begin{align*} P(X \leq 2u | X \geq u) = \frac{P(X \leq 2u \cap X \geq u)}{P(X \geq u)} = \frac{\int_u^{2u}dx}{\int_u^{1}dx} = \frac{2u - u}{1 - u} = \frac{u}{1-u} \,. \end{align*}\]

Problem 5

We have that \[\begin{align*} P(X_1 < 2X_2 \vert X_2 < 1/2) = \frac{P(X_2 > X_1/2 \cap X_2 < 1/2)}{P(X_2 < 1/2)} \,. \end{align*}\] We can approach this geometrically. The denominator is, by inspection, 1/2, so \[\begin{align*} P(X_1 < 2X_2 \vert X_2 < 1/2) = 2P(X_2 > X_1/2 \cap X_2 < 1/2) \,. \end{align*}\] The remaining expression evaluates as the area of the triangle with vertices (0,0), (1,1/2), and (0,1/2), which is (1/2)(1)(1/2) = 1/4. Thus \[\begin{align*} P(X_1 < 2X_2 \vert X_2 < 1/2) = 2 \cdot 1/4 = 1/2 \,. \end{align*}\]

Problem 6

(a) Since \(\theta\) is a lower bound, the sufficient statistic is \(X_{(1)}\), the minimum observed datum, by inspection.

(b) The cdf \(F_{(1)}(x)\) is \[\begin{align*} F_{(1)}(x) = 1 - [1 - F_X(x)]^n \,. \end{align*}\] The cdf \(F_X(x)\) is \[\begin{align*} F_X(x) = - \int_{\theta}^x \frac{1}{\theta} dy = - \frac{1}{\theta} \int_{\theta}^x dy = - \frac{1}{\theta} (x - \theta) = 1 - x/\theta \,. \end{align*}\] Thus \[\begin{align*} F_{(1)}(x) = 1 - [1 - (1 - x/\theta)]^n = 1 - \left(\frac{x}{\theta}\right)^n \,, \end{align*}\]

(c) If the null is true, we cannot observe a value of \(X_{(1)}\) that is smaller than \(\theta_o\). So the “trivial rejection region” is \(X_{(1)} < \theta_o\). This is “trivial” because we can write it down via inspection (and it does not depend on \(\alpha\)).

(d) We can only reject the null if \(X_{(1)} > \theta_o\), so we have to “all the \(\alpha\)” on that side of \(\theta_o\): \(1 - \alpha\).

(e) We have that \[\begin{align*} 1 - \left(\frac{x_{RR}}{\theta_o}\right)^n &= 1 - \alpha ~~~ \Rightarrow ~~~ \left(\frac{x_{RR}}{\theta_o}\right)^n = \alpha ~~~ \Rightarrow ~~~ x_{RR} = \theta_o \alpha^{1/n} \,. \end{align*}\]

Problem 7

(a) By inspection, \(\hat{\theta}_{MLE} = X_{(n)}\).

(b) The cdf for \(X_{(n)}\) is \([F_X(x)]^n = (x/\theta)^{2n}\), and the pdf is thus \[\begin{align*} f_{(1)}(x) = \frac{d}{dx} F_{(1)}(x) = \frac{2n}{\theta^{2n}} x^{2n-1} \,. \end{align*}\] Thus \[\begin{align*} E[X_{(n)}] &= \int_0^\theta x f_{(1)}(x) dx = \frac{2n}{\theta^{2n}} \int_0^\theta x^{2n} dx = \frac{2n}{\theta^{2n}} \left. \frac{x^{2n+1}}{2n+1}\right|_0^{\theta} = \frac{2n}{2n+1}\theta \,. \end{align*}\]

(c) Since \[\begin{align*} E[X_{(n)}] &= \frac{2n}{2n+1}\theta \,, \end{align*}\] we have that \[\begin{align*} E\left[\frac{2n+1}{2n}X_{(n)}\right] &= \theta \end{align*}\] and thus \[\begin{align*} \hat{\theta}_{MVUE} = \frac{2n+1}{2n}X_{(n)} \,. \end{align*}\]

Problem 8

(a) As \(\theta\) is a lower bound, \(X_{(1)}\) is a sufficient statistic.

(b) The MLE is the sufficient statistic in (a): \(\hat{\theta}_{MLE} = X_{(1)}\).

(c) The pdf for the minimum datum is \[\begin{align*} f_{(1)}(x) = n f_X(x) [1-F_X(x)]^{n-1} = n e^{-(x-\theta)} \left(e^{-(x-\theta)}\right)^{n-1} = n e^{-n(x-\theta)} \,. \end{align*}\]

(d) The expected value of \(X_{(1)}\) is \[\begin{align*} E[X_{(1)}] = \int_{\theta}^\infty x n e^{-n(x-\theta)} dx = n \int_{\theta}^\infty x n e^{-n(x-\theta)} dx \,. \end{align*}\] Let \(u = n(x-\theta)\). Then \(du = n dx\), and if \(x = \theta\), \(u = 0\), and if \(x = \infty\), \(u = \infty\). Thus \[\begin{align*} E[X_{(1)}] &= n \int_0^\infty \left(\frac{u}{n}+\theta\right) e^{-u} \frac{du}{n} = \int_0^\infty \frac{u}{n} e^{-u} du + \int_0^\infty \theta e^{-u} du = \frac{1}{n} \int_0^\infty u e^{-u} du + \theta \int_0^\infty e^{-u} du \\ &= \frac{1}{n} \Gamma(2) + \theta \Gamma(1) = \frac{1}{n} \cdot 1! + \theta \cdot 0! = \theta + \frac{1}{n} \,. \end{align*}\] Hence \(\hat{\theta}_{MVUE} = X_{(1)} - 1/n\).

Problem 9

(a) Since \(X_1\) and \(X_2\) are independent, \(P(X_1 > 1/2 \vert X_2 < 1/2) = P(X_1 > 1/2) = 1/2\).

(b) We have that \[\begin{align*} P\left(X_1 > \frac12 \vert X_1 < \frac34\right) = \frac{P(X_1 > 1/2 \cap X_1 < 3/4)}{P(X_1 < 3/4)} = \frac{P(1/2 < X_1 < 3/4)}{P(X_1 < 3/4)} = \frac{0.25}{0.75} = \frac13 \,. \end{align*}\]

(c) \(X_1 < 3X_2\) is equivalent to \(X_2 > \frac13 X_1\), i.e., \(X_2\) lies above the line with intercept 0 and slope 1/3. The area of this region is 1 minus the area of the triangle with vertices (0,0), (1,0), and (1,1/3) or \(1 - 1/6\) = 5/6.

(d) We have that \[\begin{align*} P\left(X_2 < X_1 \vert X_2 < \frac12\right) = \frac{P(X_2 < X_1 \cap X_2 < 1/2)}{P(X_1 < 1/2)} = 2P(X_2 < X_1 \cap X_2 < 1/2) \,. \end{align*}\] The probability is the area of the polygon with vertices (0,0), (1,0), (1,1/2), and (1/2,1/2), or 3/8. So \(P\left(X_2 < X_1 \vert X_2 < \frac12\right) = 6/8\) or 3/4.

Problem 10

(a) The expected value indicates that we are on the “yes” line of the confidence interval reference table, hence we want to solve \[\begin{align*} 1 - e^{-n(y_{\rm obs}-\theta)} - (1-\alpha) = 0 \end{align*}\] for \(\theta\): \[\begin{align*} e^{-n(y_{\rm obs}-\theta)} &= \alpha \\ \Rightarrow ~~~ -n(y_{\rm obs}-\theta) &= \log(\alpha) \\ \Rightarrow ~~~ \theta - y_{\rm obs} &= \frac{1}{n}\log(\alpha) \\ \Rightarrow ~~~ \hat{\theta}_L &= y_{\rm obs} + \frac{1}{n}\log(\alpha) \,. \end{align*}\]

(b) Note that although this is a two-tailed test, it is impossible to sample a statistic value less than \(\theta\), so we derive the rejection region boundary as if we are performing an upper-tail test. We are on the “yes” line of the hypothesis test reference table, hence we want to solve \[\begin{align*} 1 - e^{-n(y_{\rm RR}-\theta_o)} - (1-\alpha) = 0 \end{align*}\] for \(y_{\rm RR}\): \[\begin{align*} e^{-n(y_{\rm RR}-\theta_o)} &= \alpha ~~~ \Rightarrow ~~~ -n(y_{\rm RR}-\theta_o) &= \log(\alpha) ~~~ \Rightarrow ~~~ y_{\rm RR} - \theta_o = -\frac{1}{n}\log(\alpha) ~~~ \Rightarrow ~~~ y_{\rm RR} = \theta_o - \frac{1}{n}\log(\alpha) \,. \end{align*}\]

Chapter 6

Problem 1

(a) We have that \[\begin{align*} \int_0^1 \left( \int_0^1 dx_2 k (x_1 + x_2^2) \right) dx_1 &= 1 = k \left[ \int_0^1 x_1 \left( \int_0^1 dx_2 \right) dx_1 + \int_0^1 \left( \int_0^1 x_2^2 dx_2 \right) dx_1 \right] \\ &= k \left[ \int_0^1 x_1 dx_1 + \int_0^1 \frac13 dx_1 \right] = k \left[ \left. \frac{x_1^2}{2} \right|_0^1 + \left. \frac{x_1}{3} \right|_0^1 \right] \\ &= k \left( \frac{1}{2} + \frac{1}{3} \right) = k \frac{5}{6} \,. \end{align*}\] Thus \(k = 6/5\).

(b) We have that \[\begin{align*} f_{X_1 \vert X_2}(x_1 \vert x_2) = \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_2}(x_2)} = \frac{k (x_1 + x_2^2)}{f_{X_2}(x_2)} \,. \end{align*}\] So we need to compute the marginal density: \[\begin{align*} f_{X_2}(x_2) = k \int_0^1 dx_1 (x_1+x_2^2) = k \left[ \int_0^1 x_1 dx_1 + \int_0^1 x_2^2 dx_1 \right] = k \left[ \left. \frac{x_1^2}{2} \right|_0^1 + x_2^2 (\left. x_1\right|_0^1) \right] = k \left( \frac{1}{2} + x_2^2 \right) \,. \end{align*}\] Thus \[\begin{align*} f_{X_1 \vert X_2}(x_1 \vert x_2) = \frac{k (x_1 + x_2^2)}{k ( \frac{1}{2} + x_2^2 )} = \frac{2x_1 + 2x_2^2}{1 + 2x_2^2} \,. \end{align*}\] This may initially appear strange (in that if \(x_1 > 1/2\), \(f_{X_1 \vert X_2}(x_1 \vert x_2) > 1\)), but we simply need to remind ourselves that \(f_{X_1 \vert X_2}(x_1 \vert x_2)\) is a conditional probability density function, not a probability itself.

Problem 2

(a) Cov(\(X_1,X_2\)) = \(E[X_1X_2] - E[X_1]E[X_2]\) = \(1 \cdot 1 \cdot 0.1 - (1 \cdot 0.4 + 1 \cdot 0.1)^2 = 0.1 - 0.25 = -0.15\).

(b) \(\rho\) = Cov(\(X_1,X_2\))/(\(\sigma_1\sigma_2\)), where \[\begin{align*} \sigma_1 = \sqrt{E[X_1^2] - (E[X_1])^2} = \sqrt{0.5 - (0.5)^2} = \sqrt{0.25} = 0.5 = \sigma_2 \,. \end{align*}\] So \(\rho = -0.15/0.5/0.5 = -0.15/0.25 = -0.6\).

(c) \(E[X_1 \vert X_2 < 1]\) is equivalent to \(E[X_1 \vert X_2 = 0]\), i.e., the expected value for data drawn from the first row of the given table. \[\begin{align*} E[X_1 \vert X_2 = 0] &= \sum_{x_1=0}^1 x_1 p(x_1 \vert x_2=0) = \sum_{x_1=0}^1 x_1 \frac{p(x_1,x_2=0)}{p(p_2{x_2}=0)} \\ &= \frac{0 \cdot p(x_1=0,x_2=0) + 1 \cdot p(x_1=1,x_2=0)}{p(x_1=0,x_2=0)+p(x_1=1,x_2=0)} = \frac{0.4}{0.5} = 0.8 \,. \end{align*}\] This answer could be reasoned out by inspecting the table.

(d) We have that \[\begin{align*} V[X_2 \vert X_1=1] &= E[X_2^2 \vert X_1=1] - (E[X_2\vert X_1=1])^2 \\ &= \sum_{x_2=0}^1 x_2^2 p(x_2 \vert x_1=1) - \left[\sum_{x_2=0}^1 x_2 p(x_2 \vert x_1=1)\right]^2 \\ &= \sum_{x_2=0}^1 x_2^2 \frac{p(x_1=1,x_2)}{p(p_1{x_1=1})} - \left[\sum_{x_2=0}^1 x_2 \frac{P(x_1=1,x_2)}{p(p_1{x_1=1})}\right]^2 \\ &= 1 \cdot \frac{p(x_1=1,x_2=1)}{p(x_1=1,x_2=0)+p(x_1=1,x_2=1)} - &~~~~~\left[1 \cdot \frac{p(x_1=1,x_2=1)}{p(x_1=1,x_2=0)+p(x_1=1,x_2=1)}\right]^2 \\ &= \frac{0.1}{0.5} - \left(\frac{0.1}{0.5}\right)^2 = 0.2 - 0.04 = 0.16 \,. \end{align*}\] This answer could also be reasoned out by inspecting the table.

Problem 3

(a) We have that \(X \vert p\) \(\sim\) Bin(\(n,p\)) and that \(p \sim\) Uniform(0,0.1). The expected value of \(X \vert p\) is \(np\) and the expected value of \(p\) is \((0+0.1)/2 = 0.05\). Thus \[\begin{align*} E[X] = E[E[X \vert p]] = E[np] = nE[p] = 0.05n \,. \end{align*}\]

(b) We have that \[\begin{align*} V[X] &= E[V[X \vert p]] + V[E[X \vert p]] = E[np(1-p)] + V[np] = n(E[p] - E[p^2]) + n^2V[p] \\ &= n[E[p] - (V[p]+E[p]^2)] + n^2V[p] = n(n-1)V[p] + nE[p] - n(E[p])^2 \,. \end{align*}\]

Problem 4

(a) The area of integration lies between the \(x_1\) axis, the \(x_2\) axis, and the line \(x_2 = 1-x_1\), in the first quadrant. \[\begin{align*} 1 &= \int_0^1 \int_0^{1-x_1} k x_1^2 x_2 dx_2 dx_1 = k \int_0^1 x_1^2 \int_0^{1-x_1} x_2 dx_2 dx_1 = k \int_0^1 x_1^2 \left( \left. \frac{x_2^2}{2}\right|_0^{1-x_1}\right) dx_1 \\ &= \frac{k}{2} \int_0^1 x_1^2 (1-x_1)^2 dx_1 = \frac{k}{2} B(3,3) = \frac{k \Gamma(3) \Gamma(3)}{2 \Gamma(6)} = \frac{4k}{240} \,. \end{align*}\] So \(k = 60\).

(b) \(P(X_1 > 0.25 \vert X_2 = 0.5) = \int_{0.25}^{0.5} f(x_1 \vert x_2=0.5)dx_1\). \(f_{X_1 \vert X_2}(x_1 \vert x_2) = f_{X_1,X_2}(x_1,x_2)/f_{X_2}(x_2)\), and \[\begin{align*} f_{X_2}(x_2) = \int_0^{1-x_2} 60 x_1^2 x_2 dx_1 = 60 x_2 \left( \left.\frac{x_1^3}{3}\right|_0^{1-x_2}\right) = 20x_2(1-x_2)^3 \,. \end{align*}\] So: \[\begin{align*} f_{X_1 \vert X_2}(x_1 \vert x_2) = \frac{60x_1^2y_2}{20x_2(1-y_2)^3} = \frac{3x_1^2}{(1-x_2)^3} \,. \end{align*}\] Plugging in \(x_2 = 0.5\), we get \(24x_1^2\). Finally: \[\begin{align*} \int_{1/4}^{1/2} 24 x_1^2 dx_1 = \left.8 x_1^3\right|_{1/4}^{1/2} = 7/8 \,. \end{align*}\]

(c) The region over which \(f_{X_1 \vert X_2}(x_1 \vert x_2)\) is non-zero is not rectangular: \(X_1\) and \(X_2\) are dependent random variables.

Problem 5

(a) We sum over rows:

\(x_2\)	0	1	2
\(p_{x_2}(x_2)\)	0.4	0.4	0.2

(b) We have that

\(x_2\)	\((-\infty,0)\)	[0,1)	[1,2)
\(F_{X_2}(x_2)\)	0	0.4	0.8

(c) We have that \[\begin{align*} E[X] &= \sum_x x p_X(x) = 0 \cdot 0.4 + 1 \cdot 0.4 + 2 \cdot 0.2 = 0.8 \\ E[Y^2] &= \sum_x x^2 p_X(x) = 0^2 \cdot 0.4 + 1^2 \cdot 0.4 + 2^2 \cdot 0.2 = 1.2 \\ \end{align*}\] so \(V[X] = E[X^2] - (E[X])^2 = 0.56\) and \(\sigma_X = \sqrt{0.56} = 0.748\).

Problem 6

The region of integration lies between the \(x\) axis, the \(y\) axis, and the line \(x_2 = 1-x_1\), in the first quadrant.

(a) \({\rm Cov}(X_1,X_2) = E[X_1X_2] - E[X_1]E[X_2]\)…so we need to compute \(E[X_1X_2]\): \[\begin{align*} E[X_1X_2] &= \int_0^1 \int_0^{1-x_1} x_1 x_2 60 x_1^2 x_2 dx_2 dx_1 = 60 \int_0^1 x_1^3 \int_0^{1-x_1} x_2^2 dx_2 dx_1 \\ &= 60 \int_0^1 x_1^3 \frac{(1-x_1)^3}{3} dx_1 = 20 B(4,4) = 20 \frac{3! 3!}{7!} = 1/7 \,. \end{align*}\] Hence Cov(\(X_1,X_2\)) = 1/7 - 1/2(1/3) = \(-1/42\).

(b) We have that \[\begin{align*} {\rm Corr}(X_1,X_2) = \frac{{\rm Cov}(X_1,X_2)}{\sigma_{X_1}\sigma_{X_2}} = \frac{-1/42}{\sqrt{2/7-1/4} \sqrt{1/7-1/9}} = \cdots = -1/\sqrt{2} \,. \end{align*}\]

Problem 7

Let \(N\) be the number of laid egges: \(N \sim\) Poi(\(\lambda\)), and \(E[N] = V[N] = \lambda\). Let \(X\) be the number of hatched eggs: \(X \vert N \sim\) Bin(\(N,p\)), and \(E[X \vert N] = Np\) and \(V[X \vert N] = Np(1-p)\).

(a) \(E[X] = E[E[X\vert N]] = E[Np] = pE[N] = \lambda p\).

(b) We have that \[\begin{align*} V[X] &= V[E[X\vert N]] + E[V[X \vert N]] = V[Np] + E[Np(1-p)] \\ &= p^2V[N] + p(1-p)E[N] = \lambda p^2 + \lambda p - \lambda p^2 = \lambda p \,. \end{align*}\]

Problem 8

(a) \(X_1\), \(X_2\) are independent \(\Rightarrow f_{X_1,X_2}(x_1, x_2) = f_{X_1}(x_1) f_{X_2}(x_2) = \left[ k_1 x_1 e^{-x_1/2}\right]\left[k_2x_2(1-x_2) \right]\). \(X_1 \sim \text{Gamma}(2,2) \left[ \text{or } \chi^2_4\right]\), and \(X_2 \sim \text{Beta}(2,2)\), thus \[\begin{align*} k = k_1 k_2 = \frac{1}{\beta_1^{\alpha_1}\Gamma(\alpha_1)}\underbrace{\frac{\Gamma(\alpha_2 + \beta_2)}{\Gamma(\alpha_2)\Gamma(\beta_2)}}_{1/B(\alpha_2,\beta_2)} = \frac{1}{2^21!}\frac{3!}{1!1!} = \frac{3}{2} \,. \end{align*}\]

(b) We have that \[\begin{align*} V[X_1 - X_2] &= V[X_1] + V[X_2] = \underbrace{\alpha_1 \beta_1^2}_{\text{Gamma}} + \underbrace{\alpha\beta/\left[(\alpha_2 + \beta_2)^2(\alpha_2 + \beta_2 + 1)\right]}_{\text{Beta}}\\ &= 8 + 4/(16 \cdot 5) = 8 + \frac{1}{20} = \frac{161}{20} \,. \end{align*}\]

Problem 9

(a) \(X_1\) and \(X_2\) are uniformly distributed. In a plane described by \(X_1\), \(X_2\), the region where \(f_{X_1,X_2}(x_1, x_2)>0\) corresponds to a rectangular trapezoid, with long base going from the point with coordinate \((0,0)\) to the point with coordinate \((2,0)\), and short base going from the point with coordinate \((0,1)\) to the point with coordinate \((1,1)\). Therefore the segment connecting the two points \((1,1)-(2,0)\) corresponds to the line \(X_2 = -X_1 + 2\), or \(X_1 = 2 - X_2\). Thus \[\begin{align*} f_{X_1,X_2}(x_1, x_2) = \frac{1}{\text{area of the region for which }f_{X_1,X_2}(x_1, x_2)>0 }= \frac{1}{3/2} = \frac{2}{3} \,. \end{align*}\]

(b) \(f_{X_2}(x_2) = \int_{x_1 = 0}^{x_1 = 2-x_2} k dx_1 = k \left[ x_1|_0^{2-x_2}\right] = k(2-x_2)\),i if \(0\leq x_2 \leq 1\). Therefore, \(f_{X_2}(x_2) = k(2-x_2)\) for \(x_2 \in [0,1]\).

(c) \(f_{X_1 | X_2}(x_1 | x_2) = f_{X_1,X_2}(x_1, x_2)/f_{X_2}(x_2) = 1/(2-x_2)\), for \(x_1 \in [0,2-x_2]\) and \(x_2 \in [0,1]\).

(d) We have that \[\begin{align*} E[X_1] =& \int_0^1 \int_{x_1 = 0}^{x_1 = 2-x_2} x_1 k dx_1 dx_2 = k \int_0^1 \left[\frac{x_1^2}{2}\bigg|_{0}^{2-x_2} \right] dx_2 \\ =& k \int_0^1 \frac{1}{2} (2-x_2)^2 dx_2 = \frac{k}{2} \int_0^1 (4 - 4x_2 + x_2^2) dx_2\\ =& \frac{k}{2} \left[ 4x_2\bigg|_0^1 - 2x_2^2\bigg|_0^1 + \frac{x_2^3}{3}\bigg|_0^1\right] = \frac{k}{2}\left(4 -2 +\frac{1}{3}\right) = \frac{7}{6}k \,. \end{align*}\]

(e) They are dependent since the region over which \(f_{X_1,X_2}(x_1, x_2) >0\) is non-rectangular.

Problem 10

(a) \(p \sim {\rm Beta}(2,2)\), and \(X|p \sim {\rm Bin}(5,p)\), thus \[\begin{align*} E\left[ E\left[ X|p\right] \right] = E[5p] = 5E[p] = 5\frac{\alpha}{\alpha + \beta} = 2.5 \,. \end{align*}\]

(b) We have that \[\begin{align*} V[X] &= V\left[ E\left[ X|p\right] \right] + E\left[ V\left[ X|p\right] \right] = V[5p] + E[5p(1-p)] = 25V[p] + 5\left[E[p] - E[p^2] \right]\\ &= 25V[p] + 5E[p] -5\left[ V[p] + (E[p])^2 \right] = 20V[p] + 5E[p] -5(E[p])^2\\ &= 20 \frac{\alpha \beta}{(\alpha + \beta)^2(\alpha + \beta + 1)} + 5\frac{\alpha}{\alpha + \beta} - 5\left(\frac{\alpha}{\alpha + \beta}\right)^2\\ &= 20 \frac{4}{16 \cdot 5} + 2.5 -1.25 = 1 + 2.5 - 1.25 = 2.25 \,. \end{align*}\]

Problem 11

The region where \(f_{X_1,X_2}(x_1, x_2)>0\) can be represented by a triangle with vertices (0,0), (1,0), and (1,1). Thus: \[\begin{align*} E[X_1 X_2] &= \int_0^1 \left( \int_0^{x_1} x_1 x_2 (3 x_1) dx_2 \right) dx_1 = \int_0^1 3x_1^2 \left( \int_0^{x_1} x_2 dx_2\right) dx_1 \\ &= \int_0^1 3x_1^2 \frac{x_1^2}{2} dx_1 = \frac{3}{2} \int_0^1x_1^4 dx_1 = \frac{3}{10} x_1^5\bigg|_0^1 = \frac{3}{10} \,. \end{align*}\] Therefore \[\begin{align*} \rho = \frac{{\rm Cov}(X_1,X_2)}{\sqrt{V[X_1] V[X_2]}} = \frac{E[X_1X_2] - E[X_1] E[X_2]}{\sqrt{V[X_1] V[X_2]}} = \frac{\frac{3}{10} - \frac{9}{32}}{\sqrt{\frac{3}{80}\frac{19}{320}}} = 0.397 \,. \end{align*}\]

Problem 12

The region of integration lies in the first quadrant, below the line \(x_2 = x_1\).

(a) We have that \[\begin{align*} \int_0^\infty \left[ \int_0^{x_1} k e^{-x_1} dx_2 \right] dx_1 = k \int_0^\infty e^{-x_1} \int_0^{x_1} dx_2 dx_1 = k \int_0^\infty x_1 e^{-x_1} dx_1 = k \Gamma(2) = 1 \Rightarrow k = 1 \,. \end{align*}\]

(b) We have that \[\begin{align*} P(X_2 < 1) &= \int_0^1 \left[ \int_{x_2}^\infty e^{-x_1} dx_1 \right] dx_2 = \int_0^1 \left(-\left.e^{-x_1}\right|_{x_2}^\infty \right) dx_2 \\ &= \int_0^1 e^{-x_2} dx_2 = -\left.e^{-x_2}\right|_0^1 = -(e^{-1}-1) = 1-e^{-1} \,. \end{align*}\]

(c) The marginal distribution is \[\begin{align*} f_{X_2}(x_2) = \int_{x_2}^\infty e^{-x_1} dx_1 = -\left.e^{-x_1}\right|_{x_2}^\infty = -(0-e^{-x_2}) = e^{-x_2} \,, \end{align*}\] or, in full, \[\begin{align*} f_{X_2}(x_2) = \left\{ \begin{array}{cl} e^{-x_2} & x_2 \geq 0 \\ 0 & \mbox{otherwise} \end{array} \right. \,. \end{align*}\]

(d) The conditional pdf is \[\begin{align*} f_{X_1 \vert X_2}(x_1 \vert x_2) = \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_2}(x_2)} = \frac{e^{-x_1}}{e^{-x_2}} = e^{x_2-x_1} \,, \end{align*}\] or, in full, \[\begin{align*} f_{X_1 \vert X_2}(x_1 \vert x_2) = \left\{ \begin{array}{cl} e^{x_2-x_1} & x_1,x_2 \geq 0 \\ 0 & \mbox{otherwise} \end{array} \right. \,. \end{align*}\]

(e) We have that \[\begin{align*} P(X_1 > 2 \vert X_1 = 1) &= \int_2^\infty f_{X_1 \vert X_2=1}(x_1 \vert x_2=1) dx_1 = \int_2^\infty e^{1-x_1} dx_1 \\ &= e^1 \left(-\left.e^{-x_1}\right|_2^\infty\right) = e^1(-(0-e^{-2})) = e^{-1} \,. \end{align*}\]

Problem 13

We are given that \(X|\lambda \sim\) Poisson\((\lambda)\), and \(\lambda \sim\) NegBinom\((4,1/2)\). Thus \(E[X|\lambda] = \lambda\) and \(E[\lambda] = \frac{r}{p} = 8\), and \(V[X|\lambda] = \lambda\) and \(V[\lambda] = \frac{r(1-p)}{p^2} = \frac{2}{1/4} = 8\).

(a) \(E[X] = E\left[ E\left[ X|\lambda\right] \right] = E[\lambda] = 8\).

(b) \(V[X] = E\left[ V\left[ X|\lambda\right] \right] + V\left[ E\left[ X|\lambda\right] \right] = E[\lambda] + V[\lambda] = 8 + 8 = 16\).

Problem 14

The region where \(f_{X_1,X_2}(x_1, x_2)\) is positive can be described as the union of two rectangular triangle. The first one with vertexes \((-2,0) - (-2, 2) - (0,0)\). The second one with vertexes \((2,0) - (2, 2) - (0,0)\).

(a) The region is not rectangular: \(X_1, X_2\) are not independent.

(b) Uniformity \(\Rightarrow k = \frac{1}{\text{geometric area}} = \frac{1}{4}\).

(c) We have that \[\begin{align*} E[X_2] &= \int_0^2 \left[\int_{-2}^{-x_2} \frac{x_2}{4} dx_1 + \int_{x_2}^{2} \frac{x_2}{4} dx_1 \right] dx_2 = 2\int_0^2 \left[\int_{x_2}^{2} \frac{x_2}{4} dx_1 \right] dx_2\\ &=\frac{1}{2}\int_0^2 x_2 \left[\int_{x_2}^{2} dx_1 \right] dx_2 =\frac{1}{2}\int_0^2 x_2(2-x_2) dx_2 = \frac{1}{2} \left[ x_2^2\bigg|_0^2 - \frac{x_2^3}{3}\bigg|_0^2\right] \\ &= \frac{1}{2} \left[ 4 - \frac{8}{3}\right]= \frac{2}{3} \,. \end{align*}\]

Problem 15

(a) We have that \[\begin{align*} \int_0^1 \int_0^1 f_{X_1,X_2}(x_1, x_2) dx_1 dx_2 = 1 = k \left[ \int_0^1 x_1 dx_1 \int_0^1 x_2 dx_2 \right] = k \left[ \frac{x_1^2}{2}\bigg|_0^1 \frac{x_2^2}{2}\bigg|_0^1 \right] = \frac{k}{4} \,. \end{align*}\] Therefore \(k = 4\).

(b) \(X_1, X_2\) are independent, so \[\begin{align*} P(X_2 > 1/2|X_1 = 1/2) &= P(X_2 > 1/2) = \int_{1/2}^1 \left[ \int_0^1 4 x_1 x_2 dx_1\right] dx_2\\ &= \int_{1/2}^1 x_2 \left[ \int_0^1 4 x_1 dx_1\right] dx_2 = \int_{1/2}^1 x_2 \frac{1}{2} dx_2 = 2 \left( \frac{x_2^2}{2}\bigg|_{1/2}^1\right) \\ &= 1^2 - (1/2)^2 = \frac{3}{4} \,. \end{align*}\]

Problem 16

We have that \[\begin{align*} V[U] = a^T\Sigma a = \begin{bmatrix} 2 & -1 \end{bmatrix} \begin{bmatrix} 3 & 2\\ 2 & 3 \end{bmatrix} \begin{bmatrix} 2 \\ -1 \end{bmatrix} = \begin{bmatrix} 2 & -1 \end{bmatrix} \begin{bmatrix} 4 \\ 1 \end{bmatrix} = 8 - 1 =7 \,. \end{align*}\] Alternatively, \[\begin{align*} V[U] = a_1^2V[X_1] + a_2^2 V[X_2] + 2a_1a_2 \text{Cov}(X_1, X_2) = 4 \cdot 3 + 1 \cdot 3 + 2 (2)(-1)(2) = 15-8 = 7 \,. \end{align*}\]

Problem 17

Before we begin, we can write down that \(X \sim \text{Bernoulli}(1/2)\), so \(E[X] = \frac{1}{2}\) and \(V[X] = \frac{1}{2}\left( 1 - \frac{1}{2}\right) = \frac{1}{4}\), and that \(U \vert X \sim \text{Uniform}(X,2)\), so \(E[U \vert X] = \frac{X+2}{2}\) and \(V[U \vert X] = \frac{1}{12}\left( 2 - X^2 \right)\).

We have that \[\begin{align*} V[U] &= E\left[V \left[ U|X \right] \right] + V\left[E \left[ U|X \right] \right] \\ &= E\left[\frac{1}{12}\left( 2 - X^2 \right) \right] + V\left[\frac{X+2}{2} \right] \,, \end{align*}\] where \[\begin{align*} V\left[\frac{X+2}{2}\right] = V\left[\frac{X}{2}\right] = \frac{1}{4}V[X] = \frac{1}{16} \end{align*}\] and \[\begin{align*} E\left[\frac{1}{12}\left( 2 - X^2 \right) \right] &= \frac{1}{12} E[4 - 4X + X^2] = \frac{1}{3} - \frac{1}{6} + \frac{1}{12}\left[V[X] + (E[X])^2 \right]\\ &= \frac{1}{6} + \frac{1}{12} \left[ \frac{1}{4} \frac{1}{4}\right] = \frac{4}{24} + \frac{1}{24} = \frac{5}{24} \,. \end{align*}\] Thus \(V[U] = 5/24 + 1/16 = 13/48\).

Problem 18

(a) Is the region rectangular? Yes. And \(f_{X_1,X_2}(x_1,x_2) = f_{X_1}(x_1) f_{X_2}(x_2) = f_{X_1}(x_1)\), with \(f_{X_2}(x_2) = 1\), so \(X_1, X_2\) are independent random variables.

(b) \(f_{X_1,X_2}(x_1, x_2) =f_{X_1}(x_1) f_{X_2}(x_2) = 12 x_1^2(1-x_1) \times 1\), therefore \(f_{X_2}(x_2) = 1\) for \(x_2 \in [0,1]\), or \(X_2 \sim \text{Uniform}(0,1)\). Other ways to derive this result include \[\begin{align*} f_{X_2}(x_2) &= \underbrace{\int_0^1 12 x_1^2(1-x_1) dx_1}_{\text{Beta}(3,2)} = 1 \\ &= \int_0^1 12 x_1^2(1-x_1) dx_1 = 12 \left[ \frac{x_1^3}{3}\bigg|_0^1 - \frac{x_1^4}{4}\bigg|_0^1 \right] = \frac{12}{12} = 1 \,. \end{align*}\]

(c) \(X_1 \sim \text{Beta}(3,2)\), hence \(E[X_1] = \alpha/(\alpha + \beta) = \frac{3}{5}\). Another way to derive this result is \[\begin{align*} f_{X_1}(x_1) &= \int_0^1 12 x_1^2(1-x_1) dx_1 = 12 x_1^2(1-x_1) \\ E[X_1] &= \int_0^1 x_1 f_{X_1}(x_1) dx_1 = \int_0^1 12 x_1^3(1-x_1) dx_1 = 12 \left[ \frac{x_1^4}{4}\bigg|_0^1 - \frac{x_1^5}{5}\bigg|_0^1 \right] = \frac{12}{20} = \frac{3}{5} \,. \end{align*}\]

Problem 19

(a) The area where \(f_{X_1,X_2}(x_1,x_2)>0\) is a triangle with vertices at (0,0), (1,1), and (2,0). For a bivariate uniform, \(c = 1/\)(the area of the region), so \(c = 1\). Another way to derive this is via brute force: \[\begin{align*} \int_0^1 \left[\int_{x_2}^{2 - x_2} c dx_1\right] dx_2 &= c \int_0^1 (2- x_2) - x_2 dx_2\\ &= c \int_0^1 2(1- x_2) dx_2 = 2c \left[ x_2\bigg|_0^1 - \frac{x_2^2}{2}\bigg|_0^1 \right] = c = 1 \,. \end{align*}\]

(b) \(f_{X_2}(x_2) = \int_{x_2}^{2 - x_2} dx_1 = 2(1- x_2)\) for \(x_2 \in [0,1]\).

(c) \(f_{X_1|X_2}(x_1|x_2) =\frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_2}(x_2)} = 1/[2(1-x_2)]\) for \(x_1 \in [0,2]\), and \(x_2 \in [0,x_1]\) and \(x_2 \in [0,2-x_1]\).

Problem 20

(a) We can recognize immediately that \(X_1,X_2\) are independent, thus \(f_{X_1}(x_1) = 2x_1\) for \(x_1 \in [0,1]\). We can also derive this result via brute force: \[\begin{align*} f_{X_1}(x_1) = \int_0^1 4 x_1 x_2 dx_2 = 4x_1 \frac{x_2^2}{2}\bigg|_0^1 = 2x_1 \,. \end{align*}\]

(b) We have that \(f_{X_2|X_1}(x_2|x_1) = \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_1}(x_1)} = \frac{4 x_1 x_2}{2x_1} = 2x_2\) for \(x_1 \in [0,1]\) and \(x_2 \in [0,1]\).

(c) We have that \[\begin{align*} P(X_2 < 1/2 |X_1 = x_1) = \int_0^{1/2} f_{X_2|X_1}(x_2|x_1) dx_2 = \int_0^{1/2}2x_2dx_2 = x_2^2\bigg|_0^{1/2} = \frac{1}{4} \,. \end{align*}\]

Problem 21

We begin by noting that \(V[X_1] = E[X_1^2] - E[X_1]^2\), and that \[\begin{align*} E[X_1] &= \int_0^1 \int_0^{1-x_1} 2x_1 dx_2 dx_1 = 2 \int_0^1 x_1(1-x_1) dx_1 = 2B(2,2) = 2 \frac{\Gamma(2)\Gamma(2)}{\Gamma(4)} = \frac{1!1!}{3!} = \frac{1}{3} \,. \\ E[X_1^2] &= \int_0^1 \int_0^{1-x_1} 2x_1 x_1 dx_2 dx_1 = 2 \int_0^1 x_1^2(1-x_1) dx_1 = 2B(3,2) = 2 \frac{2!1!}{4!} = \frac{1}{6} \,. \end{align*}\] Thus \(V[X_1] = 1/6 - \left( 1/3 \right)^2 = 1/18\).

Problem 22

We have that \(V[X_1 - X_2] = V[X_1] + V[X_2] - 2\)Cov\((X_1,X_2)\). So we need to compute every part of the formula above: \[\begin{align*} E[X_1] &= \sum \sum x_1 p_{X_1,X_2}(x_1,x_2) = 1 \cdot \frac{2}{9} + 1 \cdot \frac{2}{9} + 2 \cdot \frac{1}{9} = \frac{6}{9}\\ V[X_1] &= E[X_1^2] - E[X_1]^2 = \frac{8}{9} - \frac{36}{81} = \frac{4}{9}\\ E[X_2] &= \sum \sum x_2 p_{X_1,X_2}(x_1,x_2) = 1 \cdot \frac{3}{9} + 1 \cdot \frac{2}{9} + 1 \cdot \frac{1}{9} = \frac{6}{9}\\ V[X_1] &= E[X_1^2] - E[X_1]^2 = \frac{6}{9} - \frac{36}{81} = \frac{2}{9}\\ E[X_1X_2] &= \sum \sum x_1 x_2 p_{X_1,X_2}(x_1,x_2) = 1 \cdot 1 \cdot \frac{2}{9} + 2 \cdot 1 \cdot \frac{1}{9} = \frac{4}{9}\\ \mbox{Cov}(X_1, X_2) &= \frac{4}{9} - \left( \frac{6}{9}\right)^2 = 0 \,. \end{align*}\] Thus \(V[X_1 - X_2] = 6/9 = 2/3\).

Problem 23

We recognize that \(X|\beta \sim \text{Gamma}(\alpha, \beta)\) and that \(\beta \sim \text{Exp}(\gamma)\), with \(E[\beta] = \gamma, V[\beta] = \gamma^2\), and \(E[X|\beta] = \alpha \beta\), \(V[X|\beta] = \alpha \beta^2\).

(a) \(E[X] = E\left[ E\left[ X|p\right] \right] = E[\alpha \beta] = \alpha E[\beta] = \alpha \gamma\).

(b) We have that \[\begin{align*} V[X] &= V\left[ E\left[ X|p\right] \right] + E\left[ V\left[ X|p\right] \right] = E[\alpha \beta^2] + V[\alpha \beta]\\ &= \alpha E[\beta^2] + \alpha^2V[\beta]= \alpha \left[ V[\beta] + E[\beta]^2\right] + \alpha^2 \gamma^2\\ &= \alpha \left[ \gamma^2+\gamma^2\right] + \alpha^2 \gamma^2 = (2\alpha + \alpha^2) \gamma^2 \,. \end{align*}\]

Problem 24

The region of integration is a triangle with vertices at (0,0), (0,1), and (1,1).

(a) The area of the triangle is 1/2, so \(f_{X_1,X_2}(x_1,x_2) = 2\).

(b) Geometry will not directly help us here; we still have to integrate. The expected value \(E[X_1]\) is \[\begin{align*} E[X_1] &= \int_0^1 \left[ \int_0^{x_1} 2 x_1 dx_2 \right] dx_1 = 2 \int_0^1 x_1 \left[ \int_0^{x_1} dx_2 \right] dx_1 \\ &= 2 \int_0^1 x_1 x_1 dx_1 = 2 \left.\frac{x_1^3}{3}\right|_0^1 = \frac23 \,. \end{align*}\]

(c) We have that Cov[\(X_1,X_2\)] = \(E[X_1X_2] - E[X_1]E[X_2]\). So we need to compute \(E[X_1X_2]\): \[\begin{align*} E[X_1X_2] &= \int_0^1 \left[ \int_0^{x_1} 2 x_1 x_2 dx_2 \right] dx_1 = 2 \int_0^1 x_1 \left[ \int_0^{x_1} x_2 dx_2 \right] dx_1 \\ &= 2 \int_0^1 x_1 \left[ \left.\frac{x_2^2}{2}\right|_0^{x_1} \right] dx_1 = \int_0^1 x_1 x_1^2 dx_1 = \left.\frac{x_1^4}{4}\right|_0^1 = \frac14 \,. \end{align*}\] So the covariance is \(1/4 - (2/3)(1/3) = 9/36 - 8/36 = 1/36\).

Problem 25

(a) The long way to do this involves integration. The short way is to see by inspection that \(X_1\) and \(X_2\) are independent (the region of integration is “rectangular,” and \(x_2\) doesn’t directly appear in \(f_{X_1,X_2}(x_1,x_2)\), so \(f_{X_1,X_2}(x_1,x_2)\) can be trivially split into two functions \(g(x_1)h(x_2)\)), and to see that \[\begin{align*} X_1 \sim {\rm Exp}(\beta = 2) ~{\rm and}~ X_2 \sim {\rm Unif}(0,2) \,. \end{align*}\] We know \(f_{X_1,X_2}(x_1,x_2) = f_{X_1}(x_1)f_{X_2}(x_2) = (1/\beta){\rm exp}(-x_1/2) \cdot (1/2)\), so \(k = 1/(2\beta) = 1/4\).

(b) The key here is to realize that since \(X_1\) and \(X_2\) are independent, \(E[X_1]\) is just the expected value of \(f_{X_1}(x_1)\), i.e., it is the expected value of an exponential distribution with mean 2. So: \(E[X_1] = \beta = 2\).

Problem 26

The region of integration is a triangle with vertices (0,0), (2,0), and (1,1). The area of the region of integration is 1, so \(f_{X_1,X_2}(x_1,x_2) = 1\).

(a) We have that \[\begin{align*} f_{X_2}(x_2) &= \int_{x_2}^{2-x_2} dx_1 = 2(1-x_2) ~~~ x_2 \in [0,1] \,. \end{align*}\]

(b) We have that \[\begin{align*} f_{X_1 \vert X_2}(x_1 \vert x_2) &= \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_2}(x_2)} = \frac{1}{2(1-x_2)} ~~~ x_1 \in [x_2,2-x_2] ~~~ x_2 \in [0,1] \,. \end{align*}\]

(c) The new region of integration is a triangle with vertices at (0,0), (1,0), and (1/2,1/2). Since we are dealing with a bivariate uniform, we know that the probability of sampling data from this triangle is the ratio of its area to the area of the total region of integration for the pdf. Thus \(P(X_1 + X_2 \leq 1) = (1/2 \cdot 1 \cdot 1/2)/1 = 1/4\).

Problem 27

We are given that \(X \vert \mu \sim N(\mu,1)\) and that \(\mu \sim N(0,1)\). We thus know that \(E[X \vert \mu] = \mu\) and \(V[X \vert \mu] = 1\), while \(E[\mu] = 0\) and \(V[\mu] = 1\). Thus \[\begin{align*} V[X] &= V\left[E[X \vert \mu]\right] + E\left[V[X \vert \mu]\right] = V[\mu] + E[1] = 1 + 1 = 2 \,. \end{align*}\]

Problem 28

(a) The conditional expected value is \[\begin{align*} E[X_1 \vert X_2=1] &= (x_1 = 0) \cdot p(x_1=0 \vert x_2=1) + (x_1 = 1) \cdot p(x_1=1 \vert x_2=1) \\ &= 1 \cdot \frac{p(1,1)}{p_2(1)} = 1 \cdot \frac{0.1}{0.3+0.1} = 0.25 \,. \end{align*}\]

(b) The conditional variance is \[\begin{align*} V[X_1 \vert X_2=1] &= E[X_1^2 \vert X_2=1] - (E[X_1 \vert X_2=1])^2 \,. \end{align*}\] We know the second term. The first term can be derived in a manner similar to above: \[\begin{align*} E[X_1^2 \vert X_2=1] &= \ldots = 1^2 \cdot \frac{p(1,1)}{p_2(1)} = 1 \cdot \frac{0.1}{0.3+0.1} = 0.25 \,. \end{align*}\] Thus the conditional variance is \(V[X_1 \vert X_2=1] = 0.25 - 0.25^2 = 1/4 - 1/16 = 3/16\).

Problem 29

(a) Because the boundary of the domain is not rectangular (see \(x_1 + x_2 \leq 2\)), \(X_1\) and \(X_2\) are not independent.

(b) We need the distribution to integrate to 1: \[\begin{align*} 1 &= \int_{x_1} \int_{x_2} c x_1^2 dx_2 dx_1 \,. \end{align*}\] The domain is a triangle with vertices (0,0), (2,0), and (0,2), and thus there is no real advantage gained by utilizing either order of integration, so we’ll keep the integral over \(x_2\) as our “inner” integral: \[\begin{align*} \int_0^2 \int_0^{2-x_1} c x_1^2 dx_2 dx_1 &= c \int_0^2 x_1^2 \left( \int_0^{2-x_1} dx_2 \right) dx_1 \\ &= c \int_0^2 x_1^2 (2 - x_1) dx_1 \\ &= c \left( 2 \int_0^2 x_1^2 dx_1 - \int_0^2 x_1^3 dx_1 \right) \\ &= c \left( 2 \left. \frac{x_1^3}{3}\right|_0^2 - \left. \frac{x_1^4}{4}\right|_0^2 \right) \\ &= c \left( \frac{16}{3} - 4 \right) = c\frac{4}{3} \,. \end{align*}\] Hence \(c = 3/4\).

(c) We actually already derived the marginal distribution above: \[\begin{align*} f_{X_1}(x_1) &= c x_1^2 \int_0^{2-x_1} dx_2 \\ &= c x_1^2 (2 - x_1) ~~~ x_1 \in [0,2] \,. \end{align*}\]

(d) The conditional distribution is \[\begin{align*} f_{X_2 \vert X_1}(x_2 \vert x_1) = \frac{f_{X_1,X_2}(x_1,x_2)}{f_{X_1}(x_1)} = \frac{cx_1^2}{cx_1^2(2-x_1)} = \frac{1}{2-x_1} ~~~ x_2 \in [0,2-x_1] \,. \end{align*}\]

(e) Given the domain and the flat pdf, we know that \(f_{X_2 \vert X_1}(x_2 \vert x_1)\) is a “uniform” distribution.

Problem 30

(a) We need to determine \(E[X_1]\), \(E[X_2]\), and \(E[X_1X_2]\): \[\begin{align*} E[X_1] &= 1 \cdot (0.1 + 0.5) = 0.6 \\ E[X_2] &= 1 \cdot (0.1 + 0.5) = 0.6 \\ E[X_1X_2] &= 1 \cdot 1 \cdot 0.5 = 0.5 \,. \end{align*}\] Thus Cov(\(X_1,X_2\)) = \(E[X_1X_2] - E[X_1]E[X_2] = 0.5 - 0.6^2 = 0.14\).

(b) Now we need \(E[X_1^2]\) and \(E[X_2^2]\): \[\begin{align*} E[X_1^2] &= 1^2 \cdot (0.1 + 0.5) = 0.6 \\ E[X_2^2] &= 1^2 \cdot (0.1 + 0.5) = 0.6 \,. \end{align*}\] Thus \[\begin{align*} V[X_1] &= E[X_1^2] - (E[X_1])^2 = 0.6 - 0.6^2 = 0.24 \\ V[X_2] &= E[X_2^2] - (E[X_2])^2 = 0.6 - 0.6^2 = 0.24 \,, \end{align*}\] and \(\sigma_1\sigma_2 = \sqrt{V[X_1]V[X_2]} = 0.24\), and \(\rho_{X_1,X_2} = 0.14/0.24 = 7/12\).

(c) We have that \[\begin{align*} V[Y] &= a_1^2 V[X_1] + a_2^2 V[X_2] + 2 a_1 a_2 \mbox{Cov}(X_1,X_2) \\ &= 4 \cdot 0.24 + 1 \cdot 0.24 - 2 \cdot 2 \cdot 1 \cdot 0.14 \\ &= 1.2 - 4 \cdot 0.14 = 0.64 \,. \end{align*}\]