Quadratic Forms and Chi-Squared Distributions

(20 points) Let \(Z = (Z_{1}, Z_{2}, Z_{3})^{\top } \sim \mathcal {N}(0, I_{3})\) and define the random vector \[ X = \begin {pmatrix} X_{1} \\ X_{2} \end {pmatrix} = \begin {pmatrix} Z_{1} + Z_{2} \\ Z_{2} + Z_{3} \end {pmatrix}. \]

1. Compute the mean vector \(\mathbb {E}[X]\) and the covariance matrix \(\Sigma = \operatorname {Cov}[X]\).
2. Does \(X\) have a probability density function on \(\mathbb {R}^{2}\)? Briefly justify your answer.
3. Does the joint moment generating function satisfy \[ M_{X}(t_{1}, t_{2}) = M_{X}(t_{1}, 0)M_{X}(0, t_{2})? \] Explain your reasoning.
4. Let \(Q = \|X\|^{2}\). Identify a symmetric matrix \(A\) such that \(Q = Z^{\top }AZ\) and compute \(\mathbb {E}[Q]\).

Let \((X_{1}, X_{2})\) be independent normal random variables with \(\mathbb {E}X_{1} = \mathbb {E}X_{2} = 0\) and \(\mathbb {E}X_{1}^{2} =\) \(1, \mathbb {E}X_{2}^{2} = 4\). Define \(Y_{1} = X_{1} + X_{2}\) and \(Y_{2} = X_{1} + 2X_{2}\) and let \(\mathbf {Y} = (Y_{1}, Y_{2})^{\top }\).

1. Compute the covariance matrix of \(\mathbf {Y}\).
2. Find a matrix \(\mathbf {D}\) such that \(\mathbf {Y}^{\top }\mathbf {D}\mathbf {Y}\) has a chi-squared distribution.

Let \(X_1,\ldots ,X_n\) be independent and identically distributed random variables with (unknown) density function \(F\). Let \[ F_0(t)=(1-e^{-t})1\{t>0\}. \] We wish to test \(F(t)=F_0(t)\) for all \(t\). Explain the application of the \(\chi ^2\) goodness-of-fit test.

Let \(g:\mathbb {R}\to \mathbb {R}\) be bounded and continuous and fix \(\lambda >0\). Use properties of the Poisson distribution to show that \[ \sum _{k=0}^{\infty }g(k/n)\frac {(n\lambda )^k}{k!}e^{-n\lambda } \to g(\lambda ) \qquad \text {as } n\to \infty . \]

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with a Beta\((\theta ,\theta )\) distribution, i.e. their density is \[ f(x;\theta )=\frac {\Gamma (2\theta )}{\Gamma (\theta )^2} x^{\theta -1}(1-x)^{\theta -1}, \qquad 0<x<1. \] Show that a best critical region for testing \(H_0:\theta =1\) versus \(H_A:\theta =2\) is \[ \left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

Let \(X_1,X_2,X_3\) be iid and distributed according to \(f(x)=2x\), \(0<x<1\). Let \(X_{(1)}<X_{(2)}<X_{(3)}\) be the order statistics. Compute the probability that the smallest of the \(X_i\) exceeds the median of the distribution.

Suppose that \(X_1,\ldots ,X_n\) form a random sample from a distribution for which the pdf \(f(x|\theta _1,\theta _2)\) is as follows: \[ f(x|\theta _1,\theta _2)= \begin {cases} \dfrac {1}{\theta _2-\theta _1}, & \theta _1<x<\theta _2,\\ 0, & \text {otherwise.} \end {cases} \] The values of \(\theta _1\) and \(\theta _2\) are unknown, other than \(-\infty <\theta _1<\theta _2<\infty \). Find their MLEs.

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a \(N(\theta ,100)\) distribution. Find a uniformly most powerful critical region of size \(\alpha =.10\) for testing \(H_0:\theta =75\) against \(H_A:\theta >75\).

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with a Beta\((\theta ,\theta )\) distribution, i.e. their density is \[ f(x;\theta )=\frac {\Gamma (2\theta )}{\Gamma (\theta )^2} x^{\theta -1}(1-x)^{\theta -1}, \qquad 0<x<1. \] Show that a best critical region for testing \(H_0:\theta =1\) versus \(H_A:\theta =2\) is \[ \left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

Let \(X_1,X_2,X_3\) be iid and distributed according to \(f(x)=2x\), \(0<x<1\). Let \(X_{(1)}<X_{(2)}<X_{(3)}\) be the order statistics. Compute the probability that the smallest of the \(X_i\) exceeds the median of the distribution.

Suppose that \(X_1,\ldots ,X_n\) form a random sample from a distribution for which the pdf \(f(x|\theta _1,\theta _2)\) is as follows: \[ f(x|\theta _1,\theta _2)= \begin {cases} \dfrac {1}{\theta _2-\theta _1}, & \theta _1<x<\theta _2,\\ 0, & \text {otherwise.} \end {cases} \] The values of \(\theta _1\) and \(\theta _2\) are unknown, other than \(-\infty <\theta _1<\theta _2<\infty \). Find their MLEs.

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a \(N(\theta ,100)\) distribution. Find a uniformly most powerful critical region of size \(\alpha =.10\) for testing \(H_0:\theta =75\) against \(H_A:\theta >75\).

If \(X_1,X_2,\ldots ,X_n\) is a random sample from a distribution having pdf of the form \(f(x;\theta )=\theta x^{\theta -1}\), \(0<x<1\) and zero elsewhere, show that a best critical region for testing \(H_0:\theta =1\) against \(H_1:\theta =2\) is \[ C=\left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

Let \(X_1\) and \(X_2\) be two independent random variables. Suppose that \(X_1\) is \(\chi ^2(r_1)\) and \(X_1+X_2\) is \(\chi ^2(r)\), where \(r_1,r\) are the respective degrees of freedom and \(r_1<r\). Show that \(X_2\) is \(\chi ^2(r-r_1)\).

1000 individuals were classified according to sex and to whether or not they were color-blind, with the following results: male normal 442; female normal 514; male color-blind 38; and female color-blind 6. According to a genetic model, the four relative frequencies should be of the form \(p/2\); \(p^2/2+p(1-p)\); \((1-p)/2\); and \((1-p)^2/2\), respectively. Suppose we want to test whether the data fit the model.

(a) Find the maximum likelihood estimator \(\hat p\) of \(p\).

(b) Describe a size \(\alpha =0.10\) test of the null hypothesis that the data fit the model, assuming that \(\hat p=0.913\). Specify the critical region as precisely as possible, perhaps in terms of a quantile of a certain \(\chi ^2\) distribution.

Let \(X_1,X_2,\ldots ,X_n\) be a random sample from a \(N(\theta ,\sigma ^2)\) distribution (\(\sigma ^2\) known), and let the prior distribution of \(\theta \) be \(N(\mu _0,\sigma _0^2)\), where \(\mu _0\) and \(\sigma _0^2\) are known.

(a) Find the posterior distribution of \(\theta \).

(b) Find the Bayes estimator under squared error loss.

By considering the Poisson distribution, show that \[ e^{-n}\left (1+n+\frac {n^2}{2!}+\cdots +\frac {n^n}{n!}\right )\to \frac {1}{2} \] as \(n\to \infty \).

By considering the Poisson distribution, show that \[ e^{-n}\left (1+n+\frac {n^2}{2!}+\cdots +\frac {n^n}{n!}\right )\to \frac {1}{2} \] as \(n\to \infty \).

Let \(X_1,\ldots ,X_n\) be a random sample from a \(N(\theta ,1)\) distribution, and let the prior distribution of \(\theta \) be \(N(\mu _0,1)\), where \(\mu _0\) is known.

(a) Find the posterior distribution of \(\theta \).

(b) Find the Bayes estimator of \(\theta \) under squared error loss, and show that it is a weighted average of the prior mean \(\mu _0\) and the sample mean \(\bar X\).

Let \(X\) and \(Y\) be independent, each having the standard normal distribution, and let \((R,\Theta )\) be the polar coordinates for \((X,Y)\).

(a) Show that \(X+Y\) and \(X-Y\) are independent, and that \(R^2=[(X+Y)^2+(X-Y)^2]/2\), and conclude that the conditional distribution of \(R^2\) given \(X-Y\) is the chi-squared with one degree of freedom translated by \((X-Y)^2/2\).

(b) Show that the conditional distribution of \(R^2\) given \(\Theta \) is chi-squared with two degrees of freedom.

(c) If \(X-Y=0\), the conditional distribution of \(R^2\) is chi-squared with one degree of freedom. If \(\Theta =\pi /4\) or \(\Theta =5\pi /4\), the conditional distribution of \(R^2\) is chi-squared with two degrees of freedom. But the events \(\{X-Y=0\}\) and \(\{\Theta =\pi /4\}\cup \{\Theta =5\pi /4\}\) are the same. Resolve the apparent contradiction.

Compute the distributions of: \[ \begin {array}{ll} \text {(a)} & \max _{1\leq i\leq n} X_i,\\ \text {(b)} & \min _{1\leq i\leq n} X_i,\\ \text {(c)} & X_1+\cdots +X_n. \end {array} \]

Suppose \(X_{\alpha ,\beta }\) has a Gamma distribution with parameters \((\alpha ,\beta )\). That is, the density function of \(X_{\alpha ,\beta }\) is \[ f(x)=\frac {\beta ^\alpha }{\Gamma (\alpha )}x^{\alpha -1}e^{-\beta x}, \qquad x>0, \] where \(\Gamma (\alpha )=\int _0^\infty x^{\alpha -1}e^{-x}\,dx\) for all \(\alpha >0\). Prove that \((\beta X_{\alpha ,\beta }-\alpha )/\sqrt {\alpha }\Rightarrow N(0,1)\).

Let \(X_1,X_2,\ldots ,X_n\) be independent exponential \((\theta _i)\) random variables, i.e. the density of \(X_i\) is \[ f(t,\theta _i)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ \dfrac {1}{\theta _i}e^{-t/\theta _i}, & \text {if } 0\leq t<\infty . \end {cases} \] We wish to test \(H_0:\theta _1=\theta _2=\cdots =\theta _n\) against the alternative that \(H_0\) is not true.

(a) Find the likelihood ratio test.

(b) Show that the likelihood ratio test is distribution free under \(H_0\).

(c) Show that the likelihood ratio test is equivalent with the \(\chi ^2\) test under \(H_0\).

(d) Approximate the critical values of the likelihood ratio test under \(H_0\).

Let \(X_1,X_2,\ldots ,X_n\) be independent random variables. We assume that \(X_i\) has a Poisson\((\lambda _i)\) distribution. We wish to test \[ H_0:\lambda _1=\lambda _2=\cdots =\lambda _n \] against the alternative \[ H_A:H_0\text { is not true.} \]

1. Show that the likelihood ratio test and the \(\chi ^2\) tests are equivalent under \(H_0\). (15 points)
2. Explain why and how the normal as well as the \(\chi ^2\) distributions can be used to approximate the critical values. (5 points)

Let \(X_1,\ldots ,X_n\) be a random sample from \(N(\theta ,\theta )\) (i.e., normal with mean and variance both equal to \(\theta \)), where \(\theta >0\).

(a) Give three pivotal quantities (or pivots) involving the entire sample, which respectively have a standard normal distribution, a \(t\)-distribution, and a chi-squared distribution, and indicate the numbers of degrees of freedom. (5 points)

(b) Use the normal pivotal quantity in part (a) to obtain a \(100(1-\alpha )\) percent confidence interval for \(\theta \). (5 points)

Suppose \(X_1,\cdots ,X_m\) are independent with \(X_j\sim \operatorname {Binomial}(n_j,p_j)\).

(a) Find the UMVUE’s of \(p_1,\cdots ,p_m\).

(b) Suppose you know that \(p_1=p_2=\cdots =p_m\). Let \(p\) denote this common (but unknown) value. Find the UMVUE of \(p\).

(c) Find the likelihood ratio statistic, \(\lambda \), for \(H_0:p_1=\cdots =p_m\) versus \(H_1:p_i\ne p_j\), for some \(i\) and \(j\).

(d) It can be shown that \(2\log \lambda \) is approximately the same as the usual \(\chi ^2\) statistics. Using this fact, find an approximate test for \(H_0\) vs \(H_1\) above.