Multivariate Normal Distributions

(20 points) Let \(Z = (Z_{1}, Z_{2}, Z_{3})^{\top } \sim \mathcal {N}(0, I_{3})\) and define the random vector \[ X = \begin {pmatrix} X_{1} \\ X_{2} \end {pmatrix} = \begin {pmatrix} Z_{1} + Z_{2} \\ Z_{2} + Z_{3} \end {pmatrix}. \]

1. Compute the mean vector \(\mathbb {E}[X]\) and the covariance matrix \(\Sigma = \operatorname {Cov}[X]\).
2. Does \(X\) have a probability density function on \(\mathbb {R}^{2}\)? Briefly justify your answer.
3. Does the joint moment generating function satisfy \[ M_{X}(t_{1}, t_{2}) = M_{X}(t_{1}, 0)M_{X}(0, t_{2})? \] Explain your reasoning.
4. Let \(Q = \|X\|^{2}\). Identify a symmetric matrix \(A\) such that \(Q = Z^{\top }AZ\) and compute \(\mathbb {E}[Q]\).

(20 points) Researchers compare three classroom settings and measure a standardized reading score. The observed scores are: \[ \begin {aligned} \text {Group 1: }&18, 21, 20 &&(n_{1} = 3),\\ \text {Group 2: }&23, 22, 24, 21 &&(n_{2} = 4),\\ \text {Group 3: }&26, 28, 27, 29, 25 &&(n_{3} = 5). \end {aligned} \]

Suppose the reading score for the \(i\)th individual in group \(j\) is modeled as \[ Y_{ij} = \mu _{j} + \epsilon _{ij}, \qquad \epsilon _{ij} \sim \mathcal {N}(0, \sigma ^{2}), \] with all errors independent.

Equivalently, this can be written in matrix form as \[ Y = \mathbf {X}\beta + \epsilon . \]

1. Write the model in matrix form by explicitly constructing the design matrix \(\mathbf {X}\), the parameter vector \(\beta \), and the response vector \(Y\).
2. Express the null hypothesis “all three group means are equal” in the form \[ H_{0}: A\beta = c, \] for explicit matrices \(A\) and \(c\).
3. You are told that the residual sum of squares under the full model is \[ RSS = 38.4 \] and under the reduced model (i.e. under \(H_{0}\)) it is \[ RSS_{H_{0}} = 62.1. \] Compute the \(F\)-statistic for testing \(H_{0}\). State the degrees of freedom and give the conclusion at level \(\alpha = 0.05\).
4. Could a \(t\)-test be used instead of an \(F\)-test here? Briefly explain.

(20 points) Researchers study how a child’s reading score \(Y\) depends on home reading time \(x_{1}\) (hours/week) and classroom support level \(x_{2}\) (a small integer score). It is believed that extra home reading time is more effective when classroom support is higher, so the mean is modeled with an interaction between \(x_{1}\) and \(x_{2}\). To make the intercept interpretable, home reading time is centered at \(1\) hour/week.

Consider the linear model \[ Y_{i} = \beta _{0} + \beta _{1}(x_{1,i} - 1) + \beta _{2}(x_{1,i} \times x_{2,i}) + \epsilon _{i}, \qquad i = 1, \ldots , 5, \] where \(\epsilon _{i} \sim \mathcal {N}(0, \sigma ^{2})\) are independent. The observed data consist of: \[ (x_{1}, x_{2}, Y) = (0, 1, 4), (1, 0, 3), (2, 1, 7), (1, 2, 6), (3, 1, 9). \]

A least squares fit yields \(\hat {\beta } = (\hat {\beta }_{0}, \hat {\beta }_{1}, \hat {\beta }_{2})^{\top } = (2.10, 0.90, 1.20)^{\top }\) and \(RSS = 1.50\).

1. Write down the design matrix \(\mathbf {X}\) for this regression model and the parameter vector \(\beta = (\beta _{0}, \beta _{1}, \beta _{2})^{\top }\). Identify the estimated variance \(S^{2}\).
2. We wish to test the joint null hypothesis that there is no effect of the predictors beyond the baseline, \[ H_{0}: \beta _{1} = \beta _{2} = 0. \] Express this hypothesis in matrix form \(A\beta = 0\) for an appropriate matrix \(A\), and state \(q = \operatorname {rank}(A)\).
3. Write down the \(F\)-statistic for testing \(H_{0}\) and state the distribution of \(F\) under \(H_{0}\). (You do not need to evaluate the formula.)
4. Using Scheffé’s method, construct simultaneous 95% confidence intervals for \(\beta _{0}, \beta _{1}, \beta _{2}\). You may express these intervals in terms of \((\mathbf {X}^{\top }\mathbf {X})^{-1}\), but plug in all other quantities.

Let \(\epsilon _{1}, \ldots , \epsilon _{n}\) and \(\eta _{1}, \ldots , \eta _{m}\) be independent normal random variables with \(\mathbb {E}\epsilon _{i} = \mathbb {E}\eta _{j} = 0\) and \(\mathbb {E}\epsilon _{i}^{2} = 1, \mathbb {E}\eta _{j}^{2} = 4\). We have the models

where the \(x_{i}, y_{i}\) are given and non-random.

1. Find the maximum likelihood estimator for \((\beta _{1}, \beta _{2}, \beta _{3} )\).
2. Compute the distribution of the estimator.

Let \((X_{1}, X_{2})\) be a bivariate normal with \(\mathbb {E}X_{1} = \mathbb {E}X_{2} = 0\) and covariance matrix \[ \begin {bmatrix} 2 & \rho \\ \rho & 2 \end {bmatrix}. \] Determine the \(\rho \) such that \(2X_{1} + X_{2}\) and \(X_{1} + 3X_{2}\) are independent.

Let \((X_{1}, X_{2})\) be independent normal random variables with \(\mathbb {E}X_{1} = \mathbb {E}X_{2} = 0\) and \(\mathbb {E}X_{1}^{2} =\) \(1, \mathbb {E}X_{2}^{2} = 4\). Define \(Y_{1} = X_{1} + X_{2}\) and \(Y_{2} = X_{1} + 2X_{2}\) and let \(\mathbf {Y} = (Y_{1}, Y_{2})^{\top }\).

1. Compute the covariance matrix of \(\mathbf {Y}\).
2. Find a matrix \(\mathbf {D}\) such that \(\mathbf {Y}^{\top }\mathbf {D}\mathbf {Y}\) has a chi-squared distribution.

Let \(Z_n\) be a martingale difference with respect to a filtration \(\mathcal {F}_n\), i.e. \(\mathbb {E}[Z_{n+1}\mid \mathcal {F}_n]=0\) for all \(n\geq 0\). Assume the following hold:

1. \(n^{-1/2}\max _{m\leq n}|Z_m|\to 0\) in probability.
2. \(n^{-1}\sum _{m=1}^{n}|Z_m|^2\to 1\) in probability.
3. \(\sum _{m=1}^{n}|Z_m|^2\leq 2n\) almost surely, for all \(n\).

Prove that \(M_n=n^{-1/2}\sum _{m=1}^{n}Z_m\) converges in distribution to a standard normal.

Hint: Prove that \(|1+ix|\leq e^{x^2}\) for all \(x\in \mathbb {R}\) and \(e^{ix}=(1+ix)e^{-x^2/2+r(x)}\) with \(|r(x)|\leq x^3\) for all \(x\in \mathbb {R}\). Let \[ T_n=\prod _{m=1}^{n}\left (1+itn^{-1/2}Z_m\right ) \] and \[ U_n= e^{-\frac {t^2}{2n}\sum _{m=1}^{n}Z_m^2 +\sum _{m=1}^{n}r(tn^{-1/2}Z_m)}. \] Compute \(\mathbb {E}[T_n]\) then prove that \(\mathbb {E}[T_nU_n]\to e^{-t^2/2}\) and conclude.

Let \(X\) and \(Y\) be independent \(N(0,1)\) random variables, and let \(Z=X+Y\).

1. Show that \(\mathbb {E}[Z\mid X>0,Y>0]=2\sqrt {2/\pi }\).
2. Find the distribution and the density of \(Z\) given that \(X>0\) and \(Y>0\). (You can use the CDF of a standard normal, \(\Phi (x)=\mathbb {P}(X\leq x)\), in your expression.)

Let \(g:\mathbb {R}\to \mathbb {R}\) be bounded and continuous and fix \(\lambda >0\). Use properties of the Poisson distribution to show that \[ \sum _{k=0}^{\infty }g(k/n)\frac {(n\lambda )^k}{k!}e^{-n\lambda } \to g(\lambda ) \qquad \text {as } n\to \infty . \]

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with a Beta\((\theta ,\theta )\) distribution, i.e. their density is \[ f(x;\theta )=\frac {\Gamma (2\theta )}{\Gamma (\theta )^2} x^{\theta -1}(1-x)^{\theta -1}, \qquad 0<x<1. \] Show that a best critical region for testing \(H_0:\theta =1\) versus \(H_A:\theta =2\) is \[ \left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

Let \(X_1,X_2,X_3\) be iid and distributed according to \(f(x)=2x\), \(0<x<1\). Let \(X_{(1)}<X_{(2)}<X_{(3)}\) be the order statistics. Compute the probability that the smallest of the \(X_i\) exceeds the median of the distribution.

Suppose that \(X_1,\ldots ,X_n\) form a random sample from a distribution for which the pdf \(f(x|\theta _1,\theta _2)\) is as follows: \[ f(x|\theta _1,\theta _2)= \begin {cases} \dfrac {1}{\theta _2-\theta _1}, & \theta _1<x<\theta _2,\\ 0, & \text {otherwise.} \end {cases} \] The values of \(\theta _1\) and \(\theta _2\) are unknown, other than \(-\infty <\theta _1<\theta _2<\infty \). Find their MLEs.

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a \(N(\theta ,100)\) distribution. Find a uniformly most powerful critical region of size \(\alpha =.10\) for testing \(H_0:\theta =75\) against \(H_A:\theta >75\).

Let \(\{X_i:i\geq 1\}\) be i.i.d. random variables distributed uniformly on \([0,1]\). Prove that the distribution of \[ \frac {4\sum _{i=1}^{n} iX_i-n^2}{n^{3/2}} \] converges weakly and identify the limiting distribution.

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with a Beta\((\theta ,\theta )\) distribution, i.e. their density is \[ f(x;\theta )=\frac {\Gamma (2\theta )}{\Gamma (\theta )^2} x^{\theta -1}(1-x)^{\theta -1}, \qquad 0<x<1. \] Show that a best critical region for testing \(H_0:\theta =1\) versus \(H_A:\theta =2\) is \[ \left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

Let \(X_1,X_2,X_3\) be iid and distributed according to \(f(x)=2x\), \(0<x<1\). Let \(X_{(1)}<X_{(2)}<X_{(3)}\) be the order statistics. Compute the probability that the smallest of the \(X_i\) exceeds the median of the distribution.

Suppose that \(X_1,\ldots ,X_n\) form a random sample from a distribution for which the pdf \(f(x|\theta _1,\theta _2)\) is as follows: \[ f(x|\theta _1,\theta _2)= \begin {cases} \dfrac {1}{\theta _2-\theta _1}, & \theta _1<x<\theta _2,\\ 0, & \text {otherwise.} \end {cases} \] The values of \(\theta _1\) and \(\theta _2\) are unknown, other than \(-\infty <\theta _1<\theta _2<\infty \). Find their MLEs.

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a \(N(\theta ,100)\) distribution. Find a uniformly most powerful critical region of size \(\alpha =.10\) for testing \(H_0:\theta =75\) against \(H_A:\theta >75\).

Assume that the lifetimes of 100 lightbulbs are iid random variables with an Exponential(5) distribution, meaning that their common pdf is \[ f(x;\theta )=\frac {1}{5}e^{-x/5}, \qquad x>0. \] Use normal approximation to estimate the probability that in total the 100 lightbulbs last longer than 550 hours. You can express your answer as an integral.

Let \(X_1,\ldots ,X_m\) be \(m\geq 2\) i.i.d. standard normal random variables, and define the random vector \[ \mathbf {X}:= \begin {bmatrix} X_1\\ \vdots \\ X_m \end {bmatrix}. \] Prove that the distribution of \(\mathbf {X}\) is the same as the distribution of \(A\mathbf {X}\) for every \(m\times m\) orthogonal matrix \(A\) that is non random.

Fix \(0<\lambda <\rho \) and let \(X\), \(Y\), and \(Z\) be three independent Gamma distributed random variables with common scale parameter 1 and shape parameters \(\lambda \), \(\rho -\lambda \), and \(\rho \), respectively. (A Gamma random variable with scale parameter 1 and shape parameter \(\lambda \) has pdf \(\frac {1}{\Gamma (\lambda )}x^{\lambda -1}e^{-x}\mathbf {1}_{\{x>0\}}\).) Let \[ U=\frac {ZX}{X+Y},\qquad V=\frac {ZY}{X+Y},\qquad \text {and}\qquad W=X+Y. \] Prove that random vector \((U,V,W)\) has the same distribution as \((X,Y,Z)\). (It may be useful to note that \(Z=U+V\).)

Consider an array \(\{X_{i,j}\}\) of random variables such that, for every \(n\geq 1\), the collection \(X_{1,n},\ldots ,X_{n,n}\) is i.i.d. with common distribution \[ \mathbb {P}\{X_{1,n}=1\} = \mathbb {P}\{X_{1,n}=-1\} = \frac {1}{2}\left (1-\frac {1}{n^2}\right ); \] and \(\mathbb {P}\{X_{1,n}=n^2\}=n^{-2}\). Let \(S_n:=X_{1,n}+\cdots +X_{n,n}\). Prove that \(S_n/\sqrt {n}\Rightarrow \mathrm {N}(0,\sigma ^2)\) for some \(\sigma \in (0,\infty )\), although \(\mathbb {E}(S_n/\sqrt {n})\to \infty \) as \(n\to \infty \).

If \(X_1,X_2,\ldots \) are independent standard normal random variables, then find non-random sequences \(a_n,b_n\to \infty \) such that \(a_n\min _{1\leq i\leq n}X_i+b_n\) converges weakly. Identify the limiting distribution.

Let \(X_1,X_2,\ldots ,X_n\) be iid with common pdf \(f(x)=2x\), \(0<x<1\) and zero elsewhere. Find the limiting distribution of \(n(1-X_{(n)})\), where \(X_{(n)}=\max \{X_1,\ldots ,X_n\}\).

If a random sample of size \(n\) is taken from a distribution having pdf \(f(x;\theta )=2x/\theta ^2\), \(0<x\leq \theta \) and zero elsewhere, find:

(a) the MLE \(\hat \theta \) for \(\theta \),

(b) the constant \(c\) so that \(\mathbb {E}[c\hat \theta ]=\theta \),

(c) the MLE for the median of the distribution.

If \(X_1,X_2,\ldots ,X_n\) is a random sample from a distribution having pdf of the form \(f(x;\theta )=\theta x^{\theta -1}\), \(0<x<1\) and zero elsewhere, show that a best critical region for testing \(H_0:\theta =1\) against \(H_1:\theta =2\) is \[ C=\left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a normal distribution with mean \(\theta \) and variance 100. Find a uniformly most powerful critical region of size \(\alpha =0.10\) for testing \(H_0:\theta =75\) against \(H_1:\theta >75\).

Let \(X_1,X_2,\ldots ,X_n\) be a random sample from a Gamma\((\alpha ,\theta )\) distribution, with \(\alpha \) known and \(\theta \) unknown. Find the Fisher information of this distribution and show that the MLE of \(\theta \) is an efficient estimator of \(\theta \).

Let \(X_1\) and \(X_2\) be independent with joint pdf \(f(x)=x^{-2}\) for \(1\leq x<\infty \) and 0 otherwise.

(a) Find the joint pdf of \(U=X_1X_2\) and \(V=X_2\).

(b) Find the marginal pdf of \(U\).

Let \(X_1,X_2,\ldots ,X_n\) be iid exponential(1) random variables and \[ \bar X_n=\frac {1}{n}\sum _{i=1}^n X_i \] be the sample mean. Find two sequences of numbers \(a_n\) and \(b_n\) such that \[ \frac {\log \bar X_n-a_n}{b_n} \] converges in distribution to a standard normal random variable.

Suppose that \(X_1,\ldots ,X_n\) are iid from a \(\BETA (\theta _1,\theta _2)\) distribution, meaning that the pdf is \[ f(x;\theta _1,\theta _2)= \frac {\Gamma (\theta _1+\theta _2)}{\Gamma (\theta _1)\Gamma (\theta _2)} x^{\theta _1-1}(1-x)^{\theta _2-1}, \qquad 0<x<1. \] Find jointly sufficient statistics for \(\theta _1\) and \(\theta _2\), and use them to find a UMVUEs for \(\theta _1\) and \(\theta _2\).

Let \(X_1,X_2,\ldots ,X_n\) be iid from a distribution with CDF \(F(x)=1-1/x\) for \(1\leq x<\infty \), and zero otherwise.

(a) Derive the CDF of the smallest order statistic \(X_{(1)}\).

(b) Find the limiting distribution of \(X_{(1)}\).

(c) Find the limiting distribution of \(X_{(1)}^n\).

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables uniform on \([0,1]\) and let \(\alpha >0\). Show that there are numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n=\frac {\sum _{k=1}^{n}k^{\alpha }X_k-a_n}{b_n} \] converges in distribution to a standard normal random variable.

We assume that the number of people entering a store follows a Poisson distribution with parameter \(\lambda \). Every customer spends money in the store, independently of each other according to a uniform distribution on \([10,100]\). We observe \(X\), the total amount of money spent in the store on a given day.

(a) Estimate \(\lambda \) using the method of moments.

(b) Let assume that we have the stores income \(X_1,X_2,\ldots ,X_n\) for \(n\) days and \(X_1,X_2,\ldots ,X_n\) are independent and distributed as \(X\). What would be the moment estimator for \(\lambda \) based on \(X_1,X_2,\ldots ,X_n\)?

Let \(U_1,U_2,\ldots ,U_n\) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) denote the order statistics.

(a) Compute the limit distribution of \(Z=n(U_{3,n}-U_{2,n})\).

(b) Show that \(Z_1=n(U_{3,n}-U_{2,n})\) and \(Z_2=n(U_{4,n}-U_{3,n})\) are asymptotically independent.

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with distribution function \(F\). We wish to test that \(F\) is a normal distribution function. Provide at least two different methods on to test for the normality of \(F\).

Let \(X_1,X_2,\ldots ,X_n\) be a random sample from the Poisson distribution with parameter \(\lambda >0\). Use the Rao–Blackwell theorem and/or the Lehmann–Scheffé theorem to find a UMVUE of \(\mathbb {P}\{X_1=0\}=e^{-\lambda }\).

Let \(X_1\) and \(X_2\) be independent \(\operatorname {Exponential}(1)\) random variables (density \(e^{-x}\), \(x>0\)). Find the joint density of \(Y_1=X_1/X_2\) and \(Y_2=X_1+X_2\). Find the marginal densities of \(Y_1\) and \(Y_2\) as well.

Let \(X\) and \(Y\) be independent random variables. Show that if the distribution of \(X\) is absolutely continuous with respect to the Lebesgue measure, then the distribution of \(X+Y\) is also absolutely continuous with respect to the Lebesgue measure.

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables uniform on \([0,1]\). Show that for all \(\alpha >-1\) there are numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n= \frac {\sum _{1\leq k\leq n} k^{\alpha }X_k-a_n}{b_n} \] converges in distribution to a standard normal random variable.

Let \(X,X_1,X_2,\ldots \) be independent random variables, all distributed according to the standard normal distribution.

(a) Prove that there exist finite constants \(c_1,c_2,x_0>0\), such that \[ \frac {c_1}{x}e^{-x^2/2}\leq \mathbb {P}\{X>x\}\leq \frac {c_2}{x}e^{-x^2/2} \qquad \text {for all }x\geq x_0. \]

(b) Use the inequalities of part (a) in order to prove that if \(X_1,X_2,\ldots \) are independent standard normal random variables then \[ \limsup _{n\to \infty }\frac {X_n}{\sqrt {2\ln n}} = -\liminf _{n\to \infty }\frac {X_n}{\sqrt {2\ln n}} =1 \qquad \text {a.s.} \]

(a) Let \(\Phi \) and \(\phi \) be the standard normal distribution and density functions, respectively. Show that \[ \lim _{x\to \infty }\frac {1-\Phi (x)}{(1/x)\phi (x)}=1. \]

(b) Let \(X_1,X_2,\ldots \) be independent identically distributed standard normal random variables. Show that \[ \limsup _{n\to \infty }\frac {1}{\sqrt {\log n}}|X_n|=c \quad \text {almost surely} \] and compute the value of \(c\).

Let \(Z_n\) be a Galton–Watson branching process with offspring distribution \(\{p_k,\ k=0,1,2,\ldots \}\) and \(Z_0=x\) (with \(x\) a positive integer), and let \[ f(\theta )=\sum p_k\theta ^k \] be the associated pgf. Suppose that \(\rho \in (0,1)\) satisfies \(f(\rho )=\rho \). Show that \(\rho ^{Z_n}\) is a martingale, and use this to conclude that \(\mathbb {P}(Z_n=0\text { for some }n\geq 0)=\rho ^x\).

By considering the Poisson distribution, show that \[ e^{-n}\left (1+n+\frac {n^2}{2!}+\cdots +\frac {n^n}{n!}\right )\to \frac {1}{2} \] as \(n\to \infty \).

Let \(Z_n\) be a Galton–Watson branching process with offspring distribution \(\{p_k,\ k=0,1,2,\ldots \}\) and \(Z_0=x\) (with \(x\) a positive integer), and let \[ f(\theta )=\sum p_k\theta ^k \] be the associated pgf. Suppose that \(\rho \in (0,1)\) satisfies \(f(\rho )=\rho \). Show that \(\rho ^{Z_n}\) is a martingale, and use this to conclude that \(\mathbb {P}(Z_n=0\text { for some }n\geq 0)=\rho ^x\).

By considering the Poisson distribution, show that \[ e^{-n}\left (1+n+\frac {n^2}{2!}+\cdots +\frac {n^n}{n!}\right )\to \frac {1}{2} \] as \(n\to \infty \).

Let \(X_1\) and \(X_2\) be independent random variables. The density function function of \(X_1\) is \[ f(t)= \begin {cases} 0, & \text {if } t\notin [0,1],\\ \dfrac {e^t}{e-1}, & \text {if } 0\leq t\leq 1. \end {cases} \] The distribution of \(X_2\) is \[ \mathbb {P}\{X_2=1\}=p \quad \text {and}\quad \mathbb {P}\{X_2=-1\}=q,\qquad p+q=1. \] Compute the moment generating function of \(X_1X_2\).

Let \(X_1,\ldots ,X_n\) be an i.i.d. sample from a \(\operatorname {Uniform}(0,\theta )\) distribution, where \(\theta >0\) is unknown. Find a 95% confidence interval for \(\theta \).

Let \(X_1,\ldots ,X_n\) denote an independent sample from an exponential distribution with [unknown] mean \(\theta >0\). What does the Neyman–Pearson lemma say about \(H_0:\theta =1\) versus \(H_a:\theta =2\)? Explain carefully, and identify explicitly the rejection region.

The following data are times (in hours) between failures of air conditioning equipment in a particular airplane: \[ 74\quad 57\quad 48\quad 29\quad 502\quad 12\quad 70\quad 21\quad 29\quad 386\quad 59\quad 27\quad 153\quad 26\quad 326. \] Assume that the data are observed values of an i.i.d. random sample from an exponential distribution, \(X_i\sim \operatorname {Exponential}(\theta )\). Test \(H_0:\theta =125\) versus \(H_a:\theta \ne 125\). (A chi-square table is provided.)

Let \(X\) and \(Y\) be independent, standard-normal random variables. Find the distribution of \(X/Y\)?

Assume a sample of size \(n\) from \(\operatorname {Uniform}[\eta -\theta ,\eta +\theta ]\), which is a location-scale family. Here \(\eta \) is real and \(\theta >0\).

(a) Find a \(100(1-\alpha )\%\) confidence interval for \(\theta \).

(b) Find a \(100(1-\alpha )\%\) confidence interval for \(\eta \).

It is not necessary to find the explicit distribution of the pivotal quantities; just denote the needed quantiles by \(x_p\) and \(y_p\) for an appropriate \(p\).

Let \(U_1,U_2,\ldots ,U_n\) be independent random variables uniform on \([0,1]\) and let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) be the corresponding order statistics.

a. Compute the asymptotic distribution of \(nU_{2,n}\). (5 points)

b. Show that \(nU_{1,n}\) and \(n(1-U_{n,n})\) are asymptotically independent. (5 points)

Let \(y_i=\alpha x_i+\epsilon _i\), \(1\leq i\leq n\), where \(\epsilon _1,\epsilon _2,\ldots ,\epsilon _n\) are independent identically distributed normal \(N(0,\sigma ^2)\) random variables.

a. Determine the maximum likelihood estimators for \(\alpha \) and \(\sigma ^2\). (5 points)

b. Compute the distribution of the maximum likelihood estimator for \(\alpha \). (5 points)

c. Compute the distribution of the maximum likelihood estimator for \(\sigma ^2\). (5 points)

Let \(X\) and \(Y\) be two independent random variables with distribution functions \(F\) and \(G\) and density functions \(f\) and \(g\).

a. Determine the distribution function of \(XY\). (5 points)

b. Does \(XY\) always have a density function? (5 points)

c. Assuming that \(XY\) has a density function, compute it. (5 points)

Let \(\Phi \) and \(\phi \) denote the standard normal distribution and density functions.

(a) Prove that for all \(x>0\) \[ 1-\Phi (x)\leq \frac {1}{x}\phi (x). \] (5 points)

Let \(X_{1,n}<X_{2,n}\leq \cdots \leq X_{n,n}\) denote the order statistics from a sample of \(n\) independent identically distributed standard normal random variables.

(b) Show that \(X_{n,n}\) converges in probability to infinity. (5 points)

(c) Show that for all \(\epsilon >0\) \[ \lim _{n\to \infty } \mathbb {P}\left \{X_{n,n}\leq \sqrt {(2+\epsilon )\log n}\right \}=1. \] (5 points)

Let \(X\) and \(Y\) be independent exponential random variables with parameters \(\lambda \) and \(\mu \). (For example, \(\mathbb {P}(X>x)=e^{-\lambda x}\) for \(x>0\).)

(a) Show that \(Z:=\min (X,Y)\) is independent of the event \(\{X<Y\}\).

(b) Find \(\mathbb {P}(X=Z)\).

(c) Find the distribution of \(U:=(X-Y)^+=\max (X-Y,0)\).

(d) Find the distribution of \(V:=\max (X,Y)-\min (X,Y)\).

Let \(X\) and \(Y\) have joint density \[ f(x,y)=g\left (\sqrt {x^2+y^2}\right ),\qquad (x,y)\in \mathbb {R}^2, \] for some function \(g\). Show that \(Z=X/Y\) has a Cauchy density.

Let \(X_1,X_2,\ldots \) be i.i.d. mean 0, variance 1 random variables and \(S_n=X_1+\cdots +X_n\) for each \(n\geq 1\). Show directly that the characteristic function of \(S_n/\sqrt {n}\) converges pointwise to the characteristic function of the standard normal distribution. Notice that this is a key step in the proof of the central limit theorem, so you are not allowed to use the central limit theorem in your derivation.

Let \(X\) and \(Y\) be independent, each having the standard normal distribution, and let \((R,\Theta )\) be the polar coordinates for \((X,Y)\).

(a) Show that \(X+Y\) and \(X-Y\) are independent, and that \(R^2=[(X+Y)^2+(X-Y)^2]/2\), and conclude that the conditional distribution of \(R^2\) given \(X-Y\) is the chi-squared with one degree of freedom translated by \((X-Y)^2/2\).

(b) Show that the conditional distribution of \(R^2\) given \(\Theta \) is chi-squared with two degrees of freedom.

(c) If \(X-Y=0\), the conditional distribution of \(R^2\) is chi-squared with one degree of freedom. If \(\Theta =\pi /4\) or \(\Theta =5\pi /4\), the conditional distribution of \(R^2\) is chi-squared with two degrees of freedom. But the events \(\{X-Y=0\}\) and \(\{\Theta =\pi /4\}\cup \{\Theta =5\pi /4\}\) are the same. Resolve the apparent contradiction.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We use \[ \hat f_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n} K\left (\frac {t-X_i}{h}\right ) \] to estimate \(f=F'\).

We assume that the following conditions are satisfied: \[ \int _{-1}^{1}K(u)\,du=1, \qquad K(t)=0,\quad \text {if } t\notin [-1,1]. \] \(F\) is three times differentiable and the derivatives are uniformly bounded on the real line \[ h=h(n)\to 0. \]

(i) Compute the bias of \(\hat f_n(t)\) for any \(t\) and any \(n\). (7 points)

(ii) Prove that \(\hat f_n(t)\) is asymptotically unbiased for any \(t\) as \(n\to \infty \). (7 points)

(iii) Compute the mean square error of \(\hat f_n(t)\) (7 points)

Let \(X\) be a random variable with distribution function \(F\). Prove that \(X\) is symmetric around 0 (i.e. \(F(x)=1-F(-x)\) for all \(x\)) if and only if the characteristic function of \(X\) is real. (7 points)

Let \(X\) have the binomial distribution with parameters \(n=10\) and \(p=\theta \) unknown. Suppose we impose a uniform-\((0,1)\) prior on \(\theta \). Compute the posterior mean of \(\theta \). Is it consistent?

Let \(X_1, X_2, \ldots \) denote an i.i.d. sample with common distribution function \(F\). Define \[ F_n(x) = n^{-1}\sum _{j=1}^{n} I\{X_j \leq x\} \] for all \(x \in (-\infty , \infty )\); i.e., \(F_n\) is the empirical distribution function based on the \(X_i\)’s. Prove that for all \(x \in (-\infty , \infty )\) fixed, \[ \operatorname {Var}\bigl (F_n(x)-F(x)\bigr ) \leq \frac {1}{4n}. \] Use this to conclude that \(F_n(x)\) is a consistent estimator for \(F(x)\).

From here on let \(X_1, X_2, \ldots \) denote an i.i.d. sample from an exponential density with mean \(1/\theta \).

Compute the distributions of: \[ \begin {array}{ll} \text {(a)} & \max _{1\leq i\leq n} X_i,\\ \text {(b)} & \min _{1\leq i\leq n} X_i,\\ \text {(c)} & X_1+\cdots +X_n. \end {array} \]

Let \(\{X_i\}_{i=1}^{\infty }\) be i.i.d. with the “standard Cauchy distribution.” That is, the density function of \(X_1\) is \[ f(x)=\frac {1}{\pi (1+x^2)} \qquad -\infty <x<\infty . \] Prove that \(\max _{1\le i\le n}X_i/n\) converges weakly. Identify the limit.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with a continuous distribution function \(F\). Let \(x_{1/4}\) be defined by \(F(x_{1/4})=1/4\).

(a) Find a \(1-\alpha \) lower confidence bound for \(x_{1/4}\).

(b) Provide a large sample approximation.

Let \(\epsilon _i\), \(1\leq i\leq n\), be independent identically distributed normal \(N(0,\sigma ^2)\) random variables and consider the model \[ y_i=i\alpha +\gamma i^2+\epsilon _i,\qquad 1\leq i\leq n. \] We observe \(y_i\), \(1\leq i\leq n\).

(a) Find the maximum likelihood estimators for \(\alpha \), \(\gamma \), and \(\sigma ^2\).

(b) Compute the joint distribution of the maximum likelihood estimators for \((\alpha ,\gamma )\).

Suppose \(X_{\alpha ,\beta }\) has a Gamma distribution with parameters \((\alpha ,\beta )\). That is, the density function of \(X_{\alpha ,\beta }\) is \[ f(x)=\frac {\beta ^\alpha }{\Gamma (\alpha )}x^{\alpha -1}e^{-\beta x}, \qquad x>0, \] where \(\Gamma (\alpha )=\int _0^\infty x^{\alpha -1}e^{-x}\,dx\) for all \(\alpha >0\). Prove that \((\beta X_{\alpha ,\beta }-\alpha )/\sqrt {\alpha }\Rightarrow N(0,1)\).

Let \(\{X_n\}_{n=1}^{\infty }\) be i.i.d. random variables with distribution, \(\mathbb {P}\{X_1=2\}=\mathbb {P}\{X_1=-2\}=1/2\). Define \(S_n=\sum _{i=1}^{n}X_i\) and \(T=\inf \{n\ge 1:S_n\in \{-2,6\}\}\). Prove that \(T<\infty \) a.s., and then compute \(\mathbb {P}\{S_T=-2\}\).

Let \(\{X_n\}_{n=1}^{\infty }\) be a sequence of i.i.d. random variables with \(\mathbb {E}[X_1]=0\) and \(\mathbb {E}[X_1^2]=1\). Define \(S_k=X_1+\cdots +X_k\) for all \(k\ge 1\). Let \(N_m\) be another independent random variable with a mean-\(m\) Poisson distribution. Prove that there exist non-random \(\alpha _m\) and \(\beta _m\) such that \((S_{N_m}-\alpha _m)/\beta _m\) converges weakly to a standard-normal distribution as \(m\to \infty \). Describe an explicit example of \(\alpha _m\) and \(\beta _m\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential (1) random variables with common density function \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ e^{-t}, & \text {if } 0\leq t<\infty . \end {cases} \] (a) Show that \(\left (\sum _{1\leq i\leq n}X_i\right )^{1/2}-n^{1/2}\) converges in distribution to a normal random variable.

(b) Compute the mean and the variance of the limiting normal random variable.

\(X\) and \(Y\) are two independent identically distributed normal \(N(0,\sigma ^2)\) random variables. Let \[ Z=X^2+Y^2 \] and \[ W=\frac {X}{(X^2+Y^2)^{1/2}}. \] (a) Show that \(Z\) and \(W\) are independent.

(b) Compute the marginal distributions of \(Z\) and \(W\).

Let \(y_i=\alpha x_i+\epsilon _i\), where \(\epsilon _i\), \(1\leq i\leq n\), are independent, identically distributed normal \(N(0,\sigma ^2)\) random variables.

(a) Find the maximum likelihood estimators for \(\alpha \) and \(\sigma ^2\).

(b) Find the distributions of the maximum likelihood estimators.

(c) Compute the least-squares estimator for \(\alpha \).

Let \(X\) be such that \(\mathbb {E}[X^+]=\infty \) and \(\mathbb {E}[X^-]<\infty \), and let \(X_1,X_2,X_3,\ldots \) be i.i.d. with common distribution that of \(X\). With \(S_n=X_1+X_2+\cdots +X_n\), show that \(\lim _{n\to \infty }S_n/n=\infty \) a.s.

Let \(X_1,X_2,X_3,\ldots \) be i.i.d. nonnegative random variables with mean 1 and variance 1, and put \(S_n=X_1+X_2+\cdots +X_n\). Show that \(2(\sqrt {S_n}-\sqrt {n})\) converges in distribution to a standard normal as \(n\to \infty \).

Let \(X_1,X_2,X_3,\ldots \) be a sequence of random variables, each of which has all moments finite, and suppose that \(\lim _{n\to \infty }\mathbb {E}[(X_n)^k]=k!\) for each \(k\ge 1\). Show that \(X_n\) converges in distribution.

Let \(\mathbb {P}\{X=1\}=\mathbb {P}\{X=-1\}=1/2\), let \(X_1,X_2,X_3,\ldots \) be i.i.d. with common distribution that of \(X\), and put \(S_n=X_1+X_2+\cdots +X_n\). Define \(T=\min \{n\ge 0:S_n=-A\text { or }S_n=B\}\), where \(A\) and \(B\) are positive integers. Using the martingales of problem 5, find \(\mathbb {P}\{S_T=B\}\) and \(\mathbb {E}[T]\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with a continuous distribution function \(F\). The ordered observations are \(X_{1,n}\leq X_{2,n}\leq \cdots \leq X_{n,n}\). Let \[ T_n=\sum _{i=1}^{n-1}\left (F(X_{i+1,n})-F(X_{i,n})\right )^2. \] Show that there are numerical sequences \(a_n\) and \(b_n\) such that \((T_n-a_n)/b_n\) converges in distribution to a standard normal random variable. (20 points)

Consider the following model: \[ y_i=f(x_i)+\epsilon _i,\qquad 1\leq i\leq n, \] where \[ f(t)=a_0+a_1t+\cdots +a_pt^p. \] The parameters \(a_0,a_1,\ldots ,a_p\) are unknown. The errors \(\epsilon _1,\epsilon _2,\ldots ,\epsilon _n\) are independent normal \(N(0,\sigma ^2)\) random variables. It is assumed that \(x_i\in [0,1]\) and known. We observe \((y_i,x_i)\), \(1\leq i\leq n\).

1. Estimate the parameters \(a_0,a_1,\ldots ,a_p\) and \(\sigma ^2\). (15 points)
2. Get the distribution of the estimator for \(\sigma ^2\). (5 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We assume that \(f=F'\) has a bounded second derivative. \(K(t)\) is a non-negative function satisfying \[ K(t)=K(-t)\quad \text {(symmetric),} \] \[ \int _{-\infty }^{\infty }K(t)\,dt=1 \quad \text {and}\quad K(t)=0\text { if }t\notin [-1,1]. \] The sequence \(h=h(n)\) is positive and goes to 0. We estimate \(f\) with \[ \hat {f}_n=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ). \]

1. Compute \(B_n\), the bias of \(\hat {f}_n\). (10 points)
2. Compute the limit of \(B_n/h^2(n)\) as \(n\to \infty \). (10 points)

Let \(X_1,X_2,\ldots ,X_n\) and \(Y_1,Y_2,\ldots ,Y_m\) be independent random samples of independent observations. The distribution of \(X_i\) is normal \(N(\mu _1,\sigma _1^2)\) and the distribution of \(Y_i\) is normal \(N(\mu _2,\sigma _2^2)\). The values of \(\sigma _1\) and \(\sigma _2\) are known. We wish to test \(H_0:\mu _1=\mu _2\) against \(H_A:\mu _1\neq \mu _2\). Show that the likelihood ratio test and the t-test are equivalent under \(H_0\). (20 points)

Let \(X_1,X_2,X_3\) be i.i.d., each with an exponential distribution with mean 1. Find the joint distribution of \[ Y_1=\frac {X_1}{X_1+X_2},\qquad Y_2=\frac {X_1+X_2}{X_1+X_2+X_3},\qquad Y_3=X_1+X_2+X_3. \] (7 points) In particular, are they independent? (3 points)

Let \(X_1,X_2,\ldots \) be independent identically distributed positive random variables with \(\E X_1=\mu \) and \(\operatorname {Var}(X_1)=\sigma ^2\). Let \[ N(t)=\min \left \{k:\sum _{1\le i\le k}X_i>t\right \},\qquad 0<t<\infty . \]

(a) Show that \[ \lim _{t\to \infty }\frac {1}{t}N(t)=\frac {1}{\mu } \] almost surely. (5 points)

(b) Show that \((N(t)-t/\mu )/(t\sigma ^2/\mu ^3)^{1/2}\) goes in distribution to a standard normal random variable. (5 points)

Let \(X_1,X_2,\ldots \) be independent random variables with distribution functions \(F_1,F_2,F_3,\ldots \). Let \(Y=\sup _{1\le i<\infty }X_i\).

(a) Show that \(\mathbb {P}\{Y<\infty \}\) is 0 or 1 depending on whether \(\sum _{1\le i<\infty }(1-F_i(x))\) converges for some \(x\). (5 points)

(b) Show that if \(\mathbb {P}\{Y<\infty \}=1\), then \(\prod _{1\le i<\infty }F_i(x)\) converges for all \(x\) and it is the distribution function of \(Y\).

Let \(X_1,X_2,X_3\) be i.i.d., each with an exponential distribution with mean 1. Find the joint distribution of \[ Y_1=\frac {X_1}{X_1+X_2},\qquad Y_2=\frac {X_1+X_2}{X_1+X_2+X_3},\qquad Y_3=X_1+X_2+X_3. \] (7 points) In particular, are they independent? (3 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\) and density function \(f\). We assume that the second derivative of \(f\) is bounded. We estimate \(f\) by \[ f_n(t)=\frac {F_n(t+h_n)-F_n(t-h_n)}{2h_n}, \] where \(F_n\) is the empirical distribution function and \(h_n\to 0\) as \(n\to \infty \).

(a) Show that \(\E f_n(t)\to f(t)\) as \(n\to \infty \). (5 points)

(b) Show that if \(nh_n\to \infty \), then \[ \frac {f_n(t)-\E f_n(t)}{(\operatorname {Var}(f_n(t)))^{1/2}} \] is asymptotically standard normal as \(n\to \infty \). (5 points)

Let \(X_1,\ldots ,X_n\) be a random sample from \(N(\theta ,\theta )\) (i.e., normal with mean and variance both equal to \(\theta \)), where \(\theta >0\).

(a) Give three pivotal quantities (or pivots) involving the entire sample, which respectively have a standard normal distribution, a \(t\)-distribution, and a chi-squared distribution, and indicate the numbers of degrees of freedom. (5 points)

(b) Use the normal pivotal quantity in part (a) to obtain a \(100(1-\alpha )\) percent confidence interval for \(\theta \). (5 points)

Suppose \(X_1,\ldots ,X_n\) is an i.i.d. sample from a normal distribution with mean \(\mu \) and variance 1.

(a) Find the UMVU estimator for \(\mu \). (Prove the optimality criterion.)

(b) Put a \(N(\theta ,\tau ^2)\) prior on \(\mu \) and find the minimax estimator of \(\mu \).

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.

Construct a \((1-\alpha )\) two-sided confidence interval for the correlation coefficient of a bivariate normal distribution. (HINT. This is an exponential family.)

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.

Construct a \((1-\alpha )\) two-sided confidence interval for the correlation coefficient of a bivariate normal distribution. (HINT. This is an exponential family.)

Let \(U_1,U_2,\ldots ,U_n\) be independent identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\le U_{2,n}\le \cdots \le U_{n,n}\) denote the order statistics. Let \(X_1,X_2,\ldots ,X_{n+1}\) be independent identically distributed exponential random variables with \(\mathbb {E} X_1=1\). Define \(S(i)=X_1+\cdots +X_i\).

(a) Prove that the random vectors \[ \{U_{1,n},\ldots ,U_{n,n}\} \quad \text {and}\quad \left \{\frac {S(1)}{S(n+1)},\ldots ,\frac {S(n)}{S(n+1)}\right \} \] have the same distribution.

(b) Compute the asymptotic distribution of \(n(U_{i+3,n}-U_{i,n})\), as \(n\to \infty \) when \(i\) is fixed.