Covariance and Independence

(20 points) Let \(Z = (Z_{1}, Z_{2}, Z_{3})^{\top } \sim \mathcal {N}(0, I_{3})\) and define the random vector \[ X = \begin {pmatrix} X_{1} \\ X_{2} \end {pmatrix} = \begin {pmatrix} Z_{1} + Z_{2} \\ Z_{2} + Z_{3} \end {pmatrix}. \]

1. Compute the mean vector \(\mathbb {E}[X]\) and the covariance matrix \(\Sigma = \operatorname {Cov}[X]\).
2. Does \(X\) have a probability density function on \(\mathbb {R}^{2}\)? Briefly justify your answer.
3. Does the joint moment generating function satisfy \[ M_{X}(t_{1}, t_{2}) = M_{X}(t_{1}, 0)M_{X}(0, t_{2})? \] Explain your reasoning.
4. Let \(Q = \|X\|^{2}\). Identify a symmetric matrix \(A\) such that \(Q = Z^{\top }AZ\) and compute \(\mathbb {E}[Q]\).

Let \(\epsilon _{1}, \ldots , \epsilon _{n}\) and \(\eta _{1}, \ldots , \eta _{m}\) be independent normal random variables with \(\mathbb {E}\epsilon _{i} = \mathbb {E}\eta _{j} = 0\) and \(\mathbb {E}\epsilon _{i}^{2} = 1, \mathbb {E}\eta _{j}^{2} = 4\). We have the models

where the \(x_{i}, y_{i}\) are given and non-random.

1. Find the maximum likelihood estimator for \((\beta _{1}, \beta _{2}, \beta _{3} )\).
2. Compute the distribution of the estimator.

Let \((X_{1}, X_{2})\) be a bivariate normal with \(\mathbb {E}X_{1} = \mathbb {E}X_{2} = 0\) and covariance matrix \[ \begin {bmatrix} 2 & \rho \\ \rho & 2 \end {bmatrix}. \] Determine the \(\rho \) such that \(2X_{1} + X_{2}\) and \(X_{1} + 3X_{2}\) are independent.

Consider the linear model

where the \(\epsilon _{i}\) are independent normals with \(\mathbb {E}\epsilon _{i} = 0\) and \(\mathbb {E}\epsilon _{i}^{2} = i\sigma ^{2}\). Here \(\beta \in \mathbb {R}^{d}\) and \(\sigma ^{2} > 0\) are unknown parameters, while the \(\mathbf {x}_{i} \in \mathbb {R}^{d}\) are all known and deterministic. Compute the maximum likelihood estimator of \((\beta , \sigma ^{2} )\).

Let \((X_{1}, X_{2})\) be independent normal random variables with \(\mathbb {E}X_{1} = \mathbb {E}X_{2} = 0\) and \(\mathbb {E}X_{1}^{2} =\) \(1, \mathbb {E}X_{2}^{2} = 4\). Define \(Y_{1} = X_{1} + X_{2}\) and \(Y_{2} = X_{1} + 2X_{2}\) and let \(\mathbf {Y} = (Y_{1}, Y_{2})^{\top }\).

1. Compute the covariance matrix of \(\mathbf {Y}\).
2. Find a matrix \(\mathbf {D}\) such that \(\mathbf {Y}^{\top }\mathbf {D}\mathbf {Y}\) has a chi-squared distribution.

Let \(U_1,U_2,\ldots ,U_n\) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) be the order statistics. Show that \(nU_{3,n}\) converges in distribution and compute the limit.

Let \(X,Y\) be independent random variables with respective densities \[ f(t)= \begin {cases} 0, & \text {if } t\notin [-2,3],\\ \dfrac {1}{5}, & \text {if } t\in [-2,3], \end {cases} \] and \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-1,5],\\ \dfrac {1}{6}, & \text {if } t\in [-1,5]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X\) and \(Y\) be independent normally distributed random variables with \(\mathbb {E} X=\mu _1\), \(\operatorname {Var} X=\sigma _1^2\), \(\mathbb {E} Y=\mu _2\) and \(\operatorname {Var} Y=\sigma _2^2\). Show that \(X+Y\) and \(X-Y\) are independent if and only if \(\sigma _1^2=\sigma _2^2\).

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed exponential\((\lambda )\) random variables, meaning that their pdf is \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ \dfrac {1}{\lambda }e^{-t/\lambda }, & \text {if } 0\leq t<\infty . \end {cases} \] Find the uniformly minimum variance unbiased estimator for \(\tau =\lambda ^5\).

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<\theta ,\\ \dfrac {2\theta }{(\theta +t)^2}, & \text {if } \theta \leq t<\infty , \end {cases} \] for \(\theta >0\) a parameter. Find the maximum likelihood estimator for \(\theta \).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ h(t;\theta )=\frac {3t^2}{\theta ^3}1\{0\leq t\leq \theta \}, \] where \(\theta >0\) is a parameter. Find a \(100(1-\alpha )\%\), two-sided, equal-tailed confidence interval for \(\theta \).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed \(N(\mu ,\sigma ^2)\), where \(\mu ,\sigma ^2\) are both unknown. We wish to test \(H_0:\sigma >\sigma _0\) against \(H_a:\sigma \leq \sigma _0\). We reject \(H_0\) if \(S^2<c\), where \[ S^2=\frac {1}{n-1}\sum _{i=1}^n (X_i-\bar X)^2, \qquad \bar X=\frac {1}{n}\sum _{i=1}^n X_i. \] Find \(c\) such that the size of the test is \(\alpha \).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed with density function \[ h(t;\theta )=\theta t^{\theta -1}1\{0\leq t\leq 1\} \] for \(\theta >0\) unknown. Use the Neyman-Pearson Lemma to find the uniformly most powerful test of size \(\alpha \) for \(H_0:\theta =\theta _0\) versus \(H_a:\theta =\theta _1\), where \(\theta _1>\theta _0\).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed random variables. The density function of \(X_i\) is \[ g(t;\theta _i)=\theta _i t^{-\theta _i-1}1\{t\geq 1\}. \] Here \(\theta _i>0\) is an unknown parameter. We wish to test \[ H_0:\theta _1=\theta _2=\cdots =\theta _n \] against the alternative that \(H_0\) is not true. Find a test using the generalized likelihood ratio.

Let \(X_1,\ldots ,X_n\) be independent and identically distributed random variables with (unknown) density function \(F\). Let \[ F_0(t)=(1-e^{-t})1\{t>0\}. \] We wish to test \(F(t)=F_0(t)\) for all \(t\). Explain the application of the \(\chi ^2\) goodness-of-fit test.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\theta }e^{-t/\theta }, & \text {if } t\geq 0. \end {cases} \] Let \(\alpha >0\) and \[ \tau =\frac {1}{\theta ^\alpha }. \]

(a) How big \(n\) should be (as a function of \(\alpha \)) so that \(\tau \) has uniformly minimum variance unbiased estimator?

(b) Find the uniformly minimum variance unbiased estimator for \(\tau \) (you need to justify that your estimator has the required property).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {5t^4}{2\theta ^5}, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find the maximum likelihood estimator for \(\theta \) and compute its bias.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {10t^9}{2\theta ^{10}}, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find a moment estimator for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t\theta )=\frac {1}{2}e^{-|t-\theta |}, \qquad -\infty <t<\infty . \] The parameter \(\theta \) could be any real number. Find all maximum likelihood estimators for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables that have the uniform distribution on \([0,1]\). Let \(X_{1,n}\leq X_{2,n}\leq \cdots \leq X_{n,n}\) be the order statistics. Let \(k\geq 1\) be a fixed integer. Show that \[ Y_n=nX_{k,n} \] converges in distribution and compute the limiting distribution.

Let \(X\) and \(Y\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\theta }e^{-t/\theta }, & \text {if } t\geq 0. \end {cases} \] Compute the density function of \[ Z=\frac {X}{X+Y}. \]

Let \(X_1,\ldots ,X_n\sim N(\mu _1,1)\) and \(Y_1,\ldots ,Y_m\sim N(\mu _2,1)\), where all random variables are independent. Derive the likelihood ratio test for \[ H_0:\mu _1^2=\mu _2^2 \quad \text {vs.}H_a:\mu _1^2\ne \mu _2^2. \] What is the distribution of the test statistic?

Let \(X_1,X_2,\ldots \), and \(X\) be real-valued random variables defined on a common probability space. Note they do NOT have to be independent or identically distributed. Prove that \[ \sum _{n=1}^{\infty }\mathbb {E}[|X_n-X|]<\infty \Longrightarrow X_n\to X \quad \text {almost surely.} \]

Let \(A_1,A_2,\ldots \), be a sequence of events (not necessarily independent) such that \[ \sum _{n=1}^{\infty }\mathbb {P}(A_n)=\infty \qquad \text {and}\qquad \mathbb {P}(A_n\cap A_m)\leq \mathbb {P}(A_n)\mathbb {P}(A_m) \quad \text {for } m\ne n. \] Prove that \(\mathbb {P}(A_n\text { i.o.})=1\) or provide a counterexample. Hint: Consider the mean and variance of \[ \sum _{i=1}^{n}\mathbf {1}_{A_i}. \]

Fix numbers \(p_x,q_x\in (0,1)\), \(x\in \mathbb {N}\), such that \(p_x+q_x\leq 1\). Let \(\{Z_{n,x}:n,x\in \mathbb {N}\}\) be independent random variables that take values in \(\{-1,0,1\}\), with common distribution \[ \mathbb {P}(Z_{n,x}=1)=p_x,\qquad \mathbb {P}(Z_{n,x}=-1)=q_x,\qquad \mathbb {P}(Z_{n,x}=0)=1-p_x-q_x. \] Let \(X_0=1\) and for \(n\in \mathbb {Z}_+\) define inductively \(X_{n+1}=X_n+Z_{n,X_n}\) if \(X_n\in \mathbb {N}\) and \(X_{n+1}=0\) if \(X_n=0\). In particular, \(X_n\in \mathbb {Z}_+\) almost surely. These \(X_n\) satisfy \[ \begin {aligned} \mathbb {P}(X_{n+1}=x+1\mid X_n=x)&=p_x,\\ \mathbb {P}(X_{n+1}=x-1\mid X_n=x)&=q_x,\\ \mathbb {P}(X_{n+1}=x\mid X_n=x)&=1-p_x-q_x, \end {aligned} \] if \(x\in \mathbb {N}\) and \(\mathbb {P}(X_{n+1}=x\mid X_n=x)=1\) if \(x=0\). The process \(X_n\) is called a birth and death process, absorbed at 0. Now define the function \(\phi :\mathbb {Z}_+\to \mathbb {R}\) by \(\phi (0)=0\), \(\phi (1)=1\), and for an integer \(x\geq 2\), \[ \phi (x)=1+\sum _{i=1}^{x-1}\prod _{j=1}^{i}\frac {q_j}{p_j}. \] Prove that \(M_n=\phi (X_n)\) is a martingale in the natural filtration \(\mathcal {F}_n=\sigma (X_0,\ldots ,X_n)\).

Assume that \(A_i\), \(i=1,2,\ldots \), are independent events. Prove that \[ \frac {1}{n}\sum _{i=1}^{n}\left (\mathbf {1}_{A_i}-\mathbb {P}(A_i)\right ) \] converges to zero in probability as \(n\to \infty \), where where \(\mathbf {1}_{A_i}\) is the indicator function of the event \(A_i\).

Let \(X_k\) have characteristic function \(\phi _k\) with \(X_1,X_2,\ldots \), independent random variables. Show that \(\sum _{k=1}^{n}X_k\) converges almost surely \(\Longleftrightarrow \) there exists a neighborhood \(U\) of 0 and a function \(h\) such that \[ \prod _{k=1}^{n}\phi _k(u) \xrightarrow [n\to \infty ]{} h(u)\ne 0, \] for all \(u\in U\). Hint: For the \(\Leftarrow \) direction consider the characteristic function of the partial sums \(\sum _{k=m}^{n}X_k\).

Let \(X\) and \(Y\) be independent \(N(0,1)\) random variables, and let \(Z=X+Y\).

1. Show that \(\mathbb {E}[Z\mid X>0,Y>0]=2\sqrt {2/\pi }\).
2. Find the distribution and the density of \(Z\) given that \(X>0\) and \(Y>0\). (You can use the CDF of a standard normal, \(\Phi (x)=\mathbb {P}(X\leq x)\), in your expression.)

Let \(X_n\) be a sequence of independent random variables with \(\mathbb {P}(X_n=1)=1/n=1-\mathbb {P}(X_n=0)\). Prove that \[ \left (\sum _{i=1}^{n}X_i-\log n\right )/\sqrt {\log n} \] converges weakly to a standard normal.

Fix \(\alpha \in (0,1)\). Let \(X_n\) be a sequence of independent random variables with \(\mathbb {P}(X_n=n)=\mathbb {P}(X_n=-n)=n^{-\alpha }/2\) and \(\mathbb {P}(X_n=0)=1-n^{-\alpha }\). Prove that \((X_1+\cdots +X_n)/n^{(3-\alpha )/2}\) converges weakly to a centered normal.

Let \(X\) and \(Y\) be two standard normal random variables (defined on a common probability space and having positive variances).

1. Prove that they are independent if and only if they are jointly normal (i.e. \((X,Y)\) is a two-dimensional normal random variable) and uncorrelated.
2. Give an example of two standard normal random variables that are uncorrelated but not jointly normal.

Let \(X_1,X_2,\ldots ,X_n\) be iid \(N(\mu _1,\sigma _1^2)\) and \(Y_1,Y_2,\ldots ,Y_m\) be iid \(N(\mu _2,\sigma _2^2)\). The \(X\) and the \(Y\) are also independent of each other. Assume that \(\sigma _1^2,\sigma _2^2\) are known while \(\mu _1,\mu _2\) are unknown. We wish to test \(H_0:\mu _1=\mu _2\) against \(H_A:\mu _1\ne \mu _2\). Find a test using the likelihood method. Compute the distribution of the likelihood ratio test.

Let \(X_1\) be \(N(\mu _1,\sigma _1^2)\) and \(X_2\) be \(N(\mu _2,\sigma _2^2)\) and assume \(X_1\) and \(X_2\) are independent. Show that \(Y_1=X_1+X_2\) and \(Y_2=X_1-X_2\) are independent if and only if \(\sigma _1=\sigma _2\).

Let \(X_1,X_2,\ldots \) be independent random variables, and suppose that \(p_i:=\mathbb {P}\{X_i=0\}\) satisfies \(0<p_i<1\) for every \(i=1,2,\ldots \). For every \(n\geq 1\) let \[ N_n:=\sum _{i=1}^{n}\mathbf {1}_{\{X_i=0\}} \] denote the number of times the finite random sequence \(X_1,\ldots ,X_n\) enters zero. Prove that \[ \frac {N_n}{\sum _{i=1}^{n}p_i}\to 1 \qquad \text {in probability as }n\to \infty . \]

Suppose \(X\) and \(Y\) are two independent, real-valued random variables, and suppose that \(\mathbb {P}\{X\in A\}=0\) for all Borel sets \(A\) of zero Lebesgue measure. Prove, carefully, that there exists a measurable function \(f:\mathbb {R}\to \mathbb {R}\) such that \[ \mathbb {P}\{X+Y\in B\}=\int _B f(x)\,dx \qquad \text {for all measurable sets }B\subseteq \mathbb {R}. \]

Fix \(0<\lambda <\rho \) and let \(X\), \(Y\), and \(Z\) be three independent Gamma distributed random variables with common scale parameter 1 and shape parameters \(\lambda \), \(\rho -\lambda \), and \(\rho \), respectively. (A Gamma random variable with scale parameter 1 and shape parameter \(\lambda \) has pdf \(\frac {1}{\Gamma (\lambda )}x^{\lambda -1}e^{-x}\mathbf {1}_{\{x>0\}}\).) Let \[ U=\frac {ZX}{X+Y},\qquad V=\frac {ZY}{X+Y},\qquad \text {and}\qquad W=X+Y. \] Prove that random vector \((U,V,W)\) has the same distribution as \((X,Y,Z)\). (It may be useful to note that \(Z=U+V\).)

If \(X_1,X_2,\ldots \) are independent standard normal random variables, then find non-random sequences \(a_n,b_n\to \infty \) such that \(a_n\min _{1\leq i\leq n}X_i+b_n\) converges weakly. Identify the limiting distribution.

Suppose \(X_1,X_2,\ldots \) are independent mean-zero random variables that satisfy the condition \[ \sum _{i=1}^{\infty }\operatorname {Var}(X_i)<\infty . \]

1. Prove that \(\sum _{n=1}^{\infty }X_n\) exists a.s. and is finite a.s.
2. Use part (a) to prove that if \(\varepsilon _1,\varepsilon _2,\ldots \) are i.i.d. with \(\mathbb {P}\{\varepsilon _1=\pm 1\}=1/2\), then the random harmonic series \(\sum _{i=1}^{\infty }(\varepsilon _i/i)\) converges a.s. and is finite a.s.
3. Is \(\sum _{i=1}^{\infty }(\varepsilon _i/i)\) in \(L^1(\mathbb {P})\)?

Let \(X_1\sim N(0,4\theta )\), \(X_2\sim N(0,9\theta )\), \(X_3\sim N(0,4\theta ^2)\), \(X_4\sim N(0,12\theta ^2)\), and assume that all \(X\)s are independent of each other. Using all four \(X\) variables and the \(t\) distribution, compute a \(100(1-\alpha )\%\) confidence interval for \(\theta \).

Let \(X_1\) and \(X_2\) be two independent random variables. Suppose that \(X_1\) is \(\chi ^2(r_1)\) and \(X_1+X_2\) is \(\chi ^2(r)\), where \(r_1,r\) are the respective degrees of freedom and \(r_1<r\). Show that \(X_2\) is \(\chi ^2(r-r_1)\).

Let \(Y_1,Y_2,\ldots ,Y_n\) be independent random variables, each with the \(N(\beta x_i,\gamma ^2x_i^2)\) distribution, where the numbers \(x_1,x_2,\ldots ,x_n\) are known, not all are equal, and none of them is zero. Suppose \(\beta \) and \(\gamma \) are unknown. Find the maximum likelihood estimators of \(\beta \) and \(\gamma ^2\).

Suppose that an engineer wishes to compare the number of complaints per week filed by union stewards for two different shifts at a manufacturing plant. It is known that the number of complaints per week on the \(i\)th shift has a Poisson distribution with mean \(\theta _i\) for \(i=1,2\). One hundred independent observations on the number of complaints gave means \(\bar x=20\) for shift 1 and \(\bar y=22\) for shift 2. Using this data test \(H_0:\theta _1=\theta _2\) versus \(H_1:\theta _1\ne \theta _2\) by the likelihood ratio method, with \(\alpha =0.01\). (Hint: First find the likelihood ratio test statistic for \(\lambda \) and then use the distribution of \(-2\log \lambda \).)

Construct two uncorrelated random variables that are not independent.

Suppose \(X_1,N_1,X_2,N_2,\ldots \) are independent random variables such that the \(X\)’s are Bernoulli\((1/2)\)—that is, \(\mathbb {P}\{X_i=0\}=\mathbb {P}\{X_i=1\}=1/2\) for all \(i\geq 1\)—and each \(N_n\) is Poisson\((n)\); that is, \[ \mathbb {P}\{N_n=k\}=\frac {n^ke^{-n}}{k!} \qquad \text {for all integers }n\geq 1\text { and }k\geq 0. \] Find non-random sequences \(\{a_n\}_{n=1}^{\infty }\) and \(\{b_n\}_{n=1}^{\infty }\) such that, as \(n\to \infty \), \[ \frac {\sum _{i=1}^{N_n}X_i-a_n}{b_n}\Rightarrow \operatorname {N}(0,1). \]

Let \(X_1\) and \(X_2\) be independent with joint pdf \(f(x)=x^{-2}\) for \(1\leq x<\infty \) and 0 otherwise.

(a) Find the joint pdf of \(U=X_1X_2\) and \(V=X_2\).

(b) Find the marginal pdf of \(U\).

The distance in feet by which a parachutist misses a target is \(D=\sqrt {X_1^2+X_2^2}\), where \(X_1\) and \(X_2\) are independent with \(X_i\sim N(0,25)\). Find \(\Prob (D\leq 12.25)\) (you can write down an integral for it, you don’t need to evaluate it).

Let \(U_0,U_1,\ldots \) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(N\) be a Poisson random variable with \(\mathbb {E}[N]=1\), independent of \(U_0,U_1,\ldots \). Compute \(\mathbb {E}[Y]\), where \(Y=\max _{0\leq k\leq N} U_k\).

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables uniform on \([0,1]\) and let \(\alpha >0\). Show that there are numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n=\frac {\sum _{k=1}^{n}k^{\alpha }X_k-a_n}{b_n} \] converges in distribution to a standard normal random variable.

Let \(\{e_i,-\infty <i<\infty \}\) be independent and identically distributed random variables with \(\mathbb {E}[e_1]=0\) and \(\mathbb {E}[e_1^2]=\sigma ^2\) and \(\rho \) be a real number satisfying \(|\rho |<1\). Show that \[ Y=\sum _{i=0}^{\infty }\rho ^i e_i \] is finite with probability one. Compute \(\mathbb {E}[Y]\) and \(\mathbb {E}[Y^2]\). (You need to justify your steps when you compute these two expected values.)

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables with characteristic function \(\varphi \). Let \(N\) be a random variable with \[ \mathbb {P}(N=k)=\frac {1}{2^k}, \qquad k=1,2,\ldots . \] We assume that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent.

(a) Compute the characteristic function of \(Y=X_1+X_2+\cdots +X_N\).

(b) Can you weaken the condition that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent and the formula obtained in (a) remains valid?

Let \(X_1,X_2,\ldots ,X_n\) be independent, random variables with density functions \[ h(t;\theta _i)= \begin {cases} \dfrac {1}{\theta _i}e^{-t/\theta _i}, & \text {if } 0\leq t<\infty ,\\ 0, & \text {if } -\infty <t<0. \end {cases} \] We wish to test \(H_0:\theta _1=\theta _2=\cdots =\theta _n\) against the alternative that \(H_0\) is not true.

(a) Compute \(\lambda \), the generalized likelihood ratio and explain the application of the generalized likelihood test.

(b) Provide a large sample approximation for \(-2\log \lambda \) under \(H_0\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t;\theta )= \begin {cases} \dfrac {1}{\theta }, & \text {if } 0\leq t\leq \theta ,\\ 0, & \text {if } t\notin [0,\theta ]. \end {cases} \] We wish to test \(H_0:\theta =1\) against the alternative \(H_A:\theta <1\). We reject \(H_0\) if \[ X_{n,n}=\max _{1\leq i\leq n}X_i\leq 1-\frac {2}{n}. \] (a) Compute the significance level.

(b) Compute the power function.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t;\sigma )=\frac {1}{(2\pi \sigma ^2)^{1/2}} e^{-(t-1)^2/(2\sigma ^2)}. \] We wish to test \(H_0:\sigma =\sigma _0\) against \(H_A:\sigma \ne \sigma _0\).

(a) Find a test, using the generalized likelihood ratio \(\lambda \).

(b) Provide a large sample approximation for \(-2\log \lambda \) under \(H_0\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t;\mu )=\frac {1}{(2\pi )^{1/2}}e^{-(t-\mu )^2/2}. \] We wish to test \(H_0:\mu \leq \mu _0\) against \(H_A:\mu >\mu _0\). Find the uniformly most powerful test of size \(\alpha \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed normal \(N(\mu _1,\sigma ^2)\) random variables. Let \(Y_1,Y_2,\ldots ,Y_m\) be independent identically distributed normal \(N(\mu _2,\sigma ^2)\) random variables. The two samples are independent. The parameters \(\mu _1,\mu _2,\sigma ^2\) are unknown. We wish to test \(H_0:\mu _1=\mu _2\). Show that the two sample t-test and the likelihood ratio test are equivalent.

Let \(X_1,X_2\) be independent and identically distributed exponential\((\lambda )\) random variables, i.e. the common density is \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Find the joint density function of \[ Y_1=X_1+X_2 \quad \text {and}\quad Y_2=\frac {X_1}{X_1+X_2}. \]

We assume that the number of people entering a store follows a Poisson distribution with parameter \(\lambda \). Every customer spends money in the store, independently of each other according to a uniform distribution on \([10,100]\). We observe \(X\), the total amount of money spent in the store on a given day.

(a) Estimate \(\lambda \) using the method of moments.

(b) Let assume that we have the stores income \(X_1,X_2,\ldots ,X_n\) for \(n\) days and \(X_1,X_2,\ldots ,X_n\) are independent and distributed as \(X\). What would be the moment estimator for \(\lambda \) based on \(X_1,X_2,\ldots ,X_n\)?

Let \(X\) and \(Y\) be independent random variables with densities \(f\) and \(g\), where \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{2}\exp (-t/2), & \text {if } t\geq 0, \end {cases} \] and \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-1,2],\\ \dfrac {1}{3}, & \text {if } t\in [-1,2]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Construct an equal tail \(1-\alpha \) confidence interval for \(\lambda \). Compute the expected value of the length of the confidence interval.

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Find the uniformly minimum variance unbiased estimator for \(\theta =1/\lambda \).

Let \(U_1,U_2,\ldots ,U_n\) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) denote the order statistics.

(a) Compute the limit distribution of \(Z=n(U_{3,n}-U_{2,n})\).

(b) Show that \(Z_1=n(U_{3,n}-U_{2,n})\) and \(Z_2=n(U_{4,n}-U_{3,n})\) are asymptotically independent.

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with distribution function \(F\). We wish to test that \(F\) is a normal distribution function. Provide at least two different methods on to test for the normality of \(F\).

A scale has two pans. The measurement given by the scale is the difference between the weights on pan 1 and pan 2 plus a random error. Thus, if a weight \(\mu _1\) is put on pan 1 and a weight \(\mu _2\) is put on pan 2, then the measurement is \(Y=\mu _1-\mu _2+\epsilon \). Suppose that \(\mathbb {E}[\epsilon ]=0\), \(\operatorname {Var}(\epsilon )=\sigma ^2\), and in repeated use of the scale observations \(Y_i\) are independent.

Suppose two objects, #1 and #2, have weights \(\beta _1\) and \(\beta _2\). Measurements are then taken as follows:

• Object #1 is put on pan 1 and nothing on pan 2
• Object #2 is put on pan 2 and nothing on pan 1
• Object #1 is put on pan 1 and object #2 is put on pan 2
• Objects #1 and #2 are both put on pan 1 and nothing on pan 2

(a) Let \(\mathbf {Y}=(Y_1,Y_2,Y_3,Y_4)^\prime \) be the vector of observations. Formulate this as a linear model.

(b) Find the least squares estimators of \(\beta _1\) and \(\beta _2\).

(c) Find the variances of the least squares estimators as well as the covariance between them.

Let \(X_1,\ldots ,X_{2n+1}\) be independent lognormal\((\mu ,\sigma ^2)\). That is, \(X_1=e^{Y_1},\ldots ,X_{2n+1}=e^{Y_{2n+1}}\) with \(Y_1,\ldots ,Y_{2n+1}\) being independent \(N(\mu ,\sigma ^2)\). Use a theorem to show that the sample median \(X_{n+1:2n+1}\) is asymptotically normal and find its asymptotic mean and asymptotic variance.

Let \(X_1\) and \(X_2\) be independent \(\operatorname {Exponential}(1)\) random variables (density \(e^{-x}\), \(x>0\)). Find the joint density of \(Y_1=X_1/X_2\) and \(Y_2=X_1+X_2\). Find the marginal densities of \(Y_1\) and \(Y_2\) as well.

\(X_1,X_2,\ldots ,X_n\) be independent identically distributed positive random variables. Show that \[ \frac {1}{n}\max _{1\leq k\leq n} X_k \to 0 \qquad \text {a.s.} \] if and only if \(\mathbb {E} X_1<\infty \).

Let \(X\) and \(Y\) be independent random variables. Show that if the distribution of \(X\) is absolutely continuous with respect to the Lebesgue measure, then the distribution of \(X+Y\) is also absolutely continuous with respect to the Lebesgue measure.

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables with \(\mathbb {E}\log (1+|X_1|)<\infty \). Show that \[ \sum _{i=1}^{\infty }\prod _{j=1}^{i}X_j \] is finite with probability one if and only if \(\mathbb {E}\log |X_1|<0\).

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables with distribution function \(F(x)=\mathbb {P}\{X_1\leq x\}\). Show that \[ \sup _{-\infty <x<\infty } |F_n(x)-F(x)| \to 0 \qquad \text {a.s.} \] as \(n\to \infty \), where \[ F_n(x)=\frac {1}{n}\sum _{i=1}^{n} I\{X_i\leq x\}. \]

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential(1) random variables. Let \(k\geq 1\). Show that \[ \mathbb {P}\{X_n>\phi _{\varepsilon }(n)\ \text {i.o.}\} = \begin {cases} 0, & \text {if }\varepsilon >0,\\ 1, & \text {if }\varepsilon \leq 0, \end {cases} \] where \[ \phi _{\varepsilon }(n)=\log n+\log _{(2)}(n)+\cdots + (1+\varepsilon )\log _{(k)}(n), \] and \(\log _{(1)}(n)=\log n\), \(\log _{(k)}(n)=\log (\log _{(k-1)}(n))\).

Let \(X_1,X_2,\ldots \) be independent identically distributed positive random variables with \(\mathbb {E} X_1=\mu \). Define \[ N(t)=\min \left \{k:\sum _{i=1}^{k} X_i>t\right \}. \] Show that \[ \lim _{t\to \infty }\frac {1}{t}N(t)=\frac {1}{\mu } \qquad \text {a.s.} \]

Let \(X\) and \(Y\) be independent standard normal random variables. Compute the density function of \(Z=X/Y\).

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables uniform on \([0,1]\). Show that for all \(\alpha >-1\) there are numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n= \frac {\sum _{1\leq k\leq n} k^{\alpha }X_k-a_n}{b_n} \] converges in distribution to a standard normal random variable.

Let \(X_1,X_2,\ldots \) be independent random variables with 2 finite moments, and \[ \sum _{j=1}^{\infty }\operatorname {Var}(X_j)<\infty . \] Prove that \[ \lim _{n\to \infty }\sum _{j=1}^{n}(X_j-\mathbb {E} X_j) \] exists a.s. and in \(L^2(\mathbb {P})\).

Let \(X_1,X_2,\ldots \) be independent, identically distributed random variables with the following common distribution: \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2k^2} \qquad \text {for } k=\pm 1,\pm 2,\ldots . \]

(a) Prove that \(\mathbb {E}\exp (itX_1)=1-(3|t|/\pi )+o(|t|)\) as \(t\to 0\). You may use, without proof, the fact that \[ \int _0^{\infty }(1-\cos \theta )\theta ^{-2}\,d\theta =\pi /2. \]

(b) Use the preceding in order to prove that \(n^{-1}(X_1+\cdots +X_n)\) converges weakly to a “Cauchy random variable \(Y\) with scale parameter \(3/\pi \).” That is, the characteristic function of \(Y\) is \[ \mathbb {E}\exp (itY)=\exp (-3|t|/\pi ) \qquad \text {for all }t\in \mathbb {R}. \] You may use, without proof, the fact that the latter is a characteristic function. This matter is the topic of Problem 8 below.

Suppose \(f:[0,\infty )\to \mathbb {R}\) is bounded and measurable, and \(f(x)=0\) for all \(x\geq 2\).

(a) Prove that if \(X_1,X_2,\ldots \) are independent, all distributed uniformly in the interval \((0,1)\), then \[ \frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+X_j}{n}\right ) \xrightarrow {\mathbb {P}} \int _0^{\infty } f(x)\,dx \qquad \text {as } n\to \infty . \]

(b) (Extra credit) Deduce from this the following version of Riemann sums for Lebesgue integrals [Chaterjee’s theorem]: There exists a sequence \(\{\delta _{j,n};\ 0\leq j\leq n,\ n\geq 1\}\) of real numbers in \((0,1)\) such that \[ \lim _{n\to \infty }\frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+\delta _{j,n}}{n}\right ) = \int _0^{\infty } f(x)\,dx. \]

For every random vector \((X,Y)\) in \(\mathbf {R}^2\) we can define a function \(\mathcal {Q}\) of two variables as follows: \[ \mathcal {Q}(a,b):=\Prob \{X>a,\ Y>b\}-\Prob \{X>a\}\Prob \{Y>b\}. \]

(a) Suppose \(f,g:\mathbf {R}\to \mathbf {R}\) are bounded and have integrable, bounded, and continuous derivatives. Then prove that \[ \E [f(X)g(Y)]-\E [f(X)]\E [g(Y)] = \iint _{\mathbf {R}^2}\mathcal {Q}(a,b)f'(a)g'(b)\,da\,db. \]

(b) Suppose \(\mathcal {Q}(a,b)\geq 0\) for all \(a,b\in \mathbf {R}\). Prove that if \(X\) and \(Y\) have two finite moments, then \(\operatorname {Cov}(X,Y)\geq 0\) and there exists a finite constant \(\alpha \), depending only on \(f\) and \(g\), such that \[ |\E [f(X)g(Y)]-\E [f(X)]\E [g(Y)]|\leq \alpha \operatorname {Cov}(X,Y). \]

[In other words, if \(\mathcal {Q}\geq 0\) and \(X\) and \(Y\) have a small covariance, then \(X\) and \(Y\) are almost independent.]

Let \(\{W(t)\}_{t\geq 0}\) be a standard Brownian motion, and define \[ U(t):=e^{-t/2}W(e^t)\qquad \text {for all }t\geq 0. \]

(a) Prove that \(\{U(t)\}_{t\geq 0}\) is Gaussian process; compute its mean and covariance functions.

(b) Prove that \[ \limsup _{t\to \infty }\frac {U(t)}{\sqrt {2\ln t}} = -\liminf _{t\to \infty }\frac {U(t)}{\sqrt {2\ln t}} =1 \qquad \text {a.s.} \]

Let \(X\) be an integer-valued random variable that is non-degenerate; this means that there exist at least two integers \(n,m\) such that \(\mathbb {P}\{X=n\}\) and \(\mathbb {P}\{X=m\}\) are both strictly positive. Prove that the set \[ \mathcal {V}:=\{t\in \mathbf {R}:|\mathbb {E}[e^{itX}]|=1\} \] has zero Lebesgue measure. [Hint: If \(t\in \mathcal {V}\) then \(Z:=e^{it(X-Y)}\) has mean one, where \(Y\) is an independent copy of \(X\). Proceed by proving that \(\mathbb {E}(|Z-1|^2)=0\).]

Let \(X_1,X_2,\ldots \) be independent identically distributed random variables with characteristic function \(\varphi \). Let \(N\) be a random variable with distribution \[ \mathbb {P}\{N=k\}=\frac {1}{2^k},\qquad k=1,2,\ldots . \] It is assumed that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent.

(a) Compute the characteristic function of \(Y=X_1+\cdots X_N\).

(b) Can you weaken the condition that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent so that the formula obtained in part (a) remains true?

(a) Let \(\Phi \) and \(\phi \) be the standard normal distribution and density functions, respectively. Show that \[ \lim _{x\to \infty }\frac {1-\Phi (x)}{(1/x)\phi (x)}=1. \]

(b) Let \(X_1,X_2,\ldots \) be independent identically distributed standard normal random variables. Show that \[ \limsup _{n\to \infty }\frac {1}{\sqrt {\log n}}|X_n|=c \quad \text {almost surely} \] and compute the value of \(c\).

Let \(X_1,\ldots ,X_n\) be independent identically distributed random variables with \(\mathbb {E} X_1=\mu \) and \(0<\operatorname {Var}(X_1)=\sigma ^2<\infty \). Let \[ Y_n=\sum _{1\leq i<j\leq n}X_iX_j. \] Find numerical sequences \(a_n\) and \(b_n\) such that \((Y_n-a_n)/b_n\) has a non-degenerate limit distribution.

Let \(X_1,\ldots ,X_n\) be independent identically distributed Poisson random variables for each \(n\) with parameter \(\lambda _n\).

(a) Show that \(X_1+\cdots +X_n\) is asymptotically normal if and only if \(n\lambda _n\to \infty \).

(b) Can you weaken the condition that the X’s are identically distributed for each \(n\)?

Let \(X_1,X_2,\ldots \) be independent identically distributed random variables with characteristic function \(\varphi \). Let \(N\) be a random variable with distribution \[ \mathbb {P}\{N=k\}=\frac {1}{2^k},\qquad k=1,2,\ldots . \] It is assumed that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent.

(a) Compute the characteristic function of \(Y=X_1+\cdots X_N\).

(b) Can you weaken the condition that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent so that the formula obtained in part (a) remains true?

Let \(X_1,X_2,\ldots \) be independent identically distributed standard normal random variables. Show that \[ \limsup _{n\to \infty }\frac {1}{\sqrt {\log n}}|X_n|=c \quad \text {almost surely} \] and compute the value of \(c\).

Let \(X_1,\ldots ,X_n\) be independent identically distributed random variables with \(\mathbb {E} X_1=\mu \) and \(0<\operatorname {Var}(X_1)=\sigma ^2<\infty \). Let \[ Y_n=\sum _{1\leq i<j\leq n}X_iX_j. \] Find numerical sequences \(a_n\) and \(b_n\) such that \((Y_n-a_n)/b_n\) has a non-degenerate limit distribution.

Let \(X_1,\ldots ,X_n\) be independent identically distributed Poisson random variables for each \(n\) with parameter \(\lambda _n\).

(a) Show that \(X_1+\cdots +X_n\) is asymptotically normal if and only if \(n\lambda _n\to \infty \).

(b) Can you weaken the condition that the X’s are identically distributed for each \(n\)?

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with distribution function \[ F(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ t^3, & \text {if } 0\leq t\leq 1,\\ 1, & \text {if } t>0. \end {cases} \] Show that \[ Y_n=n^{1/3}X_{1,n} \] converges in distribution, where \(X_{1,n}=\min \{X_1,X_2,\ldots ,X_n\}\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ h(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {5}{2}\theta ^{-5}t^4, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find a moment estimator for \(\theta \).

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t,\theta )= \begin {cases} 0, & \text {if } -\infty <t<0,\\ \theta (t+1)^{-\theta -1}, & \text {if } 0\leq t<\infty , \end {cases} \] \(\theta >0\).

(a) Find the maximum likelihood estimator for \(\theta \).

(b) Is the estimator unbiased?

(c) Find the asymptotic variance of the maximum likelihood estimator for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ h(t,\theta )= \begin {cases} 0, & \text {if } t\notin [0,\theta ],\\ 2\theta ^{-2}t, & \text {if } 0\leq t\leq \theta . \end {cases} \] Find the uniformly minimum variance unbiased estimator for \(\theta \). Explain your answer. You need to prove directly that the sufficient statistic is also complete in this case.

Let \(X\) and \(Y\) be two independent random variables with density functions \[ f(t)= \begin {cases} 0, & \text {if } t\notin [0,4],\\ 1/4, & \text {if } 0\leq t\leq 4, \end {cases} \] and \[ h(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ 2e^{-2t}, & \text {if } 0\leq t<\infty . \end {cases} \] Compute the density of \(X-Y\).

Let \(X_1\) and \(X_2\) be independent random variables. The density function function of \(X_1\) is \[ f(t)= \begin {cases} 0, & \text {if } t\notin [0,1],\\ \dfrac {e^t}{e-1}, & \text {if } 0\leq t\leq 1. \end {cases} \] The distribution of \(X_2\) is \[ \mathbb {P}\{X_2=1\}=p \quad \text {and}\quad \mathbb {P}\{X_2=-1\}=q,\qquad p+q=1. \] Compute the moment generating function of \(X_1X_2\).

Let \(X_1,\ldots ,X_n\) denote an independent sample from an exponential distribution with [unknown] mean \(\theta >0\). What does the Neyman–Pearson lemma say about \(H_0:\theta =1\) versus \(H_a:\theta =2\)? Explain carefully, and identify explicitly the rejection region.

A sample of 400 people was asked their degree of support of a balanced budget and their degree of support of public education, with the following results:

Test the hypothesis of independence at \(\alpha =0.05\). (A chi-square table is provided.)

Suppose \(X_1,X_2,\ldots \) are independent, identically-distributed exponential random variables with mean \(\lambda >0\). Prove that \[ \max (X_1\ldots ,X_n)-\frac {1}{\lambda }\ln n\Rightarrow X, \] and compute \(\mathbb {P}\{X>x\}\) for all \(x>0\).

Give a rigorous proof that \(\mathbb {E}[XY]=\mathbb {E}[X]\mathbb {E}[Y]\) if \(X\) and \(Y\) are independent random variables belonging to \(L^1(\mathbb {P})\). In particular, show that \(XY\in L^1(\mathbb {P})\).

A random experiment has exactly three possible outcomes, referred to as outcomes 1, 2, and 3, with probabilities \(p_1>0\), \(p_2>0\), and \(p_3>0\), where \(p_1+p_2+p_3=1\). We consider a sequence of independent trials, at each of which the specified random experiment is performed. For \(i=1,2\), let \(N_i\) be the number of trials needed for outcome \(i\) to occur, and put \(N:=\min (N_1,N_2)\).

(a) Show that \(N\) is independent of \(1_{\{N_1<N_2\}}\).

(b) Evaluate \(\mathbb {E}[N_1\mid N_1<N_2]\).

(c) Roll a pair of dice until a total of 6 or 7 appears. Given that 6 appears before 7, what is the (conditional) expected number of rolls?

Let \(X_1,X_2,\ldots \) be independent with \[ \mathbb {P}(X_n=n^2-1)=n^{-2}=1-\mathbb {P}(X_n=-1),\qquad n\geq 1. \] Notice that \(\mathbb {E}[X_n]=0\) for all \(n\geq 1\), which might lead one to expect that \(S_n:=X_1+\cdots +X_n\) satisfies \(S_n/n\to 0\) a.s. Show that in fact \(S_n/n\to -1\) a.s.

Suppose \(X_1\) and \(X_2\) are independent random variables that satisfy the following three properties: (i) \(Y_1:=(X_1+X_2)/\sqrt {2}\) is standard normal; (ii) \(Y_2:=(X_1-X_2)/\sqrt {2}\) is standard normal; and (iii) \(Y_1\) and \(Y_2\) are independent. Prove that \(X_1\) and \(X_2\) are normally distributed. Can you compute their respective means and variances?

Let \(X\) and \(Y\) be independent, standard-normal random variables. Find the distribution of \(X/Y\)?

Let \(X_1\) and \(X_2\) be independent, each with probability density function \(f(x)=1/x^2\), \(x>1\). Find the joint probability density function of \(Y_1=X_1X_2\) and \(Y_2=X_1/X_2\). It is important to state where your formula is valid. (If you have time, check that your joint pdf integrates to 1; if it doesn’t, find the error.)

Let \(X_1,\ldots ,X_n\) be independent \(\operatorname {LogNormal}(\mu ,\sigma ^2)\) (i.e., \(X_1=e^{Y_1},\ldots ,X_n=e^{Y_n}\) with \(Y_1,\ldots ,Y_n\) being independent \(N(\mu ,\sigma ^2)\)). Argue that the sample median is asymptotically normal and find the asymptotic mean and variance.

Consider the simple linear regression model \(Y_i=\beta _0+\beta _1x_i+\epsilon _i\) (\(i=1,2,\ldots ,n\)). Assume independent \(N(0,\sigma ^2)\) errors.

(a) Find the least squares estimators \(\hat \beta _0\) of \(\hat \beta _1\) of \(\beta _0\) and \(\beta _1\).

(b) Calculate the covariance matrix of \((\hat \beta _0,\hat \beta _1)\).

Let \(X_1,X_2,\ldots \) be i.i.d. with mean \(\mu \) and variance \(\sigma ^2\), and put \(S_0:=0\) and \(S_n:=X_1+\cdots +X_n\) for each \(n\geq 1\). Let \(N\) be a nonnegative-integer-valued random variable independent of \(X_1,X_2,\ldots \). Find the mean and variance of \(S_N\) by conditioning on \(N\).

For \(n=1,2,\ldots \), let \(X_n\) have density \[ f_n(x):=\frac {n}{\pi (1+n^2x^2)},\qquad -\infty <x<\infty . \] (a) Does \(X_n\) converge to 0 in probability?

(b) Does \(X_n\) converge to 0 in \(L^1(\mathbb {P})\)?

(c) If \(X_1,X_2,\ldots \) are independent, does \(X_n\) converge to 0 a.s.?

Let \(X_1,X_2,X_3,Y_1,Y_2,Y_3\) be independent \(N(0,1)\) random variables. Let \(P_1\), \(P_2\), and \(P_3\) be the points in the plane with coordinates \((X_1,Y_1)\), \((X_2,Y_2)\), and \((X_3,Y_3)\). Let \(M\) be the midpoint of the segment \(P_1P_2\) and let \(r\) be half its length. Show that the probability that the point \(P_3\) lies within the circle of radius \(r\) centered at \(M\) is equal to \(1/4\).

Choose and fix a sequence of real numbers \(a_1,a_2,\ldots \) such that: (i) \(0<a_k<1\) for all \(k\geq 1\); and (ii) \(\sum _{k=1}^{\infty }a_k^2=\infty \). Let \(X_1,X_2,\ldots \) be independent random variables, where each \(X_k\) is distributed uniformly on \((-a_k,a_k)\). Prove that \(n^{-1/2}(X_1+\cdots +X_n)\) converges weakly to \(N(0,\sigma ^2)\) and compute \(\sigma \).

Suppose \(f\) is a probability density function on \(\mathbf {R}\), and define for all \(x,y\in \mathbf {R}\), \[ g(x,y):= \begin {cases} 2f(x)f(y), & \text {if } x\leq y,\\ 0, & \text {otherwise.} \end {cases} \] (a) Prove that \(g\) is the probability density function of some random variable \((X,Y)\) which takes values in \(\mathbf {R}^2\).

(b) Are \(X\) and \(Y\) independent? Prove or disprove.

Suppose \(X_1,X_2,\ldots \) are independent, identically-distributed random variables such that \[ \mathbb {P}\{X_1>x\}= \begin {cases} x^{-\alpha }, & \text {if } x\geq 1,\\ 1, & \text {if } x<1, \end {cases} \] where \(\alpha >0\) is fixed. Prove that \[ n^{-1/\alpha }\max (X_1,\ldots ,X_n)\Rightarrow X, \] and compute \(\mathbb {P}\{X>x\}\) for all \(x\in \mathbf {R}\).

Let \(\alpha ,\lambda \) be fixed, strictly positive, and finite. The Gamma\((\alpha ,\lambda )\) density function is \[ f(x):= \begin {cases} \dfrac {\lambda ^\alpha }{\Gamma (\alpha )}x^{\alpha -1}e^{-\lambda x}, & \text {if } x>0,\\ 0, & \text {if } x\leq 0, \end {cases} \] where \(\Gamma (\alpha ):=\int _0^\infty t^{\alpha -1}e^{-t}\,dt\). This defines a probability density function.

(a) Compute the characteristic function of \(f\).

(b) Suppose \(X_1,\ldots ,X_n\) are independent with common density Gamma\((1,\lambda )\). Prove that \(S_n:=X_1+\cdots +X_n\) has a Gamma density; also compute its parameters.

(c) Compute \(\mathbb {P}\{S_n<k<S_{n+1}\}\) for all integers \(k\geq 1\).

Let \(U_1,U_2,\ldots ,U_n\) be independent random variables uniform on \([0,1]\) and let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) be the corresponding order statistics.

a. Compute the asymptotic distribution of \(nU_{2,n}\). (5 points)

b. Show that \(nU_{1,n}\) and \(n(1-U_{n,n})\) are asymptotically independent. (5 points)

Let \(y_i=\alpha x_i+\epsilon _i\), \(1\leq i\leq n\), where \(\epsilon _1,\epsilon _2,\ldots ,\epsilon _n\) are independent identically distributed normal \(N(0,\sigma ^2)\) random variables.

a. Determine the maximum likelihood estimators for \(\alpha \) and \(\sigma ^2\). (5 points)

b. Compute the distribution of the maximum likelihood estimator for \(\alpha \). (5 points)

c. Compute the distribution of the maximum likelihood estimator for \(\sigma ^2\). (5 points)

Let \(X\) and \(Y\) be two independent random variables with distribution functions \(F\) and \(G\) and density functions \(f\) and \(g\).

a. Determine the distribution function of \(XY\). (5 points)

b. Does \(XY\) always have a density function? (5 points)

c. Assuming that \(XY\) has a density function, compute it. (5 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ f(t;\theta )= \begin {cases} 0, & \text {if } t<\theta ,\\ e^{-(t-\theta )}, & \text {if } t>\theta . \end {cases} \]

a. Find a moment estimator for \(\theta \). (5 points)

b. Find the maximum likelihood estimator for \(\theta \). (5 points)

c. Determine the asymptotic efficiency between the two estimators. (5 points)

The number of customers entering a store on a given day is a Poisson random variable with parameter \(\theta \). The money spent by a customer is uniformly distributed on \([0,\eta ]\). The number of customers and the money spent in the store are independent. We observe \(Z_1,Z_2,\ldots ,Z_n\), the total spending (the money collected by the store) on \(n\) days. We can assume that \(Z_1,Z_2,\ldots ,Z_n\) are independent.

a. Provide estimators for \(\theta \) and \(\eta \) using the method of moments. (10 points)

b. Provide estimators for \(\theta \) and \(\eta \) using the likelihood method. (10 points)

Let \(X_1,\ldots ,X_n\) be independent identically distributed Poisson random variables with parameter \(\theta _1\). Let \(Y_1,\ldots ,Y_m\) be independent identically distributed Poisson random variables with parameter \(\theta _2\). We assume that the two samples are independent. We wish to test \(H_0:\theta _1=\theta _2\) against the alternative that \(H_0\) is not true.

a. Derive the likelihood ratio test. (5 points)

b. Provide a large sample approximation for the rejection region. (2 points)

Let \(X_1,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We assume that \(F\) is strictly increasing and continuous on its support. Let \(x_{1/2}\) denote the median.

a. Find a \(1-\alpha \) confidence interval for \(x_{1/2}\). (5 points)

b. Provide a large sample approximation for the confidence interval. (5 points)

Let \(X_1,\ldots ,X_N\) be independent random variables. The distibution of \(X_i\) is binomial \((n_i,p_i)\). We wish to test \(H_0:p_1=p_2=\cdots =p_N\) against the alternative that \(H_0\) is not true.

a. Derive the likelihood ratio test. (5 points)

b. Provide a large sample approximation for the rejection region. (2 points)

Let \(X_2,X_3,\ldots \) be independent random variables such that \[ \mathbb {P}(X_n=n)=\mathbb {P}(X_n=-n)=\frac {1}{2n\log n}, \qquad \mathbb {P}(X_n=0)=1-\frac {1}{n\log n}. \] Show that this sequence obeys the weak law but not the strong law of large numbers, in the sense that \(n^{-1}\sum _{i=1}^n X_i\) converges to 0 in probability but not almost surely.

Let \(\{X_n,n\geq 1\}\) be independent \(N(0,1)\) random variables. Show that \[ \Prob \left (\limsup \frac {|X_n|}{\sqrt {\log n}}=\sqrt {2}\right )=1. \] Hint: \(1-\Phi (x)\) is asymptotic to \(\phi (x)/x\) as \(x\to \infty \).

Let \(X\) and \(Y\) be independent exponential random variables with parameters \(\lambda \) and \(\mu \). (For example, \(\mathbb {P}(X>x)=e^{-\lambda x}\) for \(x>0\).)

(a) Show that \(Z:=\min (X,Y)\) is independent of the event \(\{X<Y\}\).

(b) Find \(\mathbb {P}(X=Z)\).

(c) Find the distribution of \(U:=(X-Y)^+=\max (X-Y,0)\).

(d) Find the distribution of \(V:=\max (X,Y)-\min (X,Y)\).

Suppose \(W:=(X,Y)\) is a two-dimensional random variable and \(QW^T\) is \(N(0,1)\) for all \(2\times 2\) rotation matrices \[ Q:= \begin {pmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end {pmatrix}. \] Prove that \(X\) and \(Y\) are independent and \(N(0,1)\).

Suppose \(X\) and \(Y\) are independent bounded random variables. Prove in complete detail the following assertions:

(a) \(\mathbb {E}(XY)=\mathbb {E}(X)\mathbb {E}(Y)\),

(b) \(\mathbb {E}(X\mid Y)=\mathbb {E}(X)\).

Let \(X\) and \(Y\) be independent standard normals and consider \((U,V)=(aX+bY,cX+dY)\). Assume that at least one of \(a\) and \(b\) is nonzero, and at least one of \(c\) and \(d\) is nonzero. (Otherwise either \(U\) or \(V\) would be 0 and the problem is not interesting.)

5a) (5 points) Show that if \(ad=bc\), then \(U\) and \(V\) cannot be independent random variables.

5b) (5 points) Assume \(ad\ne bc\) and find the joint pdf of \((U,V)\).

5c) (5 points) Find a necessary and sufficient condition for \(U\) and \(V\) to be independent.

Consider the pdf \(f(x;\theta )=\theta x^{-(\theta +1)}\) (\(x\geq 1\)). Let \(X_i\) be a random sample of size \(n\) with pdf \(f(x;\theta _1)\). Let \(Y_i\) be a random sample of size \(m\) with pdf \(f(x;\theta _2)\). The two sets of random variables are independent. We wish to test \[ H_0:\theta _1=\theta _2 \quad \text {against}\quad H_a:\theta _1\ne \theta _2. \]

6a) (5 points) If one considers just the \(X\)-data, find the MLE \(\hat \theta _1\).

6b) (5 points) Derive a formula for the critical region obtained using the generalized likelihood ratio method.

Let \(X_1,X_2,\ldots ,X_n\) and \(Y_1,Y_2,\ldots ,Y_m\) be two independent random samples. We assume that \(X_1,X_2,\ldots ,X_n\) are independent identically distributed normal \((\mu _1,\sigma ^2)\) random variables and \(Y_1,Y_2,\ldots ,Y_m\) are independent identically distributed normal \((\mu _2,\sigma ^2)\) random variables (\(\sigma ^2\) is unknown). We want to test \(H_0:\mu _1=\mu _2\) against the alternative \(H_A:\mu _1\ne \mu _2\).

Show that the likelihood ratio test and the two sample t-test are equivalent. (7 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We use \[ \hat f_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n} K\left (\frac {t-X_i}{h}\right ) \] to estimate \(f=F'\).

We assume that the following conditions are satisfied: \[ \int _{-1}^{1}K(u)\,du=1, \qquad K(t)=0,\quad \text {if } t\notin [-1,1]. \] \(F\) is three times differentiable and the derivatives are uniformly bounded on the real line \[ h=h(n)\to 0. \]

(i) Compute the bias of \(\hat f_n(t)\) for any \(t\) and any \(n\). (7 points)

(ii) Prove that \(\hat f_n(t)\) is asymptotically unbiased for any \(t\) as \(n\to \infty \). (7 points)

(iii) Compute the mean square error of \(\hat f_n(t)\) (7 points)

Let \(X_1\) and \(X_2\) be independent standard normal random variables. Find the necessary and sufficient condition for the independence of \(Y_1=a_1X_1+a_2X_2\) and \(Y_2=b_1X_1+b_2X_2\). (7 points)

Let Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \[ F(t)=\left (\frac {t}{\theta }\right )^2 \quad \text {if } 0\leq t\leq \theta . \]

(i) Find the maximum likelihood estimator for \(\theta \). (7 points)

(ii) Compute the limit distribution of the maximum likelihood estimator. (7 points)

Let Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with \(\E X_1=\mu \), \(\operatorname {Var} X_1=\sigma ^2\). We assume that \(\E X_4<\infty \). The random variable \[ \hat \sigma _n^2=\frac {1}{2n}\sum _{2\leq i\leq n}(X_i-X_{i-1})^2 \] is suggested to estimate \(\sigma ^2\).

(i) Establish some basic properties of \(\hat \sigma _n^2\). (7 points)

(ii) Is \(\hat \sigma _n^2\) a better estimator than the sample variance? Justify your answer. (7 points)

Let \(X\) and \(Y\) be independent random variables with density function \(f\) and \(g\). Assume that \(Y>0\). Show that the function \[ h(t)=\int _{0}^{\infty } z f(zt)g(z)\,dz \] can be chosen as the density of \(Z=X/Y\). (7 points)

Let \(\{X_n\}_{n=1}^{\infty }\) be a sequence of independent (but not identically distributed) random variables, such that for all \(n\ge 1\), \[ \mathbb {P}\{X_n=1\}=\mathbb {P}\{X_n=-1\}=\frac {1}{2}-\frac {1}{2n^2}, \qquad \mathbb {P}\{X_n=n\}=\mathbb {P}\{X_n=-n\}=\frac {1}{2n^2}. \] Prove that \(n^{-1/2}\sum _{i=1}^{n}X_i\Rightarrow N(0,\sigma ^2)\), and compute \(\sigma ^2\) explicitly.

Suppose \(X\) and \(Y\) are two independent standard normal random variables.

(a) Prove that for all twice continuously differentiable functions \(f,g:\mathbb {R}\to \mathbb {R}\), \[ \operatorname {Cov}(f(X),g(X))=\int _0^1 \E \!\left [f'(X)g'\!\left (sX+(1-s^2)^{1/2}Y\right )\right ]\,ds. \]

(Hint: Check it first for \(f(x):=\exp (itx)\) and \(g(x):=\exp (itx)\).)

(b) Conclude the following “Poincaré inequality,” due to J. Nash (1958): \[ \Var (f(X))\le \E \!\left [(f'(X))^2\right ]. \]

Let \(X\) and \(Y\) be two standard-normal random variables.

(a) Prove that if \((X,Y)\) is Gaussian then \(X\) and \(Y\) are independent.

(b) Construct an example wherein \((X,Y)\) is not a (two-dimensional) Gaussian random variable.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ f(t,\theta _1)= \begin {cases} 0, & \text {if } t\notin [1,\infty ),\\ \theta _1 t^{-\theta _1-1}, & \text {if } 1\leq t<\infty . \end {cases} \] Let \(Y_1,\ldots ,Y_m\) be independent, identically distributed random variables with density function \[ f(t,\theta _2)= \begin {cases} 0, & \text {if } t\notin [1,\infty ),\\ \theta _2 t^{-\theta _2-1}, & \text {if } 1\leq t<\infty . \end {cases} \] We wish to test \(H_0:\theta _1=\theta _2\) against the alternative that \(\theta _1\neq \theta _2\). The two samples are independent.

(a) Find a test using the likelihood ratio.

(b) Provide a large sample approximation for the likelihood ratio test.

Let \(X_1,\ldots ,X_n\) be independent random variables with density functions \[ f(t,\theta _i)= \begin {cases} 0, & \text {if } t\notin [0,\infty ),\\ \dfrac {1}{\theta _i}e^{-t/\theta _i}, & \text {if } 0\leq t<\infty . \end {cases} \] We wish to test \(H_0:\theta _1=\theta _2=\cdots =\theta _n\) against the alternative that \(H_0\) is not true.

(a) Find a test using the likelihood ratio.

(b) Provide a large sample approximation for the likelihood ratio test.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ g(t;\theta )= \begin {cases} 0, & \text {if } -\infty <t<\theta ,\\ e^{-(t-\theta )}, & \text {if } \theta \leq t<\infty . \end {cases} \] We wish to test \(H_0:\theta \leq \theta _0\) against \(H_A:\theta >\theta _0\).

(a) Find the uniformly most powerful test of size \(\alpha \).

(b) Compute the power function.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with a continuous distribution function \(F\). Let \(x_{1/4}\) be defined by \(F(x_{1/4})=1/4\).

(a) Find a \(1-\alpha \) lower confidence bound for \(x_{1/4}\).

(b) Provide a large sample approximation.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with distribution function \[ F(t)=\frac {1}{1+e^{-t}},\qquad -\infty <t<\infty . \] Show that \[ Y_n=X_{n,n}-\log n \] converges in distribution, where \(X_{n,n}=\max \{X_1,X_2,\ldots ,X_n\}\).

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with probability mass function \[ f(t,\theta )=\theta (1-\theta )^{t-1}I\{t=1,2,\ldots \}\qquad (0<\theta <1). \] Find the uniformly minimum variance unbiased estimator for \(\theta \).

Let \(X\) and \(Y\) be independent random variables with densities, \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-2,2],\\ 1/4, & \text {if } t\in [-2,2], \end {cases} \] and \[ f(t)= \begin {cases} 0, & \text {if } t\notin [-1,1],\\ 1/2, & \text {if } t\in [-1,1]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with probability mass function \[ P\{X_i=1\}=\theta \quad \text {and}\quad P\{X_i=0\}=1-\theta . \] Let: \[ \bar {X}_n=\frac {1}{n}\sum _{1\leq i\leq n} X_i \] and \(h(t)=t^3-t\). Find the sequences \(a_n\) and \(b_n\) such that \(a_n\bigl (h(\bar {X}_n)-h(\theta )-b_n\bigr )\) has a non-degenerate limit distribution.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with a continuous probability distribution function \(F\) and \(Y_1,Y_2,\ldots ,Y_m\) be independent, identically distributed random variables with a continuous probability distribution function \(H\). The two samples are independent and \(F=H\). Let \[ T=\sum _{1\leq i\leq n}\sum _{1\leq j\leq m} I\{X_i\leq Y_j\}. \] Compute \(E T\) and \(\operatorname {var}T\).

Let \(\epsilon _i\), \(1\leq i\leq n\), be independent identically distributed normal \(N(0,\sigma ^2)\) random variables and consider the model \[ y_i=i\alpha +\gamma i^2+\epsilon _i,\qquad 1\leq i\leq n. \] We observe \(y_i\), \(1\leq i\leq n\).

(a) Find the maximum likelihood estimators for \(\alpha \), \(\gamma \), and \(\sigma ^2\).

(b) Compute the joint distribution of the maximum likelihood estimators for \((\alpha ,\gamma )\).

Suppose \(X_1,X_2,\ldots \) are independent with distribution, \[ \mathbb {P}\{X_n=n^\alpha \}=\mathbb {P}\{X_n=-n^{-\alpha }\}=\frac {1}{2}, \qquad n=1,2,\ldots . \] Prove that \(\sum _{i=1}^{n}X_n/n\to 0\) a.s. if \(0<\alpha <\frac {1}{2}\).

Let \(U_1,U_2,\ldots ,U_n\) be independent identically distributed random variables, uniformly distributed on \([0,1]\). Compute the covariance between \(U_{1,n}=\min _{1\leq i\leq n}U_i\) and \(U_{n,n}=\max _{1\leq i\leq n}U_i\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed normal \(N(\mu ,\sigma ^2)\) random variables. We wish to test \(H_0:\mu =\mu _0\) against \(H_A:\mu \neq \mu _0\) (\(\sigma \) is unknown). Show that the t-test and the likelihood tests are equivalent.

Let \(X_1\) and \(X_2\) be two independent, identically distributed exponential (1) random variables. Compute the joint density of \(Y_1=X_1+X_2\) and \(Y_2=X_1/(X_1+X_2)\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed uniform on \([0,\theta ]\), \(\theta >0\), random variables. We wish to test \(H_0:\theta \leq \theta _0\) against \(H_A:\theta >\theta _0\). Find the uniformly most powerful test of size \(\alpha \) and compute the power function. (You must explain why your test has this property.)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\notin [0,\theta ],\\ \dfrac {1}{\theta }, & \text {if } t\in [0,\theta ]. \end {cases} \] We use \(\hat {\theta }=cX_{1,n}\) and \(\hat {\eta }=\alpha \bar {X}\) to estimate \(\theta \), where \[ X_{1,n}=\min _{1\leq i\leq n}X_i\quad \text {and}\quad \bar {X}=\frac {1}{n}\sum _{1\leq i\leq n}X_i. \] (a) Find \(c\) and \(\alpha \) such that the estimators are unbiased.

(b) Compute the relative efficiency.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed Poisson \((\theta )\) random variables. Find the uniformly minimum unbiased estimator for \(\eta =\theta ^2 e^{-\theta }\). (You must explain your answer.)

Let \(X_1,X_2,\ldots ,X_n\) and \(Y_1,Y_2,\ldots ,Y_m\) be independent, identically distributed random variables with a common continuous distribution function. Compute the expected value and the variance of \[ Z=\sum _{1\leq i\leq n}\sum _{1\leq j\leq n} I\{X_i\leq Y_j\}. \]

where \(I\{A\}\) denotes the indicator function of the event \(A\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\leq \theta ,\\ e^{-(t-\theta )}, & \text {if } t>\theta . \end {cases} \] (a) Find two \(1-\alpha \) equal-tail confidence intervals for \(\theta \). One should be based on the smallest order statistic and the other should be based on the sample mean.

(b) Compare the expected lengths of the confidence intervals.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with \(P\{X_i=0\}=\theta \) and \(P\{X_i=1\}=1-\theta \) \((0<\theta <1)\). Find the numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n=a_n\hat {X}_n(1-\hat {X}_n)-b_n \] converges in distribution to a nondegenerate random variables, where \[ \hat {X}_n=\frac {1}{n}\sum _{1\leq i\leq n}X_i. \]

Construct three random variables \(X,Y\), and \(Z\), such that any distinct pair of them are independent but \((X,Y,Z)\) are not independent.

Let \(\{X_n\}_{n=1}^{\infty }\) be a sequence of i.i.d. random variables with \(\mathbb {E}[X_1]=0\) and \(\mathbb {E}[X_1^2]=1\). Define \(S_k=X_1+\cdots +X_k\) for all \(k\ge 1\). Let \(N_m\) be another independent random variable with a mean-\(m\) Poisson distribution. Prove that there exist non-random \(\alpha _m\) and \(\beta _m\) such that \((S_{N_m}-\alpha _m)/\beta _m\) converges weakly to a standard-normal distribution as \(m\to \infty \). Describe an explicit example of \(\alpha _m\) and \(\beta _m\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential (1) random variables with common density function \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ e^{-t}, & \text {if } 0\leq t<\infty . \end {cases} \] (a) Show that \(\left (\sum _{1\leq i\leq n}X_i\right )^{1/2}-n^{1/2}\) converges in distribution to a normal random variable.

(b) Compute the mean and the variance of the limiting normal random variable.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed Poisson random variables with common probability mass function \[ p(k,\theta )=\frac {\theta ^k}{k!}e^{-\theta },\qquad k=0,1,\ldots \] Let \[ \tau =\theta e^{-\theta }. \] Find the uniformly minimum variance unbiased estimator for \(\tau \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with common density function \(f\). Let \[ \hat {f}_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ), \] where \(K\) is a symmetric density function and \(K(t)=0\) if \(|t|>1\). We assume that \(h=h(n)\to 0\) as \(n\to \infty \). We assume that the second derivative of \(f\) exists and continuous in a neighborhood of \(t\). We use \(\hat {f}\) to estimate \(f\). Compute the bias of \(\hat {f}_n(t)\).

\(X\) and \(Y\) are two independent identically distributed normal \(N(0,\sigma ^2)\) random variables. Let \[ Z=X^2+Y^2 \] and \[ W=\frac {X}{(X^2+Y^2)^{1/2}}. \] (a) Show that \(Z\) and \(W\) are independent.

(b) Compute the marginal distributions of \(Z\) and \(W\).

\(Z_1\) and \(Z_2\) are two independent identically distributed exponential \((\lambda )\) random variables, i.e. the common density is \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ \dfrac {1}{\lambda }e^{-t/\lambda }, & \text {if } 0\leq t<\infty . \end {cases} \] Let \(X=Z_1\) and \(Y=Z_1+Z_1Z_2\). Compute

(a) \(E(Y\mid X=x)\)

(b) \(\operatorname {var}(E(Y\mid X))\).

Let \(X_1,X_2,X_3,\ldots \) be independent and exponential with mean 1. Use the Borel–Cantelli lemmas to show that \(\limsup _{n\to \infty }X_n/\log n=1\) a.s.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with a continuous distribution function \(F\). The ordered observations are \(X_{1,n}\leq X_{2,n}\leq \cdots \leq X_{n,n}\). Let \[ T_n=\sum _{i=1}^{n-1}\left (F(X_{i+1,n})-F(X_{i,n})\right )^2. \] Show that there are numerical sequences \(a_n\) and \(b_n\) such that \((T_n-a_n)/b_n\) converges in distribution to a standard normal random variable. (20 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed normal \(N(0,\sigma ^2)\) random variables. We wish to test \[ H_0:\sigma \leq \sigma _0 \] against \[ H_A:\sigma >\sigma _0. \]

1. Find the uniformly most powerful test of size \(\alpha \). (10 points)
2. Compute the rejection region as a function of \(\alpha \). (10 points)

Consider the following model: \[ y_i=f(x_i)+\epsilon _i,\qquad 1\leq i\leq n, \] where \[ f(t)=a_0+a_1t+\cdots +a_pt^p. \] The parameters \(a_0,a_1,\ldots ,a_p\) are unknown. The errors \(\epsilon _1,\epsilon _2,\ldots ,\epsilon _n\) are independent normal \(N(0,\sigma ^2)\) random variables. It is assumed that \(x_i\in [0,1]\) and known. We observe \((y_i,x_i)\), \(1\leq i\leq n\).

1. Estimate the parameters \(a_0,a_1,\ldots ,a_p\) and \(\sigma ^2\). (15 points)
2. Get the distribution of the estimator for \(\sigma ^2\). (5 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We assume that \(f=F'\) has a bounded second derivative. \(K(t)\) is a non-negative function satisfying \[ K(t)=K(-t)\quad \text {(symmetric),} \] \[ \int _{-\infty }^{\infty }K(t)\,dt=1 \quad \text {and}\quad K(t)=0\text { if }t\notin [-1,1]. \] The sequence \(h=h(n)\) is positive and goes to 0. We estimate \(f\) with \[ \hat {f}_n=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ). \]

1. Compute \(B_n\), the bias of \(\hat {f}_n\). (10 points)
2. Compute the limit of \(B_n/h^2(n)\) as \(n\to \infty \). (10 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables uniform on \([\theta ,\theta +1]\).

1. Find a sufficient statistic for \(\theta \). (5 points)
2. Is your statistic complete? (5 points)
3. Find the maximum likelihood estimator for \(\theta \). (10 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed normal \(N(\mu ,\sigma ^2)\) random variables. Let \[ U_n=\frac {2}{n(n-1)} \sum _{1\leq i<j\leq n}I\{|X_i-X_j|>1\}. \]

1. Compute \(EU_n\). (10 points)
2. Assume that \(\sigma \) is known. We wish to test \(H_0:\mu =\mu _0\) (\(\mu _0\) is given) against the alternative that \(H_0\) is not true. Can we use \(U_n\) to test \(H_0\) against this alternative? (5 points)
3. Now \(\mu \) and also \(\sigma \) are unknown. We wish to test the null-hypothesis that \(\sigma =\sigma _0\) against the alternative that \(H_0\) is not true. Can we use \(U_n\) now? (5 points)

Let \(X_1,X_2,X_3\) be i.i.d., each with an exponential distribution with mean 1. Find the joint distribution of \[ Y_1=\frac {X_1}{X_1+X_2},\qquad Y_2=\frac {X_1+X_2}{X_1+X_2+X_3},\qquad Y_3=X_1+X_2+X_3. \] (7 points) In particular, are they independent? (3 points)

Suppose \(\{E_1,E_2,\ldots \}\) is a sequence of measurable (otherwise arbitrary) events. Suppose \(\{F_1,F_2,\ldots \}\) is another sequence of measurable events also totally independent of all of the \(E_i\)’s. Assume the following:

(a) \(\mathbb {P}(E_k\text {, infinitely often})=1\);

(b) there exists some \(p>0\), such that for all \(k\ge 1\), \(\mathbb {P}(F_k)\ge p\).

Prove: \(\mathbb {P}(E_k\cap F_k\text {, infinitely often})\ge p>0\). (10 points)

Let \(X_1,X_2,\ldots \) be independent random variables with distribution functions \(F_1,F_2,F_3,\ldots \). Let \(Y=\sup _{1\le i<\infty }X_i\).

(a) Show that \(\mathbb {P}\{Y<\infty \}\) is 0 or 1 depending on whether \(\sum _{1\le i<\infty }(1-F_i(x))\) converges for some \(x\). (5 points)

(b) Show that if \(\mathbb {P}\{Y<\infty \}=1\), then \(\prod _{1\le i<\infty }F_i(x)\) converges for all \(x\) and it is the distribution function of \(Y\).

Let \(X_1,X_2,X_3\) be i.i.d., each with an exponential distribution with mean 1. Find the joint distribution of \[ Y_1=\frac {X_1}{X_1+X_2},\qquad Y_2=\frac {X_1+X_2}{X_1+X_2+X_3},\qquad Y_3=X_1+X_2+X_3. \] (7 points) In particular, are they independent? (3 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\) and density function \(f\). We assume that the second derivative of \(f\) is bounded. We estimate \(f\) by \[ f_n(t)=\frac {F_n(t+h_n)-F_n(t-h_n)}{2h_n}, \] where \(F_n\) is the empirical distribution function and \(h_n\to 0\) as \(n\to \infty \).

(a) Show that \(\E f_n(t)\to f(t)\) as \(n\to \infty \). (5 points)

(b) Show that if \(nh_n\to \infty \), then \[ \frac {f_n(t)-\E f_n(t)}{(\operatorname {Var}(f_n(t)))^{1/2}} \] is asymptotically standard normal as \(n\to \infty \). (5 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent Poisson random variables with parameters \(\theta _1,\theta _2,\ldots ,\theta _n\). We wish to test \(H_o:\theta _1=\theta _2=\cdots =\theta _n\) against the alternative that \(H_o\) is not true.

(a) Find the likelihood ratio test. (5 points)

(b) Show that the likelihood ratio is asymptotically normal (after centralizing and normalizing) under \(H_o\). (5 points)

Prove Cantelli’s theorem: If \(X_1,X_2,\ldots \) are independent (but not necessarily identically distributed), mean zero, random variables with \(\sup _n E[X_n^4]<\infty \), then \(S_n/n\to 0\) a.s. as \(n\to \infty \).

Prove Cantelli’s theorem: If \(X_1,X_2,\ldots \) are independent (but not necessarily identically distributed), mean zero, random variables with \(\sup _n \mathbb {E}[X_n^4]<\infty \), then \(S_n/n\to 0\) a.s. as \(n\to \infty \).

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables. Show that the following statements are equivalent:

(a) \(\mathbb {E}|X_1|^\nu <\infty \);

(b) \(X_n/n^{1/\nu }\to 0\), almost surely;

(c) \(\max _{1\le i\le n}|X_i|/n^{1/\nu }\to 0\), almost surely.

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.

Consider the linear model: \[ Y_{ij}=\beta _i+\epsilon _{ij},\qquad 1\le i\le 2,\qquad 1\le j\le J. \] Suppose \(\epsilon _{ij}\)’s are independent and for some (known) \(a_1\) and \(a_2\), \(\epsilon _{ij}\sim N(0,a_i^2\sigma ^2)\).

(a) Find the U.M.V.U.E.’s of \(\beta _1\) and \(\beta _2\).

(b) Suppose you know that for some unknown \(\beta \), \(\beta _1=\beta _2\). Find the U.M.V.U.E. of \(\beta \).

Let \(X_1,X_2,\ldots \) be independent Poisson random variables with \(EX_n=\lambda _n\) and let \(S_n=X_1+\cdots +X_n\). Show that if \(\sum \lambda _n=\infty \), then \(S_n/ES_n\to 1\) a.s.

Hint: Show that (*) \(Y_n/c_n\to 1\) a.s., provided that \(Y_n\geq 0\) is nondecreasing in \(n\), and (*) holds for a subsequence \(n(k)\) that has \(c_{n(k+1)}/c_{n(k)}\to 1\).

Suppose that \(X\) and \(Y\) are independent. Let \(f\) be a Borel function on \(\mathbb {R}^2\) with \(E[|f(X,Y)|]<\infty \) and let \(g(x)=E[f(x,Y)]\). Show that \(E[f(X,Y)\mid X]=g(X)\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \(f\). We assume that \(f'\) is bounded. Let \(K\) be a function satisfying \(\int _{-\infty }^{\infty }K(u)\,du=1\), \(K'(u)\) is bounded and \(K(u)=0\), if \(|u|\geq a\) where \(a\) is a constant. The density \(f\) is estimated by \[ \hat f_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ). \]

Show that \(\hat f_n(t)\) is an almost surely uniformly consistent estimator for \(f\) on \([\alpha ,\beta ]\), \(-\infty <\alpha <\beta <\infty \).

Let \(X_1,X_2,\ldots \) be independent Poisson random variables with \(\mathbb {E} X_n=\lambda _n\), and let \(S_n=X_1+\cdots +X_n\). Show that if \(\sum \lambda _n=\infty \), then \(S_n/\mathbb {E} S_n\to 1\) a.s.

Hint: Show that (*) \(Y_n/c_n\to 1\) a.s., provided that \(Y_n\ge 0\) is nondecreasing in \(n\), and (*) holds for a subsequence \(n(k)\) that has \(c_{n(k+1)}/c_{n(k)}\to 1\).

Suppose that \(X\) and \(Y\) are independent. Let \(f\) be a Borel function on \(\mathbb R^2\) with \(\mathbb {E}[|f(X,Y)|]<\infty \) and let \(g(x)=\mathbb {E}[f(x,Y)]\). Show that \(\mathbb {E}[f(X,Y)\mid X]=g(X)\).

Let \(U_1,U_2,\ldots ,U_n\) be independent identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\le U_{2,n}\le \cdots \le U_{n,n}\) denote the order statistics. Let \(X_1,X_2,\ldots ,X_{n+1}\) be independent identically distributed exponential random variables with \(\mathbb {E} X_1=1\). Define \(S(i)=X_1+\cdots +X_i\).

(a) Prove that the random vectors \[ \{U_{1,n},\ldots ,U_{n,n}\} \quad \text {and}\quad \left \{\frac {S(1)}{S(n+1)},\ldots ,\frac {S(n)}{S(n+1)}\right \} \] have the same distribution.

(b) Compute the asymptotic distribution of \(n(U_{i+3,n}-U_{i,n})\), as \(n\to \infty \) when \(i\) is fixed.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables, uniformly distributed on \([0,\theta ]\), \(\theta >\theta _0\). We want to test \(H_0:\theta =\theta _0\) against \(H_A:\theta >\theta _0\).

(a) Find the uniformly most powerful test. (You must prove your claim.)

(b) Show that the uniformly most powerful test and the likelihood ratio test are equivalent,

(c) Compute the power function of the most powerful test.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \(f\). We assume that \(f'\) is bounded. Let \(K\) be a function satisfying \(\int _{-\infty }^{\infty }K(u)du=1\), \(K'(u)\) is bounded and \(K(u)=0\), if \(|u|\ge a\) where \(a\) is a constant. The density \(f\) is estimated by \[ \hat f_n(t)=\frac {1}{nh}\sum _{1\le i\le n} K\left (\frac {t-X_i}{h}\right ). \] Show that \(\hat f_n(t)\) is an almost surely uniformly consistent estimator for \(f\) on \([\alpha ,\beta ]\), \(-\infty <\alpha <\beta <\infty \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with \(\mathbb {P}\{X_i=1\}=p\), \(\mathbb {P}\{X_i=0\}=1-p\).

(a) Compute the maximum likelihood estimator of \(\sigma ^2=p(1-p)\).

(b) Compute the bias, the variance and the mean-square error of the estimator.

(c) Is the estimator asymptotically efficient?

(d) Find the uniformly minimum variance unbiased estimator for \(\sigma ^2\).