Preliminary Examination 1996: Probability \& Statistics

Prove Cantelli’s theorem: If \(X_1,X_2,\ldots \) are independent (but not necessarily identically distributed), mean zero, random variables with \(\sup _n \mathbb {E}[X_n^4]<\infty \), then \(S_n/n\to 0\) a.s. as \(n\to \infty \).

(a) Let \(X_1,X_2,\ldots \) be i.i.d. Poisson random variables with mean 1, and let \(Z\) be \(N(0,1)\). Denote \(a^-=-\min \{a,0\}\), and prove that \[ \text {(*)}\qquad \mathbb {E}\left [\left (\frac {S_n-n}{\sqrt n}\right )^-\right ]\to \mathbb {E}[Z^-]. \]

(b) Evaluate both expectations in (*) explicitly, and, noting the telescoping sum, deduce Stirling’s formula, for \(n!\).

Let \((\Omega ,\mathcal F,P)\) be a probability space, let \(\mathcal G\) be a sub-\(\sigma \)-field of \(\mathcal F\), and let \(M\) be the closed subspace of \(L^2(\Omega ,\mathcal F,P)\) consisting of the \(\mathcal G\)-measurable functions in \(L^2\). Show that \(T(X)=\mathbb {E}[X\mid \mathcal G]\) coincides with the orthogonal projection of \(L^2(\Omega ,\mathcal F,P)\) onto \(M\).

Consider the Markov chain in the state space \(S=\{\ldots ,-2,-1,0,1,2,\ldots \}\) with transitions \(P(i,i+2)=p\) and \(P(i,i-1)=1-p\), where \(0<p<1\). Determine for which \(p\) this chain is recurrent and for which \(p\) it is transient.

Let \(\{B(t),\, t\ge 0\}\) be a standard Brownian motion. Prove directly, using \[ V_n=\sum _{i=1}^{2^n}\left |B(i/2^n)-B((i-1)/2^n)\right |, \] that \(B(\cdot ,\omega )\) is of unbounded variation on \([0,1]\) for a.e. \(\omega \).

Suppose \(X_1,\ldots ,X_n\) is an i.i.d. sample from a normal population with \[ \mathbb {E} X_1=\operatorname {Var}(X_1)=\mu >0. \]

(a) Compute the maximum likelihood estimator \(\hat {\mu }\) of \(\mu \);

(b) Is \(\hat {\mu }\) consistent?

(c) Is \(\hat {\mu }\) asymptotically normal?

Suppose \(X_1,\ldots ,X_n\) is an i.i.d. sample from a normal distribution with mean \(\mu \) and variance 1.

(a) Find the UMVU estimator for \(\mu \). (Prove the optimality criterion.)

(b) Put a \(N(\theta ,\tau ^2)\) prior on \(\mu \) and find the minimax estimator of \(\mu \).

Consider the linear model: \(Y_i=\beta +\epsilon _i\), \(1\le i\le n\). Here, \(\epsilon _i\)’s are i.i.d. \(N(0,\sigma ^2)\), and \(\sigma \) and \(\beta \) are unknown.

(a) Find the least squares estimator of \(\beta \);

(b) Find the UMVU estimator for \(\beta \) (prove the optimality);

(c) Find the UMP test for \(H_0:\beta =\beta _0\) versus \(H_1:\beta =\beta _1\);

(d) Discuss–without proofs–how to find the UMP test for \(H_0:\beta >\beta _0\) vs. \(H_1:\beta \le \beta _0\) from tests of the form in part (c) above.

Let \(X_1,\ldots ,X_n\) be an i.i.d. sample which is uniformly chosen from the interval \((\theta _1,\theta _2)\).

(a) Prove that there are no one-dimensional (i.e., not vector-valued) sufficient statistics for \(h(\theta _1,\theta _2)\) where \(h\) is a one-to-one measurable function.

(b) Is there a one-dimensional (i.e., not vector-valued) sufficient statistics for \(\mu =(\theta _1+\theta _2)/2\)?

Let \(X_1,\ldots ,X_n\) be an i.i.d. sample from a uniform \((0,\theta )\) distribution, where \(\theta >0\).

(a) Find the maximum likelihood estimator \(\hat {\theta }\) for \(\theta \);

(b) Construct a \(100(1-\alpha )\%\) confidence interval for \(\theta \) based on \(\hat {\theta }\).