Measure-Theoretic Probability

Let \(q\) be an integrable random variable. Prove that for each \(\epsilon > 0\) we can pick \(\delta > 0\) so that \(\mu (A) < \delta \) implies \[ \int _{A}|q|\,d\mu < \epsilon . \] Let \(f \ge 0\). (Hint: Show first that for any \(f \ge 0\), \(\int f \wedge n\,d\mu \nearrow \int f\,d\mu \) as \(n \to \infty \).)

Let \((a_{n})_{n \ge 1}\) be a nondecreasing sequence of positive numbers such that \[ \lim _{n \to \infty }\frac {a_{n}^{2}}{\sum _{k=1}^{n}a_{k}^{2}} = 0. \]

Let \((X_{n})_{n \ge 1}\) be a sequence of i.i.d. random variables with mean 0 and variance 1. Prove that \[ \frac {\sum _{k=1}^{n} a_{k}X_{k}}{\sqrt {\sum _{k=1}^{n}a_{k}^{2}}} \]

converges weakly to the standard normal distribution.

Recall the inversion formula: if \(\mu \) is a probability measure and \(\widehat {\mu }\) is its characteristic function, then for all \(a < b\):

1. Prove that if \(\widehat {\mu } \in L^{1}\), then \(\mu (\{a\}) = 0\) for all \(a \in \mathbb {R}\) and \(\mu \) has a probability density function given by

   \[ f(y) = \frac {1}{2\pi }\int _{-\infty }^{\infty } e^{-ity}\widehat {\mu }(t)\,dt. \]

2. Show that if \(X_{1}, \ldots , X_{n}\) are independent and uniformly distributed on \((-1, 1)\), then for \(n \ge 2\), \(X_{1} + \cdots + X_{n}\) has density

   \[ f(y) = \frac {1}{\pi }\int _{0}^{\infty } \left (\frac {\sin t}{t}\right )^{n}\cos (ty)\,dt. \]

Prove that if \(\{X_{i}\}_{i=1}^{\infty }\) are i.i.d. Uniform-\([0,1]\) random variables, then \[ \frac {4\sum _{i=1}^{n} iX_{i} - n^{2}}{n^{3/2}} \] converges weakly, and identify the limiting distribution. (Recall \(\sum _{i=1}^{n} i = \frac {n(n+1)}{2}\) and \(\sum _{i=1}^{n} i^{2} = \frac {n(n+1)(2n+1)}{6}\).)

Suppose \(\{r_{k}\}_{k \in \mathbb {Z}_{+}}\) is a sequence of real numbers such that \(n^{-1}\sum _{k=1}^{n}|r_{k}| \to 0\) as \(k \to \infty \). Let \(\{X_{k}\}_{k \in \mathbb {N}}\) be a sequence of \(L^{2}\) random variables such that \(\mathbb {E}[X_{k}] = 0\) and \(\mathbb {E}[X_{k}X_{\ell }] \le \) \(r_{\ell -k}\) for all \(k \le \ell \). Let \(S_{n} = X_{1} + \cdots + X_{n}\). Show that \(n^{-1}S_{n}\) converges in \(L^{2}\) as \(n \to \infty \) and identify the limit.

Answer the following two questions.

1. Let \(\{X_{n}\}_{n \in \mathbb {N}}\) be independent, almost surely finite random variables. Show that \[ \sup _{n} X_{n} < \infty \text { a.s.} \] if, and only if, \(\exists c < \infty \) such that \[ \sum _{n} P(X_{n} > c) < \infty . \]
2. Let \(\{A_{n}\}_{n \in \mathbb {N}}\) be a sequence of events with \(P(A_{n}) \ge \delta \) for each \(n\). Show that \[ P(A_{n}\text { happens infinitely often}) \ge \delta . \]

Consider a probability space \((\Omega , \mathcal {F}, P)\). Let \(X \in L^{1}\). Suppose \((\mathcal {F}_{n})_{n \in \mathbb {N}}\) is an increasing sequence of \(\sigma \)-algebras such that \(\cup _{n \in \mathbb {N}}\mathcal {F}_{n}\) generates \(\mathcal {F}\). Prove the Doob martingale convergence theorem that states that \(\mathbb {E}[X \mid \mathcal {F}_{n}]\) converges almost surely to \(X\).

Fix an integer \(n \ge 1\). Let \(\{Y_{k}\}_{k \in \mathbb {N}}\) be i.i.d. uniform on \(\{1, 2, \ldots , n\}\). Let

be the first time we get a repeated sample. Find an exponent \(\gamma \) and a nondegenerate distribution \(\mu \) such that, as \(n \to \infty \), \(n^{-\gamma }\sigma _{n}\) converges weakly to the distribution \(\mu \). (Hint: look at tail probabilities \(P(n^{-\gamma }\sigma _{n} \ge x)\).)

Let \(\{X_{n}\}_{n \in \mathbb {N}}\) be independent random variables such that for each \(n\), \(X_{n}\) takes the values \(+1\) and \(-1\) with probability \((1 - 2^{-n})/2\) each and the value \(2^{k}\) with probability \(2^{-k}\), for integers \(k > n\).

1. Show that \(\mathbb {E}[|X_{n}|] = \infty \) for all \(n \in \mathbb {N}\).
2. Show that \((X_{1} + \cdots + X_{n})/\sqrt {n}\) converges in distribution to a standard normal. (Hint: Check that we can write \(X_{n} = (1 - B_{n})Y_{n} + 2^{n}B_{n}Z_{n}\), where \(B_{n}\) is \(\operatorname {Bernoulli}(2^{-n})\), \(Y_{n}\) takes values \(\pm 1\) equally likely, \(Z_{n}\) takes the value \(2^{k}\) with probability \(2^{-k}\), for \(k \in \mathbb {N}\), and \(\{B_{n}, Z_{n}, Y_{n} : n \ge 1\}\) are mutually independent. Now, \(X_{n}\) is a small perturbation of \((1 - B_{n})Y_{n}\). Work out the details.)

Let \(X\), \(Y\), \(Z\) be random variables on some probability space. Recall that

is the conditional expectation of the indicator function of the event \(\{X \in A\}\), given the \(\sigma \)-algebra \(\sigma \{Y, Z\}\) generated by \(Y\) and \(Z\). We say that \(X\) and \(Y\) are independent, given \(Z\), if for all Borel sets \(A, B \subset \mathbb {R}\),

Show that this condition is equivalent to having \(P(X \in A \mid Y, Z) = P(X \in A \mid Z)\) \(P\)-almost surely, for all Borel sets \(A\).

Let \(Y_{k}\) be i.i.d., \(Y_{k} \ge 0\), and \(\mathbb {E}[Y_{k}] \le 1\) for all \(k\). Prove the existence of the almost sure limit \(M_{\infty } = \lim _{n \to \infty } \prod _{i=1}^{n} Y_{i}\) and describe this random variable explicitly. Do not forget the degenerate cases.

Let \(\{\mathcal {F}_{n}\}_{n \ge 0}\) be a filtration on the probability space \((\Omega , \mathcal {F}, P)\), let \(X \in L^{1}(\Omega , \mathcal {F}, P)\), and define \(M_{n} = \mathbb {E}[X \mid \mathcal {F}_{n}]\). Let \(\tau \) be a stopping time and recall the definition \[ \mathcal {F}_{\tau } = \{A \in \mathcal {F} : A \cap \{\tau = n\} \in \mathcal {F}_{n}\ \forall n \ge 0\}. \]

1. Show that \(1_{\{\tau < \infty \}}M_{\tau }\) is integrable.
2. Show that \(\mathbb {E}[X \mid \mathcal {F}_{\tau }] = 1_{\{\tau < \infty \}}M_{\tau } + 1_{\{\tau = \infty \}}X\). Do not forget the necessary measurability condition.

Let \(\{X_{n}\}_{n \ge 1}\) be i.i.d. Normal random variables with mean 0 and variance 1. Let \(S_{0} = 0\) and \(S_{n} = X_{1} + \cdots + X_{n}\) for \(n \ge 1\). Fix \(t \in \mathbb {R}\) and for \(n \ge 0\) let \[ M_{n} = e^{tS_{n} - t^{2}n/2}. \]

1. Prove that \(M_{n}\) is a mean one martingale relative to the filtration \(\mathcal {F}_{n} = \sigma (X_{1}, \ldots , X_{n})\).
2. Prove that if \(\max _{j \le n} S_{j} \ge nt\), then \(\max _{j \le n} M_{j} \ge e^{t^{2}n/2}\).
3. Conclude that \(P\{\max _{j \le n} S_{j} \ge nt\} \le e^{-t^{2}n/2}\) and then that for any \(c > \theta > 1\) we have with probability one: \(\exists k_{0} \ge 1\) such that for \(k \ge k_{0}\) \[ \frac {\max _{j \le \theta ^{k}} S_{j}} {\sqrt {2c\theta ^{k-1}\log \log \theta ^{k-1}}} \le 1. \]
4. Deduce that almost surely \[ \limsup _{n \to \infty } \frac {S_{n}}{\sqrt {2n\log \log n}} \le 1. \]

Let \(X,Y\) be independent random variables with respective densities \[ f(t)= \begin {cases} 0, & \text {if } t\notin [-2,3],\\ \dfrac {1}{5}, & \text {if } t\in [-2,3], \end {cases} \] and \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-1,5],\\ \dfrac {1}{6}, & \text {if } t\in [-1,5]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X\) and \(Y\) be independent normally distributed random variables with \(\mathbb {E} X=\mu _1\), \(\operatorname {Var} X=\sigma _1^2\), \(\mathbb {E} Y=\mu _2\) and \(\operatorname {Var} Y=\sigma _2^2\). Show that \(X+Y\) and \(X-Y\) are independent if and only if \(\sigma _1^2=\sigma _2^2\).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ h(t;\theta )=\frac {3t^2}{\theta ^3}1\{0\leq t\leq \theta \}, \] where \(\theta >0\) is a parameter. Find a \(100(1-\alpha )\%\), two-sided, equal-tailed confidence interval for \(\theta \).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed with density function \[ h(t;\theta )=\theta t^{\theta -1}1\{0\leq t\leq 1\} \] for \(\theta >0\) unknown. Use the Neyman-Pearson Lemma to find the uniformly most powerful test of size \(\alpha \) for \(H_0:\theta =\theta _0\) versus \(H_a:\theta =\theta _1\), where \(\theta _1>\theta _0\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {10t^9}{2\theta ^{10}}, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find a moment estimator for \(\theta \).

Let \(X\) and \(Y\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\theta }e^{-t/\theta }, & \text {if } t\geq 0. \end {cases} \] Compute the density function of \[ Z=\frac {X}{X+Y}. \]

Let \(X_1,X_2,\ldots \), and \(X\) be real-valued random variables defined on a common probability space. Note they do NOT have to be independent or identically distributed. Prove that \[ \sum _{n=1}^{\infty }\mathbb {E}[|X_n-X|]<\infty \Longrightarrow X_n\to X \quad \text {almost surely.} \]

Let \(X\) be a non-negative, integer-valued random variable.

a) Prove that \(\mathbb {E}[X]=\sum _{n=1}^{\infty }\mathbb {P}(X\geq n)\).

b) A dresser has \(k\) distinct pairs of socks (so \(2k\) socks total) and the socks are unmatched. We select, at random and without replacement, one sock at a time until a pair has been drawn. Compute the expectation of the total number of draws needed.

Let \(\{X_n:n\in \mathbb {N}\}\) be a sequence of i.i.d. positive integrable random variables. Define \(T_n=X_1+\cdots +X_n\), \(n\in \mathbb {N}\), and \[ N(t)=\max \{n\in \mathbb {N}:T_n\leq t\}, \qquad t\geq 0. \] Show that with probability one, \[ \frac {N(t)}{t}\to \frac {1}{\mathbb {E}[X_1]} \qquad \text {as } t\to \infty . \] (Hint: Sketch what the process \(t\mapsto N(t)\) looks like for a fixed realization of the \(X_n\). What do you know about the asymptotic behavior of the jump times?)

Assume the same setting as in the previous problem. You can assume that the claim in that problem is true and solve this problem independently.

a) For an integer \(a\geq 0\) let \(T_a=\inf \{n\geq 0:X_n=a\}\). Fix \(a\geq 2\) and prove that \[ \mathbb {P}(T_0<\infty \text { or }T_a<\infty )=1. \] (Hint: As long as the process is not at 0 it has a positive probability to go up \(a\) times in a row. How do you turn this observation into a proof?)

b) Calculate \(\mathbb {P}(T_a<T_0\mid X_0=1)\). (Carefully explain why the conditions of the theorems you use are satisfied.)

Recall that the characteristic function of a random variable \(X\) is defined as \(\phi _X(t)=\mathbb {E}[e^{itX}]\) for \(t\in \mathbb {R}\). Prove that if \(X\in L^1\) then \(\phi '_X(0)=\mathbb {E}[X]\). Make sure to justify all of your steps.

Consider the symmetric random variable \(X\) taking values in \(\mathbb {Z}\setminus \{-1,0,1\}\) with distribution \[ \mathbb {P}(X=k)=\mathbb {P}(X=-k)=\frac {C}{k^2\log k}, \qquad k=2,3,\ldots \]

a) Prove that there is a finite \(C<\infty \) such that the above is a probability distribution.

b) Show that \(\mathbb {E}[|X|]=\infty \), hence \(\mathbb {E}[X]\) is not defined.

c) However, prove that \(\phi '_X(0)=0\), which shows that the converse direction of the previous problem is not always true. This requires showing that the limit defining \(\phi '_X(0)\) exists and equals zero. (Hint: Prove then use the inequality \(|\cos x-1|\leq \min (x^2/2,1)\).)

Let \((Z_n)\) be i.i.d. random variables with \(\mathbb {P}(Z_1=1)=\mathbb {P}(Z_1=2)=1/2\). Show that \(\mathbb {E}[Z_1\cdots Z_n]=(1.5)^n\) and yet \(Z_1\cdots Z_n/(1.45)^n\to 0\) almost surely. Identify the set of numbers \(a>0\) for which \(Z_1\cdots Z_n/a^n\to 0\) almost surely.

(Pólya’s Urn) An urn initially contains \(r\) red and \(b\) blue balls. A ball is chosen uniformly at random (i.e. with probability \(1/(r+b)\) each). If it comes up red (resp. blue), then it is returned and another red (resp. blue) ball is added to the urn. The process is repeated indefinitely. Let \(R_n\) be the number of red balls in the urn after \(n\) draws.

1. A vector \((X_1,X_2,\ldots ,X_n)\) of random variables is said to be exchangeable if \[ (X_1,\ldots ,X_n)\sim (X_{\pi (1)},\ldots ,X_{\pi (n)}) \qquad \text {for all permutations }\pi , \] where \(\sim \) denotes equality in distribution. If \(X_i\) is the indicator that a red ball is drawn from the urn at time \(i\), prove that \((X_1,\ldots ,X_n)\) is exchangeable.
2. Find the mean and variance of \(S_n=X_1+\cdots +X_n\), the total number of red balls added to the urn up to time \(n\).

Let \(X_1,X_2,\ldots \) be independent random variables with \[ X_n= \begin {cases} 1, & \text {with probability }1/(2n),\\ 0, & \text {with probability }1-1/n,\\ -1, & \text {with probability }1/(2n). \end {cases} \] Let \(Y_1=X_1\) and for \(n\geq 2\) define \[ Y_n= \begin {cases} X_n, & \text {if }Y_{n-1}=0,\\ nY_{n-1}|X_n|, & \text {if }Y_{n-1}\ne 0. \end {cases} \] Show that \(Y_n\) is a martingale with respect to \(\mathcal {F}_n=\sigma (Y_1,\ldots ,Y_n)\). Show that \(Y_n\) does not converge almost surely. Why does the martingale convergence theorem not apply?

Let \(X_n\) be a sequence of i.i.d. random variables. Assume that \(\mathbb {P}(X_n\to \infty )>0\). Prove that \(\mathbb {P}(X_0=\infty )=1\). Hint: first prove that \(\mathbb {P}(X_n\to \infty )=1\).

Let \(X\) and \(Y\) be two random variables such that \(\mathbb {P}(X\leq Y)=1\). Assume also that \(\mathbb {P}(X\leq x)\leq \mathbb {P}(Y\leq x)\) for all \(x\in \mathbb {R}\). Prove that \(\mathbb {P}(X=Y)=1\).

Fix \(\alpha \in (0,1)\). Let \(X_n\) be a sequence of independent random variables with \(\mathbb {P}(X_n=n)=\mathbb {P}(X_n=-n)=n^{-\alpha }/2\) and \(\mathbb {P}(X_n=0)=1-n^{-\alpha }\). Prove that \((X_1+\cdots +X_n)/n^{(3-\alpha )/2}\) converges weakly to a centered normal.

Let \(X_1,X_2,\ldots \) be iid random variables with common pdf \[ f(x)= \begin {cases} 0, & |x|\leq 2,\\ \dfrac {c}{x^2\log |x|}, & |x|>2. \end {cases} \]

1. Prove that there is a finite constant \(c\) which makes \(f\) a pdf.
2. Let \(S_n=X_1+X_2+\cdots +X_n\). Prove that \(S_n/n\to 0\) in probability as \(n\to \infty \). Hint: Prove that \(S_n/n\) converges to 0 in distribution, then explain why you can upgrade to convergence in probability for free.
3. Does \(S_n/n\to 0\) almost surely as \(n\to \infty \)?

Let \(X\) and \(Y\) be two integer-valued random variables defined on the same probability space. The total variation distance between \(X\) and \(Y\) is defined as \[ d_{TV}(X,Y)=2\sup _{A\subseteq \mathbb {Z}} |\mathbb {P}(X\in A)-\mathbb {P}(Y\in A)|. \] Prove that \(\mathbb {P}(X=Y)\leq 1-\frac {1}{2}d_{TV}(X,Y)\).

Let \(X_1,X_2\) be iid with density \[ f(x;\theta )= \begin {cases} 3x^2/\theta ^3, & 0<x<\theta ,\\ 0, & \text {otherwise.} \end {cases} \] Find a complete, sufficient statistic for \(\theta \in [0,\infty )\) based on \(X_1\) and \(X_2\).

Let \(X\) be discrete taking values in \(\{0,1\}\) and have the pdf \(f(x;\theta )=\theta ^x(1-\theta )^{1-x}\) for \(x\in \{0,1\}\). We test the simple hypothesis \(H_0:\theta =1/4\) agains the alternative composite hypothesis \(H_A:\theta <1/4\) by taking a random sample of size 10 and rejecting \(H_0\) if and only if the observed values \(x_1,\ldots ,x_{10}\) satisfy \[ \sum _{i=1}^{10}x_i\leq 1. \] Find the power function \(K(\theta )\) of this test for \(0\leq \theta \leq 1/4\).

Consider the density function \[ p(x;\alpha ,\lambda )=(2\pi x^3)^{-3/2} \exp \left \{(\alpha \lambda )^{1/2}-\frac {1}{2}\log \lambda -\frac {1}{2}\alpha x-\frac {\lambda }{2}x^{-1}\right \}, \qquad 0<x<\infty . \] Given \(X_1,X_2,\ldots ,X_n\) iid from this density find a sufficient statistic (that involves all of the \(X_i\)) for the parameters \((\alpha /2,\lambda /2)\).

Let \(X_1,X_2,\ldots ,X_n\) be iid with a Poisson\((\theta )\) distribution, so that their common probability mass function is \[ \mathbb {P}(X_i=k)=e^{-\theta }\frac {\theta ^k}{k!}, \qquad k=0,1,2,\ldots ,\ \theta >0. \] Find the UMVUE of \(e^{-\theta }\).

Let \(\{X_i:i\geq 1\}\) be a sequence of non-negative, i.i.d. random variables such \(\mathbb {E}[X_1]=\infty \). Prove that \[ \lim _{n\to \infty }\frac {X_1+\cdots +X_n}{n}=\infty \qquad \text {almost surely.} \]

Let \(\mu \) and \(\nu \) be probability measures on \((\mathbb {R},\mathcal {B}(\mathbb {R}))\). Let \(\lambda =\frac {1}{2}\mu +\frac {1}{2}\nu \). Let \(\mathcal {F}\) be a \(\sigma \)-algebra contained in \(\mathcal {B}(\mathbb {R})\). Let \(\mu _{\mathcal {F}}\), \(\nu _{\mathcal {F}}\), and \(\lambda _{\mathcal {F}}\) be the restrictions of \(\mu \), \(\nu \), and \(\lambda \) to \(\mathcal {F}\), respectively.

1. Prove that \(\mu _{\mathcal {F}}\ll \lambda _{\mathcal {F}}\) and \(\nu _{\mathcal {F}}\ll \lambda _{\mathcal {F}}\).
2. What is \(f_\mu =\frac {d\mu _{\mathcal {F}}}{d\lambda _{\mathcal {F}}}\) in terms of \(\frac {d\mu }{d\lambda }\)? Of course, a similar relationship holds for \(f_\nu =\frac {d\nu _{\mathcal {F}}}{d\lambda _{\mathcal {F}}}\).
3. For a bounded \(\mathcal {B}(\mathbb {R})\)-measurable function \(g\), compute \(\mathbb {E}^\lambda [g\mid \mathcal {F}]\) in terms of \(\mathbb {E}^\mu [g\mid \mathcal {F}]\), \(\mathbb {E}^\nu [g\mid \mathcal {F}]\), \(f_\mu \), and \(f_\nu \)?

Let \(\{X_i:i\geq 1\}\) be i.i.d. integrable random variables. Prove that \[ \lim _{n\to \infty } \frac {1}{\log n}\sum _{i=1}^{n}\frac {X_i}{i} = \mathbb {E}[X_1]. \]

Prove the identity \[ \frac {\sin \theta }{\theta } = \int _0^1 e^{i\theta (2x-1)}\,dx. \] Then use this to show that \[ \frac {\sin \theta }{\theta } = \prod _{n=1}^{\infty }\cos (\theta /2^n). \] Hint: Use that a Uniform\((0,1)\) random variable can be written as an infinite weighted sum of iid Bernoulli\((1/2)\) random variables.

Let \(Z_1,Z_2,\ldots \) be iid \(N(0,1)\) random variables. First prove Mills’ ratio \[ \mathbb {P}(Z_i>\lambda ) \leq \frac {1}{\lambda \sqrt {2\pi }}e^{-\lambda ^2/2} \] for any \(\lambda >0\). Use this to show that for any \(\epsilon >0\) \[ \mathbb {P}\left ( \limsup _{n\to \infty } \frac {\max _{k\leq n}Z_k}{\sqrt {(2-\epsilon )\log n}} >1 \right )=1. \]

Show that for every \(\rho \in (-1,1)\) the function \[ f(x,y) = \frac {1}{2\pi \sqrt {1-\rho ^2}} \exp \left ( -\frac {x^2-2\rho xy+y^2}{2(1-\rho ^2)} \right ) \] is a pdf on \(\mathbb {R}^2\), and that if \((X,Y)\) is a pair of random variables with pdf \(f\) then \[ \mathbb {P}(XY<0)=\frac {1}{\pi }\arccos \rho . \]

Let \(X_1,X_2\) be iid with density \[ f(x;\theta )= \begin {cases} 3x^2/\theta ^3, & 0<x<\theta ,\\ 0, & \text {otherwise.} \end {cases} \] Find a complete, sufficient statistic for \(\theta \in [0,\infty )\) based on \(X_1\) and \(X_2\).

Let \(X\) be discrete taking values in \(\{0,1\}\) and have the pdf \(f(x;\theta )=\theta ^x(1-\theta )^{1-x}\) for \(x\in \{0,1\}\). We test the simple hypothesis \(H_0:\theta =1/4\) agains the alternative composite hypothesis \(H_A:\theta <1/4\) by taking a random sample of size 10 and rejecting \(H_0\) if and only if the observed values \(x_1,\ldots ,x_{10}\) satisfy \[ \sum _{i=1}^{10}x_i\leq 1. \] Find the power function \(K(\theta )\) of this test for \(0\leq \theta \leq 1/4\).

Consider the density function \[ p(x;\alpha ,\lambda )=(2\pi x^3)^{-3/2} \exp \left \{(\alpha \lambda )^{1/2}-\frac {1}{2}\log \lambda -\frac {1}{2}\alpha x-\frac {\lambda }{2}x^{-1}\right \}, \qquad 0<x<\infty . \] Given \(X_1,X_2,\ldots ,X_n\) iid from this density find a sufficient statistic (that involves all of the \(X_i\)) for the parameters \((\alpha /2,\lambda /2)\).

Let \(X_1,X_2,\ldots ,X_n\) be iid with a Poisson\((\theta )\) distribution, so that their common probability mass function is \[ \mathbb {P}(X_i=k)=e^{-\theta }\frac {\theta ^k}{k!}, \qquad k=0,1,2,\ldots ,\ \theta >0. \] Find the UMVUE of \(e^{-\theta }\).

Let \((X_n)\) be a sequence of real-valued random variables and define the tail \(\sigma \)-algebra \(\mathcal {T}=\cap _{n\geq 1}\sigma (X_n,X_{n+1},\ldots )\). Let \(S_n=X_1+\cdots +X_n\). Prove or disprove the following statements.

1. \(\{\lim _{n\to \infty }S_n\text { exists}\}\in \mathcal {T}\).
2. \(\{\overline {\lim }_{n\to \infty }S_n>0\}\in \mathcal {T}\).
3. \(\{\overline {\lim }_{n\to \infty }S_n/c_n>1\}\in \mathcal {T}\) if \(c_n\to \infty \) as \(n\to \infty \).

1. Prove the Borel-Cantelli lemma. That is, prove that if \(\{A_n\}_{n\geq 1}\) is any sequence of events satisfying \(\sum _{n\geq 1}\mathbb {P}(A_n)<\infty \), then \(\mathbb {P}(A_n\text { occurs for infinitely many }n)=0\).
2. Prove that if \((X_n)\) is any sequence of real-valued random variables, then there are constants \(c_n\) so that \(X_n/c_n\to 0\) almost surely as \(n\to \infty \).

Let \(X\) and \(Y\) be two standard normal random variables (defined on a common probability space and having positive variances).

1. Prove that they are independent if and only if they are jointly normal (i.e. \((X,Y)\) is a two-dimensional normal random variable) and uncorrelated.
2. Give an example of two standard normal random variables that are uncorrelated but not jointly normal.

Let \((X_n)\) be i.i.d. Cauchy distributed random variables. That is, they have probability density function \(\frac {1}{\pi (1+x^2)}\). Abbreviate \(S_n=X_1+\cdots +X_n\).

1. Prove that \[ \lim _{n\to \infty }\frac {1}{n}\sum _{i=1}^{n}|X_i|=\infty \qquad \text {almost surely.} \]
2. Using the fact that the characteristic function of a Cauchy distributed random variable is given by \(e^{-|t|}\), prove that \(S_n/n\) is again a Cauchy distributed random variable.
3. Deduce that \[ \overline {\lim }_{n\to \infty } S_n/n=\infty \qquad \text {and}\qquad \underline {\lim }_{n\to \infty } S_n/n=-\infty , \qquad \text {almost surely.} \]

Let \((X_n)\) be i.i.d. uniform random variables on \([0,1]\). For each \(n\geq 1\), define \[ Y_n=n\min \{X_k:1\leq k\leq n\}. \] Prove that as \(n\to \infty \), \(Y_n\) converges weakly to an exponential random variable with mean one.

Give an example of a sequence of random variables \(X_n\) with densities \(f_n(x)\) so that \(X_n\) converges weakly as \(n\to \infty \) to a uniform random variable on \([0,1]\), but \(f_n(x)\) does not converge to 1 for any \(x\in [0,1]\).

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with probability mass function \[ f(t;\theta )=\theta (1-\theta )^{t-1}, \qquad t=1,2,3,\ldots \] Find the uniformly minimum variance unbiased estimator of \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be iid with density \[ f(x)=2x/\theta ^2, \qquad 0<x<\theta . \] Let \(Y=X_{(n)}=X_{n:n}\) be the largest order statistic of the \(X_i\). Show that \(Y/\theta \) is a pivotal quantity for \(\theta \) and use it to construct a 99% confidence interval for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be iid with a Poisson\((\theta )\) distribution, so that their common probability mass function is \[ \mathbb {P}(X_i=k)=e^{-\theta }\frac {\theta ^k}{k!}, \qquad k=0,1,2,\ldots ,\ \theta >0. \] Find a sufficient statistic for \(\theta \).

Assume that the lifetimes of 100 lightbulbs are iid random variables with an Exponential(5) distribution, meaning that their common pdf is \[ f(x;\theta )=\frac {1}{5}e^{-x/5}, \qquad x>0. \] Use normal approximation to estimate the probability that in total the 100 lightbulbs last longer than 550 hours. You can express your answer as an integral.

Let \(X_1,X_2,\ldots ,X_n\) be iid with common pdf \[ f(x;\theta )=\frac {\theta }{(1+x)^{\theta +1}}, \qquad 0<x<\infty ,\ \theta >0. \] What is a lower bound for the variance of an unbiased estimator of \(\theta \)?

Let \(X_1,X_2,\ldots \) be independent random variables, and suppose that \(p_i:=\mathbb {P}\{X_i=0\}\) satisfies \(0<p_i<1\) for every \(i=1,2,\ldots \). For every \(n\geq 1\) let \[ N_n:=\sum _{i=1}^{n}\mathbf {1}_{\{X_i=0\}} \] denote the number of times the finite random sequence \(X_1,\ldots ,X_n\) enters zero. Prove that \[ \frac {N_n}{\sum _{i=1}^{n}p_i}\to 1 \qquad \text {in probability as }n\to \infty . \]

Suppose \(X\) and \(Y\) are two independent, real-valued random variables, and suppose that \(\mathbb {P}\{X\in A\}=0\) for all Borel sets \(A\) of zero Lebesgue measure. Prove, carefully, that there exists a measurable function \(f:\mathbb {R}\to \mathbb {R}\) such that \[ \mathbb {P}\{X+Y\in B\}=\int _B f(x)\,dx \qquad \text {for all measurable sets }B\subseteq \mathbb {R}. \]

Let \(X_1,X_2,\ldots \) be a collection of i.i.d. random variables, taking values \(\pm 1\) with probability \(1/2\) each. Define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\), \(T_0:=0\), and then iteratively define \[ T_{k+1}:=\inf \{n>T_k:S_n=0\} \qquad [\inf \emptyset :=\infty ]\quad \text {for all }k\geq 0. \] That is, \(T_k\) denotes the \(k\)th return time to zero. Prove that \(\lim _{k\to \infty }(T_k/k)=\infty \) a.s.

Suppose \(X\) is a nonnegative random variable that satisfies the following: There exist two real numbers \(a,b>0\) such that \(a^n\leq \mathbb {E}[X^n]\leq b^n\) for every integer \(n\geq 1\). Prove that \(\mathbb {P}\{a\leq X\leq b\}=1\).

Suppose \(X\) is a discrete, non-negative random variable with mass function \(p\). Prove that \[ \lim _{n\to \infty }\left (\mathbb {E}[X^n]\right )^{1/n} = \sup \{x\in \mathbb {R}:p(x)>0\}. \]

Let \(X_1,X_2,\ldots \) be a collection of i.i.d. random variables, and define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\). Suppose \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2 k^2} \qquad \text {for all } k=\pm 1,\pm 2,\ldots . \] Prove that \(S_n/n\) converges weakly to a non-constant random variable as \(n\to \infty \).

Let \(p_n\) and \(q_n\) denote two probability mass functions on \(\mathbb {Z}^n\), for every integer \(n\geq 1\). That is, for all \(n\geq 1\):

1. \(p_n(\mathbf {x})\geq 0\) and \(q_n(\mathbf {x})\geq 0\) for all \(\mathbf {x}\in \mathbb {Z}^n\); and
2. \(\sum _{\mathbf {x}\in \mathbb {Z}^n}p_n(\mathbf {x}) =\sum _{\mathbf {x}\in \mathbb {Z}^n}q_n(\mathbf {x})=1\).

Now suppose \(\{X_i\}_{i=1}^{\infty }\) is a collection of random variables such that \((X_1,\ldots ,X_n)\) has mass function \(p_n\) for every \(n\geq 1\). That is, \[ \mathbb {P}\{X_1=x_1,\ldots ,X_n=x_n\} = p_n(x_1,\ldots ,x_n) \qquad \text {for all }(x_1,\ldots ,x_n)\in \mathbb {Z}^n. \]

1. Prove that \(p_n(X_1,\ldots ,X_n)>0\) a.s. for all \(n\geq 1\).
2. Prove that \(\{Z_n\}_{n=1}^{\infty }\) is a martingale, where, for all \(n\geq 1\), \[ Z_n:= \frac {q_n(X_1,\ldots ,X_n)}{p_n(X_1,\ldots ,X_n)}, \] almost surely on the \(\mathbb {P}\)-measure-one event \(\{p_n(X_1,\ldots ,X_n)>0\}\), and \(Z_n:=0\) otherwise.
3. Prove that \(\lim _{n\to \infty }Z_n\) exists a.s. and is finite a.s. (This is a starting point for likelihood-ratio testing in classical statistics.)

Suppose that \(X_1,X_2,\ldots \) are i.i.d. exponential random variables with parameter \(\lambda >0\). Prove that \[ \overline {\lim }_{n\to \infty }\left (\frac {X_n}{\log n}\right )=\lambda ^{-1} \qquad \text {and}\qquad \underline {\lim }_{n\to \infty }\left (\frac {\log X_n}{\log n}\right )=-1. \]

Let \(X\) and \(Y\) be two real-valued random variables. Let \(f\) and \(g\) be two non-decreasing measurable functions such that \(f(X)\) and \(g(Y)\) are integrable. Prove that \[ \mathbb {E}[f(X)g(Y)]\geq \mathbb {E}[f(X)]\mathbb {E}[g(Y)]. \]

Let \(X_1\) and \(X_2\) be two independent random variables. Suppose that \(X_1\) is \(\chi ^2(r_1)\) and \(X_1+X_2\) is \(\chi ^2(r)\), where \(r_1,r\) are the respective degrees of freedom and \(r_1<r\). Show that \(X_2\) is \(\chi ^2(r-r_1)\).

Construct two uncorrelated random variables that are not independent.

Let \(\{X_n\}_{n=1}^{\infty }\) be i.i.d. random variables, and let \(p>0\) be fixed. Prove that the following are equivalent:

(a) \(\mathbb {E}(|X_1|^p)<\infty \);

(b) \(n^{-1/p}X_n\to 0\) almost surely; and

(c) \(n^{-1/p}\max _{1\leq j\leq n}|X_j|\to 0\) almost surely.

Suppose \(\{X_n\}_{n=1}^{\infty }\) are i.i.d. exponential random variables with mean one; that is, \(\mathbb {P}\{X_n>x\}=e^{-x}\) for all \(x>0\). Prove that \[ \mathbb {P}\left \{\limsup _{n\to \infty }\frac {X_n}{\log n}=1\right \}=1. \]

Suppose \(X_1,X_2,\ldots \) are i.i.d. strictly positive random variables. Compute for all \(n,m,k\geq 1\), \[ \mathbb {E}\left ( \frac {X_1^k+\cdots +X_m^k}{X_1^k+\cdots +X_n^k} \right ). \] You might wish to start by calculating \(\mathbb {E}(X_1^k\mid X_1^k+\cdots +X_n^k)\).

Let \(X\) and \(Y\) be two positive random variables. We say that \(X\) is stochastically dominated by \(Y\) if \(\mathbb {P}\{X>a\}\leq \mathbb {P}\{Y>a\}\) for all \(a>0\). Prove that, in this case, \(\mathbb {E}(X^k)\leq \mathbb {E}(Y^k)\) for all \(k\geq 1\).

Suppose \(X_1,N_1,X_2,N_2,\ldots \) are independent random variables such that the \(X\)’s are Bernoulli\((1/2)\)—that is, \(\mathbb {P}\{X_i=0\}=\mathbb {P}\{X_i=1\}=1/2\) for all \(i\geq 1\)—and each \(N_n\) is Poisson\((n)\); that is, \[ \mathbb {P}\{N_n=k\}=\frac {n^ke^{-n}}{k!} \qquad \text {for all integers }n\geq 1\text { and }k\geq 0. \] Find non-random sequences \(\{a_n\}_{n=1}^{\infty }\) and \(\{b_n\}_{n=1}^{\infty }\) such that, as \(n\to \infty \), \[ \frac {\sum _{i=1}^{N_n}X_i-a_n}{b_n}\Rightarrow \operatorname {N}(0,1). \]

Let \(X_1,\ldots ,X_n\) be a random sample with density function \(f(x)=2e^{-2(x-\eta )}\) for \(x>\eta \), and zero otherwise. We wish to test \(H_0:\eta =4\) versus the alternative \(H_A:\eta >4\). We reject \(H_0\) if \(X_{(1)}=\min _{1\leq i\leq n}X_i\geq 4+2/n\).

(a) Compute the significance level of the test.

(b) Compute the power function.

Let \(X_1,\ldots ,X_n\) be iid exponential(1) random variables. Find the joint pdf of the pair \((X_{(1)},X_{(3)})\), where \(X_{(i)}\) denotes the \(i\)th order statistic, i.e. \[ X_{(1)}\leq X_{(2)}\leq \cdots \leq X_{(n)}. \]

Let \(X_1\) and \(X_2\) be independent with joint pdf \(f(x)=x^{-2}\) for \(1\leq x<\infty \) and 0 otherwise.

(a) Find the joint pdf of \(U=X_1X_2\) and \(V=X_2\).

(b) Find the marginal pdf of \(U\).

(a) Find a simple asymptotic formula for \(\mathbb {P}(X>x)\), where \(X\) has the GAMMA\((\theta ,\alpha )\) density, \[ f(x)=\frac {1}{\theta ^{\alpha }\Gamma (\alpha )} x^{\alpha -1}e^{-x/\theta }, \qquad x>0. \]

(b) Let \(X_1,X_2,\ldots \) be i.i.d. GAMMA\((\theta ,\alpha )\). Find an increasing sequence \(a_n\) such that \(\limsup _{n\to \infty }X_n/a_n=1\) a.s., and prove it.

Let \(U_0,U_1,\ldots \) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(N\) be a Poisson random variable with \(\mathbb {E}[N]=1\), independent of \(U_0,U_1,\ldots \). Compute \(\mathbb {E}[Y]\), where \(Y=\max _{0\leq k\leq N} U_k\).

Let \(\{e_i,-\infty <i<\infty \}\) be independent and identically distributed random variables with \(\mathbb {E}[e_1]=0\) and \(\mathbb {E}[e_1^2]=\sigma ^2\) and \(\rho \) be a real number satisfying \(|\rho |<1\). Show that \[ Y=\sum _{i=0}^{\infty }\rho ^i e_i \] is finite with probability one. Compute \(\mathbb {E}[Y]\) and \(\mathbb {E}[Y^2]\). (You need to justify your steps when you compute these two expected values.)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t;\theta )= \begin {cases} \dfrac {1}{\theta }, & \text {if } 0\leq t\leq \theta ,\\ 0, & \text {if } t\notin [0,\theta ]. \end {cases} \] We wish to test \(H_0:\theta =1\) against the alternative \(H_A:\theta <1\). We reject \(H_0\) if \[ X_{n,n}=\max _{1\leq i\leq n}X_i\leq 1-\frac {2}{n}. \] (a) Compute the significance level.

(b) Compute the power function.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t;\mu )=\frac {1}{(2\pi )^{1/2}}e^{-(t-\mu )^2/2}. \] We wish to test \(H_0:\mu \leq \mu _0\) against \(H_A:\mu >\mu _0\). Find the uniformly most powerful test of size \(\alpha \).

Let \(X_1,X_2\) be independent and identically distributed exponential\((\lambda )\) random variables, i.e. the common density is \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Find the joint density function of \[ Y_1=X_1+X_2 \quad \text {and}\quad Y_2=\frac {X_1}{X_1+X_2}. \]

Let \(X\) and \(Y\) be independent random variables with densities \(f\) and \(g\), where \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{2}\exp (-t/2), & \text {if } t\geq 0, \end {cases} \] and \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-1,2],\\ \dfrac {1}{3}, & \text {if } t\in [-1,2]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Construct an equal tail \(1-\alpha \) confidence interval for \(\lambda \). Compute the expected value of the length of the confidence interval.

Let \(X_1\) and \(X_2\) be independent \(\operatorname {Exponential}(1)\) random variables (density \(e^{-x}\), \(x>0\)). Find the joint density of \(Y_1=X_1/X_2\) and \(Y_2=X_1+X_2\). Find the marginal densities of \(Y_1\) and \(Y_2\) as well.

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables with \(\mathbb {E}\log (1+|X_1|)<\infty \). Show that \[ \sum _{i=1}^{\infty }\prod _{j=1}^{i}X_j \] is finite with probability one if and only if \(\mathbb {E}\log |X_1|<0\).

Let \(X_1,X_2,\ldots \) be independent identically distributed positive random variables with \(\mathbb {E} X_1=\mu \). Define \[ N(t)=\min \left \{k:\sum _{i=1}^{k} X_i>t\right \}. \] Show that \[ \lim _{t\to \infty }\frac {1}{t}N(t)=\frac {1}{\mu } \qquad \text {a.s.} \]

Let \(X\) and \(Y\) be independent standard normal random variables. Compute the density function of \(Z=X/Y\).

Suppose \(X=\operatorname {Poisson}(\lambda )\) for some \(\lambda >0\). Prove that \(\mathbb {P}\{X>e\lambda \}\leq e^{-\lambda }\).

Let \(X_1,X_2,\ldots \) be i.i.d. integer-valued random variables with probability mass function \(f(a):=\Prob \{X_1=a\}\) for all \(a\in \mathbb {Z}\), and suppose \(f(a)>0\) for all \(a\in \mathbb {Z}\). Now let \(g\) be another probability mass function on \(\mathbb {Z}\) and define the likelihood ratio, \[ \Lambda _n:=\prod _{j=1}^{n}\frac {g(X_j)}{f(X_j)}. \]

(a) Consider the log-likelihood \(\log \Lambda _n\), where “log” denotes the natural logarithm. Prove that \[ \lim _{n\to \infty }n^{-1}\log \Lambda _n =\mathbb {D}(g\,\|\,f) \qquad \text {a.s.,} \] where \[ \mathbb {D}(g\,\|\,f):= \sum _{a=-\infty }^{\infty } f(a)\log \left [\frac {g(a)}{f(a)}\right ], \] provided that \[ \mathbb {E}(g\,\|\,f):= \sum _{a=-\infty }^{\infty } \left |\log \left [\frac {g(a)}{f(a)}\right ]\right |f(a)<\infty . \]

(b) Verify that \(\log x\leq x-1\) for all \(x>0\), and that the inequality is strict except when \(x=1\). Use this to prove the following [Gibbs’ inequality]: \(\mathbb {D}(g\,\|\,f)<0\).

(c) Conclude from the preceding that if \(\mathbb {E}(g\,\|\,f)<\infty \), then \(\Lambda _n\to 0\) a.s. very rapidly as \(n\to \infty \).

This fact plays an important role in statistics.

Suppose \(f:[0,\infty )\to \mathbb {R}\) is bounded and measurable, and \(f(x)=0\) for all \(x\geq 2\).

(a) Prove that if \(X_1,X_2,\ldots \) are independent, all distributed uniformly in the interval \((0,1)\), then \[ \frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+X_j}{n}\right ) \xrightarrow {\mathbb {P}} \int _0^{\infty } f(x)\,dx \qquad \text {as } n\to \infty . \]

(b) (Extra credit) Deduce from this the following version of Riemann sums for Lebesgue integrals [Chaterjee’s theorem]: There exists a sequence \(\{\delta _{j,n};\ 0\leq j\leq n,\ n\geq 1\}\) of real numbers in \((0,1)\) such that \[ \lim _{n\to \infty }\frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+\delta _{j,n}}{n}\right ) = \int _0^{\infty } f(x)\,dx. \]

Let \(f\) be a continuous element of \(L^1(\mathbb {R})\), and recall that its Fourier transform is \[ \hat {f}(t):=\int _{-\infty }^{\infty }\exp (itx)f(x)\,dx. \]

(a) Suppose also that \(\hat {f}\in L^1(\mathbb {R})\). Prove the following [inversion formula]: \[ f(x)=\frac {1}{2\pi }\int _{-\infty }^{\infty }e^{-ixt}\hat {f}(t)\,dt \qquad \text {for all }x\in \mathbb {R}. \] In other words, \(\widehat {\widehat {f}}(-x)=2\pi f(x)\) for all \(x\in \mathbb {R}\).

(b) Choose and fix some \(\theta >0\). Use the preceding to prove that the Fourier transform of the probability density function \[ f(x):=\frac {\theta }{\pi (\theta ^2+x^2)} \qquad [-\infty <x<\infty ] \] is \(\hat {f}(t)=\exp (-\theta |t|)\).

Suppose \(X\) and \(Y\) are i.i.d. random variables and \(f\) and \(g\) are nondecreasing, bounded and measurable functions on \(\mathbb {R}\).

(a) Prove that \[ \mathbb {E}[(f(X)-f(Y))(g(X)-g(Y))]\geq 0. \]

(b) Conclude that \(f(X)\) and \(g(X)\) are always positively correlated; i.e., \[ \mathbb {E}[f(X)g(X)]\geq \mathbb {E}[f(X)]\cdot \mathbb {E}[g(X)]. \]

Let \(Z=\operatorname {N}(0,1)\) denote a standard normal random variable. Suppose \(f:\mathbb {R}\to \mathbb {R}\) is continuously differentiable and there exists \(c>1\) such that \[ |f(x)|+|f'(x)|\leq c\exp (c|x|) \qquad \text {for all }x\in \mathbb {R}. \] Derive the following [Stein’s differential equation]: \[ \mathbb {E}[f'(Z)]=\mathbb {E}[Zf(Z)]. \]

Suppose \(X_1,X_2,\ldots ,X_n\) are i.i.d. random variables, and \(1\leq X_1\leq 2\) a.s. Compute, for every \(1\leq i\leq j\leq n\), the quantity \[ \mathbb {E}\left (\frac {X_i+\cdots +X_j}{X_1+\cdots +X_n}\right ). \]

Suppose \(\mu \) is a probability measure on the Borel subsets of \((0,1)\) such that \[ \int |\ln (x)|\,\mu (dx)<\infty . \] Prove that \[ I(n):=\left (\int x^{1/n}\,\mu (dx)\right )^n \to \exp \left (\int \ln (x)\,\mu (dx)\right ) \qquad \text {as } n\to \infty . \]

[Hint: \(I(n)=\mathbb {E}\left (\prod _{j=1}^{n}X_j^{1/n}\right )\), where \(X_1,\ldots ,X_n\) are i.i.d.]

Let \(\mathbf {X}_n:=(X_{1,n},\ldots ,X_{d,n})\) be a random variable in \(\mathbf {R}^d\) for every \(n\geq 1\). Let \(\mathbf {Y}:=(Y_1,\ldots ,Y_d)\) be another random variable in \(\mathbf {R}^d\). Prove that \(\mathbf {X}_n\Rightarrow \mathbf {Y}\) if and only if the 1-dimensional random variable \(\sum _{i=1}^{d}\lambda _iX_{i,n}\) converges weakly to \(\sum _{i=1}^{d}\lambda _iY_i\) for every \(\lambda :=(\lambda _1,\ldots ,\lambda _d)\in \mathbf {R}^d\).

Suppose that, for every integer \(n\geq 1\), \(X(n)\) has a Gamma distribution with parameters \(n+1\) and 1; that is, the probability density function of \(X(n)\) is \[ f_n(x):=\frac {x^ne^{-x}}{n!}\mathbf {1}_{(0,\infty )}(x). \]

(a) Compute \(\mathbb {E}(X(n))\) and \(\operatorname {Var}(X(n))\) for every integer \(n\geq 1\).

(b) Prove that, as \(n\to \infty \), \[ \frac {X(n)-n}{\sqrt {n}}\Rightarrow \mathrm {N}(0,1). \]

Let \(X\) be an integer-valued random variable that is non-degenerate; this means that there exist at least two integers \(n,m\) such that \(\mathbb {P}\{X=n\}\) and \(\mathbb {P}\{X=m\}\) are both strictly positive. Prove that the set \[ \mathcal {V}:=\{t\in \mathbf {R}:|\mathbb {E}[e^{itX}]|=1\} \] has zero Lebesgue measure. [Hint: If \(t\in \mathcal {V}\) then \(Z:=e^{it(X-Y)}\) has mean one, where \(Y\) is an independent copy of \(X\). Proceed by proving that \(\mathbb {E}(|Z-1|^2)=0\).]

Let \(X_1,\ldots ,X_n\) be independent identically distributed Poisson random variables for each \(n\) with parameter \(\lambda _n\).

(a) Show that \(X_1+\cdots +X_n\) is asymptotically normal if and only if \(n\lambda _n\to \infty \).

(b) Can you weaken the condition that the X’s are identically distributed for each \(n\)?

Let \(X\) and \(Y\) be integrable random variables on \((\Omega ,\mathcal {F},P)\), and let \(\mathcal {G}\) be a sub-\(\sigma \)-algebra of \(\mathcal {F}\). If \(X=Y\) on \(G\in \mathcal {G}\), show that \(\mathbb {E}[X\mid \mathcal {G}]=\mathbb {E}[Y\mid \mathcal {G}]\) a.s. on \(G\).

Let \(X_1,\ldots ,X_n\) be independent identically distributed Poisson random variables for each \(n\) with parameter \(\lambda _n\).

(a) Show that \(X_1+\cdots +X_n\) is asymptotically normal if and only if \(n\lambda _n\to \infty \).

(b) Can you weaken the condition that the X’s are identically distributed for each \(n\)?

Let \(X\) and \(Y\) be integrable random variables on \((\Omega ,\mathcal {F},P)\), and let \(\mathcal {G}\) be a sub-\(\sigma \)-algebra of \(\mathcal {F}\). If \(X=Y\) on \(G\in \mathcal {G}\), show that \(\mathbb {E}[X\mid \mathcal {G}]=\mathbb {E}[Y\mid \mathcal {G}]\) a.s. on \(G\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ h(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {5}{2}\theta ^{-5}t^4, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find a moment estimator for \(\theta \).

Let \(X\) and \(Y\) be two independent random variables with density functions \[ f(t)= \begin {cases} 0, & \text {if } t\notin [0,4],\\ 1/4, & \text {if } 0\leq t\leq 4, \end {cases} \] and \[ h(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ 2e^{-2t}, & \text {if } 0\leq t<\infty . \end {cases} \] Compute the density of \(X-Y\).

In this problem, all random variables are nonnegative. We say that \(X\) is stochastically dominated by \(Y\) if \(\mathbb {P}\{X>a\}\leq \mathbb {P}\{Y>a\}\) for all \(a>0\). Prove that if \(X\) is stochastically dominated by \(Y\), then \(\mathbb {E}\Phi (X)\leq \mathbb {E}\Phi (Y)\) for all increasing functions \(\Phi :\mathbf {R}_+\to \mathbf {R}_+\).

Suppose \(X_1,X_2,\ldots \) are independent, identically-distributed exponential random variables with mean \(\lambda >0\). Prove that \[ \max (X_1\ldots ,X_n)-\frac {1}{\lambda }\ln n\Rightarrow X, \] and compute \(\mathbb {P}\{X>x\}\) for all \(x>0\).

Give a rigorous proof that \(\mathbb {E}[XY]=\mathbb {E}[X]\mathbb {E}[Y]\) if \(X\) and \(Y\) are independent random variables belonging to \(L^1(\mathbb {P})\). In particular, show that \(XY\in L^1(\mathbb {P})\).

Fix \(n\geq 2\) and let \(X,Y_1,\ldots ,Y_n\) be jointly distributed random variables. We say that \(Y_1,\ldots ,Y_n\) are conditionally i.i.d. given \(X\) if \[ \mathbb {P}(Y_1\leq y_1,\ldots ,Y_n\leq y_n\mid X) =\mathbb {P}(Y_1\leq y_1\mid X)\cdots \mathbb {P}(Y_1\leq y_n\mid X) \]

for all \(y_1,\ldots ,y_n\). Show that, if \(Y_1,\ldots ,Y_n\) are conditionally i.i.d. given \(X\), then \[ \operatorname {Var}(Y_1+\cdots +Y_n)=n^2\operatorname {Var}(Y_1)-n(n-1)\mathbb {E}[\operatorname {Var}(Y_1\mid X)]. \]

Let \(Z\) be a random variable with all moments finite. Choose \(X\) and \(Y\) appropriately as in the Cauchy–Schwarz inequality or the Hölder inequality to prove that \(f(p):=\ln \mathbb {E}[|Z|^p]\) is convex on \((0,\infty )\).

Let \(X\) be an exponential random variable with mean 1, that is, \(\mathbb {P}(X>u)=e^{-u}\) for all \(u>0\). Evaluate \(\mathbb {E}[X\mid X\wedge t]\) and \(\mathbb {E}[X\mid X\vee t]\) for all \(t>0\). Here \(a\wedge b:=\min (a,b)\) and \(a\vee b:=\max (a,b)\).

Let \(X\) be a random variable with values in an interval \(I\), and suppose that \(f\) and \(g\) are nondecreasing functions on \(I\) such that \(f(X)\) and \(g(X)\) have finite variance. Show that \(\operatorname {Cov}(f(X),g(X))\geq 0\).

Hint: If \(X_1\) and \(X_2\) are i.i.d. as \(X\), then \((f(X_1)-f(X_2))(g(X_1)-g(X_2))\geq 0\).

The Cauchy distribution has density \(f(x):=1/[\pi (1+x^2)]\) for \(-\infty <x<\infty \) and characteristic function \(\varphi (t):=e^{-|t|}\) for \(-\infty <t<\infty \). Let \(X_1,X_2,\ldots \) be i.i.d. Cauchy, put \(S_n:=X_1+\cdots +X_n\) for each \(n\geq 1\). Show that \(S_n/n\) does not converge almost surely or in probability, but does converge in distribution.

Let \(X_1,X_2,\ldots \) be a sequence of i.i.d. random variables, each taking the values 0 and 1 with equal probabilities \(1/2\). Define \[ U:=\sum _{n=1}^{\infty }\frac {X_n}{2^n}. \] Prove that \(U\) is distributed uniformly on \((0,a)\) and compute \(a\).

Let \(X_1\) and \(X_2\) be independent, each with probability density function \(f(x)=1/x^2\), \(x>1\). Find the joint probability density function of \(Y_1=X_1X_2\) and \(Y_2=X_1/X_2\). It is important to state where your formula is valid. (If you have time, check that your joint pdf integrates to 1; if it doesn’t, find the error.)

Let \(X_1,\ldots ,X_n\) be a random sample from \(\operatorname {Poisson}(\theta )\).

(a) Find a complete, sufficient statistic.

(b) Noting that \(\Prob _\theta (X_1=1)=\theta e^{-\theta }\), use the Rao–Blackwell and Lehmann–Scheffé theorems to find a UMVUE of \(\tau (\theta ):=\theta e^{-\theta }\).

Let \(X_1,X_2,\ldots \) be i.i.d. with mean \(\mu \) and variance \(\sigma ^2\), and put \(S_0:=0\) and \(S_n:=X_1+\cdots +X_n\) for each \(n\geq 1\). Let \(N\) be a nonnegative-integer-valued random variable independent of \(X_1,X_2,\ldots \). Find the mean and variance of \(S_N\) by conditioning on \(N\).

(a) Find a closed-form expression (i.e., one with no summation sign) for \(\mathbb {E}[(X+1)^{-1}]\) when \(X\) is binomial\((n,p)\) with \(n\geq 1\) and \(0<p<1\).

(b) Find a closed-form expression for \(\mathbb {E}[(X+1)^{-1}]\) when \(X\) is Poisson\((\lambda )\) with \(\lambda >0\). You may use the result of part (a) here.

For \(n=1,2,\ldots \), let \(X_n\) have density \[ f_n(x):=\frac {n}{\pi (1+n^2x^2)},\qquad -\infty <x<\infty . \] (a) Does \(X_n\) converge to 0 in probability?

(b) Does \(X_n\) converge to 0 in \(L^1(\mathbb {P})\)?

(c) If \(X_1,X_2,\ldots \) are independent, does \(X_n\) converge to 0 a.s.?

Let \(X_1,X_2,X_3,Y_1,Y_2,Y_3\) be independent \(N(0,1)\) random variables. Let \(P_1\), \(P_2\), and \(P_3\) be the points in the plane with coordinates \((X_1,Y_1)\), \((X_2,Y_2)\), and \((X_3,Y_3)\). Let \(M\) be the midpoint of the segment \(P_1P_2\) and let \(r\) be half its length. Show that the probability that the point \(P_3\) lies within the circle of radius \(r\) centered at \(M\) is equal to \(1/4\).

Choose and fix a sequence of real numbers \(a_1,a_2,\ldots \) such that: (i) \(0<a_k<1\) for all \(k\geq 1\); and (ii) \(\sum _{k=1}^{\infty }a_k^2=\infty \). Let \(X_1,X_2,\ldots \) be independent random variables, where each \(X_k\) is distributed uniformly on \((-a_k,a_k)\). Prove that \(n^{-1/2}(X_1+\cdots +X_n)\) converges weakly to \(N(0,\sigma ^2)\) and compute \(\sigma \).

Suppose \(f\) is a probability density function on \(\mathbf {R}\), and define for all \(x,y\in \mathbf {R}\), \[ g(x,y):= \begin {cases} 2f(x)f(y), & \text {if } x\leq y,\\ 0, & \text {otherwise.} \end {cases} \] (a) Prove that \(g\) is the probability density function of some random variable \((X,Y)\) which takes values in \(\mathbf {R}^2\).

(b) Are \(X\) and \(Y\) independent? Prove or disprove.

Suppose \(X_1,X_2,\ldots \) are independent, identically-distributed random variables such that \[ \mathbb {P}\{X_1>x\}= \begin {cases} x^{-\alpha }, & \text {if } x\geq 1,\\ 1, & \text {if } x<1, \end {cases} \] where \(\alpha >0\) is fixed. Prove that \[ n^{-1/\alpha }\max (X_1,\ldots ,X_n)\Rightarrow X, \] and compute \(\mathbb {P}\{X>x\}\) for all \(x\in \mathbf {R}\).

Let \(\alpha ,\lambda \) be fixed, strictly positive, and finite. The Gamma\((\alpha ,\lambda )\) density function is \[ f(x):= \begin {cases} \dfrac {\lambda ^\alpha }{\Gamma (\alpha )}x^{\alpha -1}e^{-\lambda x}, & \text {if } x>0,\\ 0, & \text {if } x\leq 0, \end {cases} \] where \(\Gamma (\alpha ):=\int _0^\infty t^{\alpha -1}e^{-t}\,dt\). This defines a probability density function.

(a) Compute the characteristic function of \(f\).

(b) Suppose \(X_1,\ldots ,X_n\) are independent with common density Gamma\((1,\lambda )\). Prove that \(S_n:=X_1+\cdots +X_n\) has a Gamma density; also compute its parameters.

(c) Compute \(\mathbb {P}\{S_n<k<S_{n+1}\}\) for all integers \(k\geq 1\).

Let \(\Phi \) and \(\phi \) denote the standard normal distribution and density functions.

(a) Prove that for all \(x>0\) \[ 1-\Phi (x)\leq \frac {1}{x}\phi (x). \] (5 points)

Let \(X_{1,n}<X_{2,n}\leq \cdots \leq X_{n,n}\) denote the order statistics from a sample of \(n\) independent identically distributed standard normal random variables.

(b) Show that \(X_{n,n}\) converges in probability to infinity. (5 points)

(c) Show that for all \(\epsilon >0\) \[ \lim _{n\to \infty } \mathbb {P}\left \{X_{n,n}\leq \sqrt {(2+\epsilon )\log n}\right \}=1. \] (5 points)

Let \(X_2,X_3,\ldots \) be independent random variables such that \[ \mathbb {P}(X_n=n)=\mathbb {P}(X_n=-n)=\frac {1}{2n\log n}, \qquad \mathbb {P}(X_n=0)=1-\frac {1}{n\log n}. \] Show that this sequence obeys the weak law but not the strong law of large numbers, in the sense that \(n^{-1}\sum _{i=1}^n X_i\) converges to 0 in probability but not almost surely.

Let \(\{X_n,n\geq 1\}\) be independent \(N(0,1)\) random variables. Show that \[ \Prob \left (\limsup \frac {|X_n|}{\sqrt {\log n}}=\sqrt {2}\right )=1. \] Hint: \(1-\Phi (x)\) is asymptotic to \(\phi (x)/x\) as \(x\to \infty \).

Let \(X_1,X_2,\ldots \) be independent random variables such that \[ X_n= \begin {cases} 1 & \text {with probability }(2n)^{-1},\\ 0 & \text {with probability }1-n^{-1},\\ -1 & \text {with probability }(2n)^{-1}. \end {cases} \] Let \(Y_1:=X_1\) and for \(n\geq 2\) \[ Y_n:= \begin {cases} X_n & \text {if }Y_{n-1}=0,\\ nY_{n-1}|X_n| & \text {if }Y_{n-1}\neq 0. \end {cases} \] Show that \(Y_n\) is a martingale with respect to \(\mathcal {F}_n:=\sigma (Y_1,Y_2,\ldots ,Y_n)\). Show that \(Y_n\) does not converge almost surely. Does \(Y_n\) converge in any way? Why does the martingale convergence theorem not apply?

A die is thrown 10 times. What is the probability that the sum of the scores in 27?

Hint: Use probability generating functions.

Suppose \(W:=(X,Y)\) is a two-dimensional random variable and \(QW^T\) is \(N(0,1)\) for all \(2\times 2\) rotation matrices \[ Q:= \begin {pmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end {pmatrix}. \] Prove that \(X\) and \(Y\) are independent and \(N(0,1)\).

Let \(X_1,X_2,\ldots \) and \(Y_1,Y_2,\ldots \) denote random variables on a common probability space. Suppose \(X_n\) converges to 0 in probability as \(n\to \infty \), and \(Y_n\) converges to \(Y\) weakly. Prove that \((X_n,Y_n)\) converges weakly to \((0,Y)\).

Construct two random variables \(X\) and \(Y\) such that:

(a) \(X\) is \(N(0,1)\),

(b) \(Y\) is \(N(0,1)\), and

(c) \((X,Y)\) is not a Gaussian random vector.

Suppose \(X\) and \(Y\) are independent bounded random variables. Prove in complete detail the following assertions:

(a) \(\mathbb {E}(XY)=\mathbb {E}(X)\mathbb {E}(Y)\),

(b) \(\mathbb {E}(X\mid Y)=\mathbb {E}(X)\).

Let \(X_i\) be a random sample of size \(n\) with pdf \(f(x;\eta )=e^{-(x-\eta )}\) (\(x>\eta \), \(\eta >0\)).

1a) (5 points) Find the MLE \(\hat \eta \) and compute its pdf. Do you recognize it?

1b) (5 points) Is \(\hat \eta \) unbiased, asymptotically unbiased, MSE consistent, simple consistent?

Let \(X_i\) be a random sample of size \(n\) with pdf \(f(x;\theta )=\frac {1}{\theta }e^{-(x-\theta )/\theta }\) (\(x>\theta \), \(\theta >0\)).

2a) (5 points) What is the MME of \(\theta \)? Is it a reasonable estimator?

2b) (5 points) What is the MLE of \(\theta \)?

Let \(X\) and \(Y\) be independent standard normals and consider \((U,V)=(aX+bY,cX+dY)\). Assume that at least one of \(a\) and \(b\) is nonzero, and at least one of \(c\) and \(d\) is nonzero. (Otherwise either \(U\) or \(V\) would be 0 and the problem is not interesting.)

5a) (5 points) Show that if \(ad=bc\), then \(U\) and \(V\) cannot be independent random variables.

5b) (5 points) Assume \(ad\ne bc\) and find the joint pdf of \((U,V)\).

5c) (5 points) Find a necessary and sufficient condition for \(U\) and \(V\) to be independent.

(5 points) Let \(\zeta _n\) and \(\eta _n\) be two sequences of random variables. Prove or give counterexamples to the following statements:

9a) If \(\zeta _n\xrightarrow {(d)}\zeta \) and \(\eta _n\xrightarrow {(d)}\eta \), then \(\zeta _n+\eta _n\xrightarrow {(d)}\zeta +\eta \).

9b) If \(\zeta _n\xrightarrow {P}\zeta \) and \(\eta _n\xrightarrow {P}\eta \), then \(\zeta _n+\eta _n\xrightarrow {P}\zeta +\eta \).

9c) If \(\zeta _n\xrightarrow {(d)}\zeta \) and \(\eta _n\xrightarrow {(d)}\eta \), then \(\zeta _n+\eta _n\xrightarrow {P}\zeta +\eta \).

(\(\xrightarrow {(d)}\) and \(\xrightarrow {P}\) denote convergence in distribution and in probability, respectively.)

Let \(X\) and \(Y\) have joint density \[ f(x,y)=g\left (\sqrt {x^2+y^2}\right ),\qquad (x,y)\in \mathbb {R}^2, \] for some function \(g\). Show that \(Z=X/Y\) has a Cauchy density.

Construct three random variables \(X,Y,Z\) such that \[ \mathbb {E}[\mathbb {E}[X\mid Y]\mid Z]\neq \mathbb {E}[\mathbb {E}[X\mid Z]\mid Y] \] with positive probability.

Let \(X_1,X_2,\ldots \) be a sequence of integrable random variables defined on the same probability space. Show that \(\{X_n\}_{n\geq 1}\) is a submartingale if and only if \(\mathbb {E}[X_T]\geq \mathbb {E}[X_1]\) for all bounded stopping times \(T\). (Take the filtration to be the natural one, \(\mathcal {F}_n=\sigma (X_1,\ldots ,X_n)\).)

Let \(X_1,X_2,\ldots ,X_n\) be i.i.d. positive random variables. Show that \[ \mathbb {E}\left [ \frac {\sum _{i=1}^m X_i}{\sum _{i=1}^n X_i} \right ] = \frac {m}{n},\qquad 1\leq m\leq n. \]

Show that the number of fixed points in a random permutation of length \(n\) is asymptotically Poisson with mean 1. Hint: Use characteristic functions. Write \(e^{it1_A}\) as \(1+(e^{it}-1)1_A\).

Let \(X\) and \(Y\) be independent random variables with density function \(f\) and \(g\). Assume that \(Y>0\). Show that the function \[ h(t)=\int _{0}^{\infty } z f(zt)g(z)\,dz \] can be chosen as the density of \(Z=X/Y\). (7 points)

Suppose \(X_1, \ldots , X_n\) is an i.i.d. sample from the exponential density with mean \(\theta \), where \(\theta > 0\) is unknown. Find an estimator of \(\theta \) based on \(X_1, \ldots , X_n\). Is your estimator complete? Is it sufficient? Justify your reasoning.

Let \(\{X_i\}_{i=1}^{\infty }\) be a sequence of i.i.d. random variables, and define \[ S(r):=\sum _{i=1}^{\infty }(1-r)^iX_i \qquad \text {for all } 0<r<1. \]

(a) Prove that for every fixed \(r\in (0,1)\), \(S(r)\) is a.s. absolutely convergent if and only if \(\mathbb {E}\{\log (1+|X_1|)\}<\infty \).

(b) Suppose that \(\mathbb {E}[X_1]=\mu \) and \(\operatorname {Var}(X_1)=\sigma ^2<\infty \). Then prove that there exists a constant \(c\) such that \(rS(r)\to c\) in probability, as \(r\to 0\). Compute \(c\).

Let \(\{X_n\}_{n=1}^{\infty }\) be a sequence of independent (but not identically distributed) random variables, such that for all \(n\ge 1\), \[ \mathbb {P}\{X_n=1\}=\mathbb {P}\{X_n=-1\}=\frac {1}{2}-\frac {1}{2n^2}, \qquad \mathbb {P}\{X_n=n\}=\mathbb {P}\{X_n=-n\}=\frac {1}{2n^2}. \] Prove that \(n^{-1/2}\sum _{i=1}^{n}X_i\Rightarrow N(0,\sigma ^2)\), and compute \(\sigma ^2\) explicitly.

Let \(\{X_i\}_{i=1}^{\infty }\) denote a sequence of i.i.d. random variables with \(\mathbb {P}\{X_1=0\}=p\) and \(\mathbb {P}\{X_1=1\}=q:=1-p\). Consider the random sequence \(X_1X_2\cdots \), and compute \(\mathbb {E}[T]\), where \(T\) denotes the first time that the pattern “001” appears in the said sequence.

Construct an example of a sequence \(\{A_n\}_{n=1}^{\infty }\) such that:

(a) (i) \(\sum _n\mathbb {P}(A_n)=\infty \); and (ii) only finitely-many of the \(A_n\)’s occur with positive probability.

(b) (i) \(\sum _n\mathbb {P}(A_n)=\infty \); and (ii) infinitely-many of the \(A_n\)’s occur with positive probability.

Let \(X\) and \(Y\) be two standard-normal random variables.

(a) Prove that if \((X,Y)\) is Gaussian then \(X\) and \(Y\) are independent.

(b) Construct an example wherein \((X,Y)\) is not a (two-dimensional) Gaussian random variable.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ g(t;\theta )= \begin {cases} 0, & \text {if } -\infty <t<\theta ,\\ e^{-(t-\theta )}, & \text {if } \theta \leq t<\infty . \end {cases} \] We wish to test \(H_0:\theta \leq \theta _0\) against \(H_A:\theta >\theta _0\).

(a) Find the uniformly most powerful test of size \(\alpha \).

(b) Compute the power function.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with probability mass function \[ f(t,\theta )=\theta (1-\theta )^{t-1}I\{t=1,2,\ldots \}\qquad (0<\theta <1). \] Find the uniformly minimum variance unbiased estimator for \(\theta \).

Let \(X\) and \(Y\) be independent random variables with densities, \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-2,2],\\ 1/4, & \text {if } t\in [-2,2], \end {cases} \] and \[ f(t)= \begin {cases} 0, & \text {if } t\notin [-1,1],\\ 1/2, & \text {if } t\in [-1,1]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with probability mass function \[ P\{X_i=1\}=\theta \quad \text {and}\quad P\{X_i=0\}=1-\theta . \] Let: \[ \bar {X}_n=\frac {1}{n}\sum _{1\leq i\leq n} X_i \] and \(h(t)=t^3-t\). Find the sequences \(a_n\) and \(b_n\) such that \(a_n\bigl (h(\bar {X}_n)-h(\theta )-b_n\bigr )\) has a non-degenerate limit distribution.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with a continuous probability distribution function \(F\) and \(Y_1,Y_2,\ldots ,Y_m\) be independent, identically distributed random variables with a continuous probability distribution function \(H\). The two samples are independent and \(F=H\). Let \[ T=\sum _{1\leq i\leq n}\sum _{1\leq j\leq m} I\{X_i\leq Y_j\}. \] Compute \(E T\) and \(\operatorname {var}T\).

Suppose \(\{X_n\}_{n=1}^{\infty }\) is a sequence of random variables that has the property that \(\sup _n|X_n|\le 1\) a.s. Then use Doob’s decomposition to prove that \(\sum _n X_n\) converges a.s. iff \(\sum _n\mathbb {E}[X_n\mid X_1,\ldots ,X_{n-1}]\) converges a.s.

Let \(U_1,U_2,\ldots ,U_n\) be independent identically distributed random variables, uniformly distributed on \([0,1]\). Compute the covariance between \(U_{1,n}=\min _{1\leq i\leq n}U_i\) and \(U_{n,n}=\max _{1\leq i\leq n}U_i\).

Let \(X_1\) and \(X_2\) be two independent, identically distributed exponential (1) random variables. Compute the joint density of \(Y_1=X_1+X_2\) and \(Y_2=X_1/(X_1+X_2)\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed uniform on \([0,\theta ]\), \(\theta >0\), random variables. We wish to test \(H_0:\theta \leq \theta _0\) against \(H_A:\theta >\theta _0\). Find the uniformly most powerful test of size \(\alpha \) and compute the power function. (You must explain why your test has this property.)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed Poisson \((\theta )\) random variables. Find the uniformly minimum unbiased estimator for \(\eta =\theta ^2 e^{-\theta }\). (You must explain your answer.)

Let \(\xi _n\) and \(\eta _m\) be two sequences of random variables. Prove or give counterexamples to the following statements:

(a) If \(\xi _n\overset {d}{\to }\xi \) and \(\eta _m\overset {d}{\to }\eta \), then \(\xi _n+\eta _m\overset {d}{\to }\xi +\eta \).

(b) \(\xi _n\overset {P}{\to }\xi \) and \(\eta _m\overset {P}{\to }\eta \), then \(\xi _n+\eta _m\overset {P}{\to }\xi +\eta \).

(c) \((\xi _n,\eta _n)\overset {P}{\to }(\xi ,\eta )\) then \(\xi _n+\eta _n\overset {P}{\to }\xi +\eta \).

(\(\overset {d}{\to }\) and \(\overset {P}{\to }\) denote convergence in distribution and convergence in probability.)

A sequence \(X_1,X_2,\ldots \) of random variables is said to be uniformly integrable if \[ \lim _{c\to \infty }\sup _{n\ge 1}\mathbb {E}\{|X_n|;\ |X_n|>c\}=0. \] Prove that \(X_n\to X\) in \(L^1(\mathbb {P})\) if and only if both of the following happen: (i) \(X_n\to X\) in probability; and (ii) \(\{X_n\}_{n=1}^{\infty }\) is uniformly integrable.

Construct three random variables \(X,Y\), and \(Z\), such that any distinct pair of them are independent but \((X,Y,Z)\) are not independent.

Suppose \(\{X_n\}_{n=1}^{\infty }\) and \(\{Y_n\}_{n=1}^{\infty }\) are random variables on the same probability space \((\Omega ,\mathcal {F},\mathbb {P})\). Suppose also that \(X_n\Rightarrow X\) and \(Y_n\Rightarrow Y\). True or false: \((X_n,Y_n)\Rightarrow (X,Y)\). [If true, then prove it. If false, then construct a counter-example.]

Consider two random variables \(X,Y\in L^2(\mathbb {P})\). Prove that if \(\mathbb {E}[X\mid Y]=Y\) and \(\mathbb {E}[X^2\mid Y]=Y^2\), then \(X=Y\) a.s.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed Poisson random variables with common probability mass function \[ p(k,\theta )=\frac {\theta ^k}{k!}e^{-\theta },\qquad k=0,1,\ldots \] Let \[ \tau =\theta e^{-\theta }. \] Find the uniformly minimum variance unbiased estimator for \(\tau \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with common density function \(f\). Let \[ \hat {f}_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ), \] where \(K\) is a symmetric density function and \(K(t)=0\) if \(|t|>1\). We assume that \(h=h(n)\to 0\) as \(n\to \infty \). We assume that the second derivative of \(f\) exists and continuous in a neighborhood of \(t\). We use \(\hat {f}\) to estimate \(f\). Compute the bias of \(\hat {f}_n(t)\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed Poisson random variables with common probability mass function \[ p(k,\theta )=\frac {\theta ^k}{k!}e^{-\theta },\qquad k=0,1,\ldots \] Using the Cramer-Rao inequality show that the sample mean is a uniformly minimum variance unbiased estimator for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed normal \(N(0,\sigma ^2)\) random variables. We wish to test \[ H_0:\sigma \leq \sigma _0 \] against \[ H_A:\sigma >\sigma _0. \]

1. Find the uniformly most powerful test of size \(\alpha \). (10 points)
2. Compute the rejection region as a function of \(\alpha \). (10 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed normal \(N(\mu ,\sigma ^2)\) random variables. Let \[ U_n=\frac {2}{n(n-1)} \sum _{1\leq i<j\leq n}I\{|X_i-X_j|>1\}. \]

1. Compute \(EU_n\). (10 points)
2. Assume that \(\sigma \) is known. We wish to test \(H_0:\mu =\mu _0\) (\(\mu _0\) is given) against the alternative that \(H_0\) is not true. Can we use \(U_n\) to test \(H_0\) against this alternative? (5 points)
3. Now \(\mu \) and also \(\sigma \) are unknown. We wish to test the null-hypothesis that \(\sigma =\sigma _0\) against the alternative that \(H_0\) is not true. Can we use \(U_n\) now? (5 points)

Suppose \(\{E_1,E_2,\ldots \}\) is a sequence of measurable (otherwise arbitrary) events. Suppose \(\{F_1,F_2,\ldots \}\) is another sequence of measurable events also totally independent of all of the \(E_i\)’s. Assume the following:

(a) \(\mathbb {P}(E_k\text {, infinitely often})=1\);

(b) there exists some \(p>0\), such that for all \(k\ge 1\), \(\mathbb {P}(F_k)\ge p\).

Prove: \(\mathbb {P}(E_k\cap F_k\text {, infinitely often})\ge p>0\). (10 points)

Suppose \(N\) is a positive random variable.

(a) Show that \[ \mathbb {P}(N>0)\ge \frac {(\mathbb {E} N)^2}{\mathbb {E} N^2}. \] (5 points)

(b) Let \(X_1,X_2,\ldots \) be i.i.d. random variables. Define \(S_n=X_1+\cdots +X_n\). Use part (a) to show that for any positive integer \(n\), \[ \mathbb {P}(S_j=0,\text { for some }1\le j\le n) \ge \frac {\sum _{j=1}^n\mathbb {P}(S_j=0)} {2\left (1+\sum _{j=1}^n\mathbb {P}(S_j=0)\right )}. \] (5 points)

Let \((\Omega ,\mathcal {F},P)\) be a probability space, let \(\mathcal {G}\) be a sub-\(\sigma \)-field of \(\mathcal {F}\), and let \(M\) be the closed subspace of \(L^2(\Omega ,\mathcal {F},P)\) consisting of the \(\mathcal {G}\)-measurable functions in \(L^2\). Show that \(T(X)=E[X\mid \mathcal {G}]\) coincides with the orthogonal projection of \(L^2(\Omega ,\mathcal {F},P)\) onto \(M\).

(a) Let \(X_1,X_2,\ldots \) be i.i.d. Poisson random variables with mean 1, and let \(Z\) be \(N(0,1)\). Denote \(a^-=-\min \{a,0\}\), and prove that \[ \text {(*)}\qquad \mathbb {E}\left [\left (\frac {S_n-n}{\sqrt n}\right )^-\right ]\to \mathbb {E}[Z^-]. \]

(b) Evaluate both expectations in (*) explicitly, and, noting the telescoping sum, deduce Stirling’s formula, for \(n!\).

Let \((\Omega ,\mathcal F,P)\) be a probability space, let \(\mathcal G\) be a sub-\(\sigma \)-field of \(\mathcal F\), and let \(M\) be the closed subspace of \(L^2(\Omega ,\mathcal F,P)\) consisting of the \(\mathcal G\)-measurable functions in \(L^2\). Show that \(T(X)=\mathbb {E}[X\mid \mathcal G]\) coincides with the orthogonal projection of \(L^2(\Omega ,\mathcal F,P)\) onto \(M\).

Let \(X_1,\ldots ,X_n\) be an i.i.d. sample which is uniformly chosen from the interval \((\theta _1,\theta _2)\).

(a) Prove that there are no one-dimensional (i.e., not vector-valued) sufficient statistics for \(h(\theta _1,\theta _2)\) where \(h\) is a one-to-one measurable function.

(b) Is there a one-dimensional (i.e., not vector-valued) sufficient statistics for \(\mu =(\theta _1+\theta _2)/2\)?

Let \((\Omega ,\mathcal {A},P)\) be a probability space and \(\{B(t),0\leq t\leq 1\}\) be a Brownian motion on it. Since almost all sample paths of \(B\) are continuous, \(\int _0^t B(t)\,dt\) can be defined as a usual Riemann integral. Compute the distribution of \(\int _0^1 B(t)\,dt\).

Suppose \(X\sim \operatorname {POISS}(\theta )\). Put a \(\operatorname {GAMMA}(\alpha ,\beta )\) prior on \(\theta \) and suppose we have the following loss function: \[ \ell (\theta ,a)=\frac {(a-\theta )^2}{\theta }. \] Find the Bayes’ estimator of \(\theta \).

Let \((\Omega ,\mathcal A,P)\) be a probability space and \(\{B(t),0\le t\le 1\}\) be a Brownian motion on it. Since almost all sample paths of \(B\) are continuous, \(\int _0^t B(t)dt\) can be defined as a usual Rieman integral. Compute the distribution of \(\int _0^1 B(t)dt\).

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables. Show that the following statements are equivalent:

(a) \(\mathbb {E}|X_1|^\nu <\infty \);

(b) \(X_n/n^{1/\nu }\to 0\), almost surely;

(c) \(\max _{1\le i\le n}|X_i|/n^{1/\nu }\to 0\), almost surely.

Suppose \(X\sim \operatorname {Poisson}(\theta )\). Put a \(\operatorname {Gamma}(\alpha ,\beta )\) prior on \(\theta \) and suppose we have the following loss function: \(\ell (\theta ,a)=(a-\theta )^2/\theta \). Find the Bayes’ estimator of \(\theta \).

Let \(X_1,X_2,\ldots \) be independent Poisson random variables with \(EX_n=\lambda _n\) and let \(S_n=X_1+\cdots +X_n\). Show that if \(\sum \lambda _n=\infty \), then \(S_n/ES_n\to 1\) a.s.

Hint: Show that (*) \(Y_n/c_n\to 1\) a.s., provided that \(Y_n\geq 0\) is nondecreasing in \(n\), and (*) holds for a subsequence \(n(k)\) that has \(c_{n(k+1)}/c_{n(k)}\to 1\).

Let \(X_1,X_2,\ldots \) be independent Poisson random variables with \(\mathbb {E} X_n=\lambda _n\), and let \(S_n=X_1+\cdots +X_n\). Show that if \(\sum \lambda _n=\infty \), then \(S_n/\mathbb {E} S_n\to 1\) a.s.

Hint: Show that (*) \(Y_n/c_n\to 1\) a.s., provided that \(Y_n\ge 0\) is nondecreasing in \(n\), and (*) holds for a subsequence \(n(k)\) that has \(c_{n(k+1)}/c_{n(k)}\to 1\).