Uniform Integrability and Truncation Methods

Let \(q\) be an integrable random variable. Prove that for each \(\epsilon > 0\) we can pick \(\delta > 0\) so that \(\mu (A) < \delta \) implies \[ \int _{A}|q|\,d\mu < \epsilon . \] Let \(f \ge 0\). (Hint: Show first that for any \(f \ge 0\), \(\int f \wedge n\,d\mu \nearrow \int f\,d\mu \) as \(n \to \infty \).)

At time \(n = 0\), an urn contains \(B_{0} = 1\) blue, and \(R_{0} = 1\) red ball. At each time \(n = 1, 2, 3, \ldots \), a ball is chosen at random from the urn and returned to the urn, together with a new ball of the same color. Let \(B_{n}\) and \(R_{n}\) be the number of blue, respectively, red balls in the urn after the \(n\)-th turn of this procedure. (Note that

Denote by \(\mathcal {F}_{n} = \sigma (B_{j}, 0 \le j \le n) = \sigma (R_{j}, 0 \le j \le n)\), \(n \ge 0\), the

natural filtration of the process. Let

be the proportion of blue balls in the urn just after time \(n\).

1. Show that \((M_{n})_{n \ge 0}\) is a martingale in the filtration \((\mathcal {F}_{n})_{n \ge 0}\).
2. Show that \(P(B_{n} = k) = 1/(n + 1)\) for all integers \(k \in [1, n + 1]\). (Hint: Write down the probability of choosing \(k\) blue and \(n - k\) red balls in whatever fixed order [hence, ending up with \(k + 1\) blue and \(n - k + 1\) red balls].)
3. Show that \(M_{\infty } = \lim _{n \to \infty } M_{n}\) exists almost surely and compute its distribution. (Hint: use part (b).)

Let \((a_{n})_{n \ge 1}\) be a nondecreasing sequence of positive numbers such that \[ \lim _{n \to \infty }\frac {a_{n}^{2}}{\sum _{k=1}^{n}a_{k}^{2}} = 0. \]

Let \((X_{n})_{n \ge 1}\) be a sequence of i.i.d. random variables with mean 0 and variance 1. Prove that \[ \frac {\sum _{k=1}^{n} a_{k}X_{k}}{\sqrt {\sum _{k=1}^{n}a_{k}^{2}}} \]

converges weakly to the standard normal distribution.

Let \(A_{n}\) be a sequence of independent events with \(P(A_{n}) < 1\) for all \(n\). Show that \[ P\left (\bigcup _{n} A_{n}\right ) = 1 \quad \text {implies} \quad \sum _{n} P(A_{n}) = \infty . \]

Let \(X_{1}, X_{2}, \ldots \) be i.i.d. with characteristic function \(\phi \).

1. Show that if \(\phi '(0) = ia\) and \(S_{n} = X_{1} + \cdots + X_{n}\), then \(S_{n}/n \to a\) in probability. (Hint: first show that if \(c_{n} \to c \in \mathbb {C}\), then \((1 + c_{n}/n)^{n} \to e^{c}\).)
2. Show that if \(S_{n}/n \to a\) in probability, then \(\phi (t/n)^{n} \to e^{iat}\) as \(n \to \infty \) through the integers.

Let \(Y_{1}, Y_{2}, \ldots \) be nonnegative i.i.d. random variables with \(\mathbb {E}[Y_{m}] = 1\) and \(P(Y_{m} = 1) <\) 1.

1. Show that \(X_{n} = \prod _{m \le n}Y_{m}\) defines a martingale.
2. Show that \(X_{n} \to 0\) a.s. (Hint: note that \(|X_{n+1} - X_{n}| = X_{n}|Y_{n+1} - 1|\).)
3. Use the strong law of large numbers to conclude \(\frac {1}{n}\log (X_{n}) \to c < 0\).

Let \(\{A_{i}\}_{i=1}^{\infty }\) be a sequence of independent events such that \(\sum _{i=1}^{\infty } P(A_{i}) = \infty \). Prove that the \(A_{n}\) occur infinitely often with probability 1 (the converse of the Borel-Cantelli Lemma), i.e. that \(\sum _{i=1}^{\infty }\mathbf {1}_{A_{i}} = \infty \) almost surely.

Show that if \(X_{n} \to X\) in probability, then \(X_{n} \Rightarrow X,\) i.e. \(X_{n}\) converges weakly to \(X\).

Recall the inversion formula: if \(\mu \) is a probability measure and \(\widehat {\mu }\) is its characteristic function, then for all \(a < b\):

1. Prove that if \(\widehat {\mu } \in L^{1}\), then \(\mu (\{a\}) = 0\) for all \(a \in \mathbb {R}\) and \(\mu \) has a probability density function given by

   \[ f(y) = \frac {1}{2\pi }\int _{-\infty }^{\infty } e^{-ity}\widehat {\mu }(t)\,dt. \]

2. Show that if \(X_{1}, \ldots , X_{n}\) are independent and uniformly distributed on \((-1, 1)\), then for \(n \ge 2\), \(X_{1} + \cdots + X_{n}\) has density

   \[ f(y) = \frac {1}{\pi }\int _{0}^{\infty } \left (\frac {\sin t}{t}\right )^{n}\cos (ty)\,dt. \]

Prove that if \(\{X_{i}\}_{i=1}^{\infty }\) are i.i.d. Uniform-\([0,1]\) random variables, then \[ \frac {4\sum _{i=1}^{n} iX_{i} - n^{2}}{n^{3/2}} \] converges weakly, and identify the limiting distribution. (Recall \(\sum _{i=1}^{n} i = \frac {n(n+1)}{2}\) and \(\sum _{i=1}^{n} i^{2} = \frac {n(n+1)(2n+1)}{6}\).)

Suppose \(\{r_{k}\}_{k \in \mathbb {Z}_{+}}\) is a sequence of real numbers such that \(n^{-1}\sum _{k=1}^{n}|r_{k}| \to 0\) as \(k \to \infty \). Let \(\{X_{k}\}_{k \in \mathbb {N}}\) be a sequence of \(L^{2}\) random variables such that \(\mathbb {E}[X_{k}] = 0\) and \(\mathbb {E}[X_{k}X_{\ell }] \le \) \(r_{\ell -k}\) for all \(k \le \ell \). Let \(S_{n} = X_{1} + \cdots + X_{n}\). Show that \(n^{-1}S_{n}\) converges in \(L^{2}\) as \(n \to \infty \) and identify the limit.

Answer the following two questions.

1. Let \(\{X_{n}\}_{n \in \mathbb {N}}\) be independent, almost surely finite random variables. Show that \[ \sup _{n} X_{n} < \infty \text { a.s.} \] if, and only if, \(\exists c < \infty \) such that \[ \sum _{n} P(X_{n} > c) < \infty . \]
2. Let \(\{A_{n}\}_{n \in \mathbb {N}}\) be a sequence of events with \(P(A_{n}) \ge \delta \) for each \(n\). Show that \[ P(A_{n}\text { happens infinitely often}) \ge \delta . \]

Let \(u\) be an integrable Borel function on \([0, 1)\) relative to the Lebesgue measure. For each \(n \ge 1\) and \(x \in [0, 1)\), let \(I_{n}(x) = [k2^{-n}, (k + 1)2^{-n})\) be the interval that contains \(x\) as \(k\) varies from 0 to \(2^{n} - 1\). Show that for Lebesgue-a.e. \(x\), \[ \lim _{n \to \infty } 2^{n}\int _{I_{n}(x)} u(y)\,dy = u(x). \]

(Hint: Cast this in a suitable martingale framework then use the result of problem 3.)

Fix an integer \(n \ge 1\). Let \(\{Y_{k}\}_{k \in \mathbb {N}}\) be i.i.d. uniform on \(\{1, 2, \ldots , n\}\). Let

be the first time we get a repeated sample. Find an exponent \(\gamma \) and a nondegenerate distribution \(\mu \) such that, as \(n \to \infty \), \(n^{-\gamma }\sigma _{n}\) converges weakly to the distribution \(\mu \). (Hint: look at tail probabilities \(P(n^{-\gamma }\sigma _{n} \ge x)\).)

Give an example of a sequence of independent random variables \(\{Z_{k}\}_{k \in \mathbb {N}}\) satisfying \(\mathbb {E}[Z_{k}] = 0\), \(\operatorname {Var}(Z_{k}) = 1\), for each \(k\), and such that \((Z_{1} + \cdots + Z_{n})/\sqrt {n}\) does not converge in distribution to a standard normal. Justify any claims you make.

Let \(\{X_{n}\}_{n \in \mathbb {N}}\) be independent random variables such that for each \(n\), \(X_{n}\) takes the values \(+1\) and \(-1\) with probability \((1 - 2^{-n})/2\) each and the value \(2^{k}\) with probability \(2^{-k}\), for integers \(k > n\).

1. Show that \(\mathbb {E}[|X_{n}|] = \infty \) for all \(n \in \mathbb {N}\).
2. Show that \((X_{1} + \cdots + X_{n})/\sqrt {n}\) converges in distribution to a standard normal. (Hint: Check that we can write \(X_{n} = (1 - B_{n})Y_{n} + 2^{n}B_{n}Z_{n}\), where \(B_{n}\) is \(\operatorname {Bernoulli}(2^{-n})\), \(Y_{n}\) takes values \(\pm 1\) equally likely, \(Z_{n}\) takes the value \(2^{k}\) with probability \(2^{-k}\), for \(k \in \mathbb {N}\), and \(\{B_{n}, Z_{n}, Y_{n} : n \ge 1\}\) are mutually independent. Now, \(X_{n}\) is a small perturbation of \((1 - B_{n})Y_{n}\). Work out the details.)

Let \(X\), \(Y\), \(Z\) be random variables on some probability space. Recall that

is the conditional expectation of the indicator function of the event \(\{X \in A\}\), given the \(\sigma \)-algebra \(\sigma \{Y, Z\}\) generated by \(Y\) and \(Z\). We say that \(X\) and \(Y\) are independent, given \(Z\), if for all Borel sets \(A, B \subset \mathbb {R}\),

Show that this condition is equivalent to having \(P(X \in A \mid Y, Z) = P(X \in A \mid Z)\) \(P\)-almost surely, for all Borel sets \(A\).

Let \(Y_{k}\) be i.i.d., \(Y_{k} \ge 0\), and \(\mathbb {E}[Y_{k}] \le 1\) for all \(k\). Prove the existence of the almost sure limit \(M_{\infty } = \lim _{n \to \infty } \prod _{i=1}^{n} Y_{i}\) and describe this random variable explicitly. Do not forget the degenerate cases.

Let \(\{\mathcal {F}_{n}\}_{n \ge 0}\) be a filtration on the probability space \((\Omega , \mathcal {F}, P)\), let \(X \in L^{1}(\Omega , \mathcal {F}, P)\), and define \(M_{n} = \mathbb {E}[X \mid \mathcal {F}_{n}]\). Let \(\tau \) be a stopping time and recall the definition \[ \mathcal {F}_{\tau } = \{A \in \mathcal {F} : A \cap \{\tau = n\} \in \mathcal {F}_{n}\ \forall n \ge 0\}. \]

1. Show that \(1_{\{\tau < \infty \}}M_{\tau }\) is integrable.
2. Show that \(\mathbb {E}[X \mid \mathcal {F}_{\tau }] = 1_{\{\tau < \infty \}}M_{\tau } + 1_{\{\tau = \infty \}}X\). Do not forget the necessary measurability condition.

Answer the following questions.

1. State the definition of convergence in probability. Using only this definition, show that \(X_{n} \to X\) in probability if and only if \(\mathbb {E}[1 \wedge |X_{n} - X|] \to 0\). Do not appeal to any theorems that trivialize the problem.
2. State the definition of almost sure convergence. Prove that if \(X_{n} \to X\) almost surely, then \(X_{n} \to X\) in probability. Again, do not appeal to any theorems that trivialize the problem.

Let \(\{X_{n}\}_{n \ge 1}\) be i.i.d. with mean 0 and variance 1. Let \(\{a_{n}\}_{n \ge 1}\) be a sequence of nonzero real numbers and \[ A_{n} = \left (\sum _{k=1}^{n} a_{k}^{2}\right )^{1/2}. \] Assume that \[ A_{n}^{-1}\left (\max _{1 \le k \le n} |a_{k}|\right ) \to 0 \qquad \text {as } n \to \infty . \] Find (and prove) a distributional limit for \[ A_{n}^{-1}\sum _{k=1}^{n} a_{k}X_{k} \qquad \text {as } n \to \infty . \]

Let \(U_1,U_2,\ldots ,U_n\) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) be the order statistics. Show that \(nU_{3,n}\) converges in distribution and compute the limit.

Let \(X,Y\) be independent random variables with respective densities \[ f(t)= \begin {cases} 0, & \text {if } t\notin [-2,3],\\ \dfrac {1}{5}, & \text {if } t\in [-2,3], \end {cases} \] and \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-1,5],\\ \dfrac {1}{6}, & \text {if } t\in [-1,5]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X\) and \(Y\) be independent normally distributed random variables with \(\mathbb {E} X=\mu _1\), \(\operatorname {Var} X=\sigma _1^2\), \(\mathbb {E} Y=\mu _2\) and \(\operatorname {Var} Y=\sigma _2^2\). Show that \(X+Y\) and \(X-Y\) are independent if and only if \(\sigma _1^2=\sigma _2^2\).

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed exponential\((\lambda )\) random variables, meaning that their pdf is \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ \dfrac {1}{\lambda }e^{-t/\lambda }, & \text {if } 0\leq t<\infty . \end {cases} \] Find the uniformly minimum variance unbiased estimator for \(\tau =\lambda ^5\).

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<\theta ,\\ \dfrac {2\theta }{(\theta +t)^2}, & \text {if } \theta \leq t<\infty , \end {cases} \] for \(\theta >0\) a parameter. Find the maximum likelihood estimator for \(\theta \).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ h(t;\theta )=\frac {3t^2}{\theta ^3}1\{0\leq t\leq \theta \}, \] where \(\theta >0\) is a parameter. Find a \(100(1-\alpha )\%\), two-sided, equal-tailed confidence interval for \(\theta \).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed \(N(\mu ,\sigma ^2)\), where \(\mu ,\sigma ^2\) are both unknown. We wish to test \(H_0:\sigma >\sigma _0\) against \(H_a:\sigma \leq \sigma _0\). We reject \(H_0\) if \(S^2<c\), where \[ S^2=\frac {1}{n-1}\sum _{i=1}^n (X_i-\bar X)^2, \qquad \bar X=\frac {1}{n}\sum _{i=1}^n X_i. \] Find \(c\) such that the size of the test is \(\alpha \).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed with density function \[ h(t;\theta )=\theta t^{\theta -1}1\{0\leq t\leq 1\} \] for \(\theta >0\) unknown. Use the Neyman-Pearson Lemma to find the uniformly most powerful test of size \(\alpha \) for \(H_0:\theta =\theta _0\) versus \(H_a:\theta =\theta _1\), where \(\theta _1>\theta _0\).

Let \(X_1,\ldots ,X_n\) be independent and identically distributed random variables with (unknown) density function \(F\). Let \[ F_0(t)=(1-e^{-t})1\{t>0\}. \] We wish to test \(F(t)=F_0(t)\) for all \(t\). Explain the application of the \(\chi ^2\) goodness-of-fit test.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\theta }e^{-t/\theta }, & \text {if } t\geq 0. \end {cases} \] Let \(\alpha >0\) and \[ \tau =\frac {1}{\theta ^\alpha }. \]

(a) How big \(n\) should be (as a function of \(\alpha \)) so that \(\tau \) has uniformly minimum variance unbiased estimator?

(b) Find the uniformly minimum variance unbiased estimator for \(\tau \) (you need to justify that your estimator has the required property).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {5t^4}{2\theta ^5}, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find the maximum likelihood estimator for \(\theta \) and compute its bias.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {10t^9}{2\theta ^{10}}, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find a moment estimator for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ f(t\theta )=\frac {1}{2}e^{-|t-\theta |}, \qquad -\infty <t<\infty . \] The parameter \(\theta \) could be any real number. Find all maximum likelihood estimators for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be iid with the same pdf as in question 5. Find the form of the rejection region coming from the generalized likelihood ratio test for \(H_0:\theta =\theta _0\) versus \(H_a:\theta \ne \theta _0\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables that have the uniform distribution on \([0,1]\). Let \(X_{1,n}\leq X_{2,n}\leq \cdots \leq X_{n,n}\) be the order statistics. Let \(k\geq 1\) be a fixed integer. Show that \[ Y_n=nX_{k,n} \] converges in distribution and compute the limiting distribution.

Let \(X\) and \(Y\) be independent identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\theta }e^{-t/\theta }, & \text {if } t\geq 0. \end {cases} \] Compute the density function of \[ Z=\frac {X}{X+Y}. \]

Suppose a box contains 6 marbles in total. Suppose \(\theta \) of them are white and \(6-\theta \) of them are black. Test \(H_0:\theta =2\) against \(H_a:\theta \ne 2\) as follows: draw two marbles without replacement and reject \(H_0\) if both marbles are the same color, otherwise do not reject. What is the probability of Type II error for each of the alternatives \(\theta =1\) and \(\theta =3\)?

Let \(X_1,\ldots ,X_n\) be iid random variables. The density function of \(X_i\) is \[ g(t;\theta _i)=\theta _i t^{-\theta _i-1}1\{t\geq 1\}. \] Here \(\theta _i>0\) is an unknown parameter. We wish to test \[ H_0:\theta _1=\theta _2=\cdots =\theta _n \] against the alternative that \(H_0\) is not true. Find a test using the generalized likelihood ratio.

Let \(X_1,X_2,\ldots \), and \(X\) be real-valued random variables defined on a common probability space. Note they do NOT have to be independent or identically distributed. Prove that \[ \sum _{n=1}^{\infty }\mathbb {E}[|X_n-X|]<\infty \Longrightarrow X_n\to X \quad \text {almost surely.} \]

Let \(A_1,A_2,\ldots \), be a sequence of events (not necessarily independent) such that \[ \sum _{n=1}^{\infty }\mathbb {P}(A_n)=\infty \qquad \text {and}\qquad \mathbb {P}(A_n\cap A_m)\leq \mathbb {P}(A_n)\mathbb {P}(A_m) \quad \text {for } m\ne n. \] Prove that \(\mathbb {P}(A_n\text { i.o.})=1\) or provide a counterexample. Hint: Consider the mean and variance of \[ \sum _{i=1}^{n}\mathbf {1}_{A_i}. \]

Let \(X\) be a non-negative, integer-valued random variable.

a) Prove that \(\mathbb {E}[X]=\sum _{n=1}^{\infty }\mathbb {P}(X\geq n)\).

b) A dresser has \(k\) distinct pairs of socks (so \(2k\) socks total) and the socks are unmatched. We select, at random and without replacement, one sock at a time until a pair has been drawn. Compute the expectation of the total number of draws needed.

Let \(X\) and \(Y\) be any two random variables. Suppose \(\mathbb {E}[X^2]<\infty \). The conditional variance of \(X\) given \(Y\) is defined to be \[ \operatorname {Var}(X\mid Y)=\mathbb {E}\bigl [(X-\mathbb {E}[X\mid Y])^2\mid Y\bigr ]. \] Prove the conditional variance formula: \[ \operatorname {Var}(X)=\mathbb {E}[\operatorname {Var}(X\mid Y)]+\operatorname {Var}(\mathbb {E}[X\mid Y]). \]

Let \(\{X_n:n\in \mathbb {N}\}\) be a sequence of i.i.d. positive integrable random variables. Define \(T_n=X_1+\cdots +X_n\), \(n\in \mathbb {N}\), and \[ N(t)=\max \{n\in \mathbb {N}:T_n\leq t\}, \qquad t\geq 0. \] Show that with probability one, \[ \frac {N(t)}{t}\to \frac {1}{\mathbb {E}[X_1]} \qquad \text {as } t\to \infty . \] (Hint: Sketch what the process \(t\mapsto N(t)\) looks like for a fixed realization of the \(X_n\). What do you know about the asymptotic behavior of the jump times?)

Assume the same setting as in the previous problem. You can assume that the claim in that problem is true and solve this problem independently.

a) For an integer \(a\geq 0\) let \(T_a=\inf \{n\geq 0:X_n=a\}\). Fix \(a\geq 2\) and prove that \[ \mathbb {P}(T_0<\infty \text { or }T_a<\infty )=1. \] (Hint: As long as the process is not at 0 it has a positive probability to go up \(a\) times in a row. How do you turn this observation into a proof?)

b) Calculate \(\mathbb {P}(T_a<T_0\mid X_0=1)\). (Carefully explain why the conditions of the theorems you use are satisfied.)

Use your limit theorems to prove that the following limit exists and to compute its value: \[ \lim _{n\to \infty }\int _{-\sqrt {n}}^{\sqrt {n}} \left (1-\frac {x^2}{2n}\right )^n\,dx. \] You need to fully justify the use of whatever limit theorem you choose.

Recall that the characteristic function of a random variable \(X\) is defined as \(\phi _X(t)=\mathbb {E}[e^{itX}]\) for \(t\in \mathbb {R}\). Prove that if \(X\in L^1\) then \(\phi '_X(0)=\mathbb {E}[X]\). Make sure to justify all of your steps.

Consider the symmetric random variable \(X\) taking values in \(\mathbb {Z}\setminus \{-1,0,1\}\) with distribution \[ \mathbb {P}(X=k)=\mathbb {P}(X=-k)=\frac {C}{k^2\log k}, \qquad k=2,3,\ldots \]

a) Prove that there is a finite \(C<\infty \) such that the above is a probability distribution.

b) Show that \(\mathbb {E}[|X|]=\infty \), hence \(\mathbb {E}[X]\) is not defined.

c) However, prove that \(\phi '_X(0)=0\), which shows that the converse direction of the previous problem is not always true. This requires showing that the limit defining \(\phi '_X(0)\) exists and equals zero. (Hint: Prove then use the inequality \(|\cos x-1|\leq \min (x^2/2,1)\).)

Assume that \(A_i\), \(i=1,2,\ldots \), are independent events. Prove that \[ \frac {1}{n}\sum _{i=1}^{n}\left (\mathbf {1}_{A_i}-\mathbb {P}(A_i)\right ) \] converges to zero in probability as \(n\to \infty \), where where \(\mathbf {1}_{A_i}\) is the indicator function of the event \(A_i\).

Let \(X_k\) have characteristic function \(\phi _k\) with \(X_1,X_2,\ldots \), independent random variables. Show that \(\sum _{k=1}^{n}X_k\) converges almost surely \(\Longleftrightarrow \) there exists a neighborhood \(U\) of 0 and a function \(h\) such that \[ \prod _{k=1}^{n}\phi _k(u) \xrightarrow [n\to \infty ]{} h(u)\ne 0, \] for all \(u\in U\). Hint: For the \(\Leftarrow \) direction consider the characteristic function of the partial sums \(\sum _{k=m}^{n}X_k\).

Let \(Z_1,Z_2,\ldots \) be iid with \(\mathbb {P}(Z=1)=\mathbb {P}(Z=-1)=1/2\), and let \(c_1,c_2,\ldots \) be given constants.

1. Express the characteristic function of \(\sum _{k=1}^{n}c_kZ_k\) in terms of standard elementary functions.
2. Show that \(\sum _{k\geq 1}c_kZ_k\) converges almost surely \(\Longleftrightarrow \) \(\sum _{k=1}^{\infty }c_k^2<\infty \). For this you can use results we proved in class, or other problems on this exam.

Let \((Z_n)\) be i.i.d. random variables with \(\mathbb {P}(Z_1=1)=\mathbb {P}(Z_1=2)=1/2\). Show that \(\mathbb {E}[Z_1\cdots Z_n]=(1.5)^n\) and yet \(Z_1\cdots Z_n/(1.45)^n\to 0\) almost surely. Identify the set of numbers \(a>0\) for which \(Z_1\cdots Z_n/a^n\to 0\) almost surely.

(Pólya’s Urn) An urn initially contains \(r\) red and \(b\) blue balls. A ball is chosen uniformly at random (i.e. with probability \(1/(r+b)\) each). If it comes up red (resp. blue), then it is returned and another red (resp. blue) ball is added to the urn. The process is repeated indefinitely. Let \(R_n\) be the number of red balls in the urn after \(n\) draws.

1. A vector \((X_1,X_2,\ldots ,X_n)\) of random variables is said to be exchangeable if \[ (X_1,\ldots ,X_n)\sim (X_{\pi (1)},\ldots ,X_{\pi (n)}) \qquad \text {for all permutations }\pi , \] where \(\sim \) denotes equality in distribution. If \(X_i\) is the indicator that a red ball is drawn from the urn at time \(i\), prove that \((X_1,\ldots ,X_n)\) is exchangeable.
2. Find the mean and variance of \(S_n=X_1+\cdots +X_n\), the total number of red balls added to the urn up to time \(n\).

Let \(\{X_{i,n}:i,n\geq 1\}\) be i.i.d. random variables with mass function \(\{f(x):x=0,1,2,\ldots \}\). Fix \(z\in \{1,2,\ldots \}\). Define random variables \((Z_n)\) as follows: \[ Z_0=z \qquad \text {and}\qquad Z_n=\sum _{i=1}^{Z_{n-1}}X_{i,n} \qquad \text {for } n\geq 1. \] I.e. \(Z_n\) is a branching process with offspring distribution \(f\) and initial population \(z\). For \(s\in [0,1]\) let \(g(s)=\sum _{x=0}^{\infty }f(x)s^x\). Suppose that \(s_0\in (0,1)\) solves \(g(s)=s\). Show that then \(s_0^{Z_n}\) is a martingale in the filtration \(\mathcal {F}_n=\sigma (X_{i,m}:i\geq 1,m\leq n)\). Use this to conclude that \[ \mathbb {P}(Z_n=0\text { for some }n\geq 0)=s_0^z. \]

Let \(X\) and \(Y\) be independent \(N(0,1)\) random variables, and let \(Z=X+Y\).

1. Show that \(\mathbb {E}[Z\mid X>0,Y>0]=2\sqrt {2/\pi }\).
2. Find the distribution and the density of \(Z\) given that \(X>0\) and \(Y>0\). (You can use the CDF of a standard normal, \(\Phi (x)=\mathbb {P}(X\leq x)\), in your expression.)

Suppose \(X_1,X_2,\ldots \) are iid with \(\mathbb {E}|X_i|=\infty \). Let \(S_n=X_1+\cdots +X_n\), and let \(A\) be the event that \(M_n=S_n/n\) converges to a finite limit. Let \(B\) be the event that \(|X_n|\geq n\) infinitely often.

1. State the definition of \(B\) in terms of unions and intersections.
2. Show that \(\mathbb {P}(B)=1\).
3. Verify the identity \[ M_n-M_{n+1}=\frac {M_n}{n+1}-\frac {X_{n+1}}{n+1} \] and use it to show that \(A\cap B=\emptyset \).
4. Use the above to prove that \(\mathbb {P}(A)=0\).

Let \(X_n\) be a sequence of i.i.d. random variables. Assume that \(\mathbb {P}(X_n\to \infty )>0\). Prove that \(\mathbb {P}(X_0=\infty )=1\). Hint: first prove that \(\mathbb {P}(X_n\to \infty )=1\).

Let \(X_n:\Omega \to \mathbb {R}\) be a sequence of integrable random variables with the same distribution. Prove that \(X_n/n\to 0\) almost surely. Hint: Borel-Cantelli.

Let \(X\) and \(Y\) be two random variables such that \(\mathbb {P}(X\leq Y)=1\). Assume also that \(\mathbb {P}(X\leq x)\leq \mathbb {P}(Y\leq x)\) for all \(x\in \mathbb {R}\). Prove that \(\mathbb {P}(X=Y)=1\).

(Pólya’s Urn) An urn initially contains \(r\) red and \(b\) blue balls. A ball is chosen uniformly at random (i.e. with probability \(1/(r+b)\) each). If If it comes up red (resp. blue), then it is returned and another red (resp. blue) ball is added to the urn. The process is repeated indefinitely. Let \(R_n\) be the number of red balls in the urn after \(n\) draws.

a) Prove that the fraction of red balls \(M_n=R_n/(r+b+n)\) has an almost sure limit \(M_\infty \) as \(n\to \infty \).

b) Suppose now \(r=b=1\). Prove that \(\mathbb {P}(R_n=k)=1/(n+1)\) for \(1\leq k\leq n+1\). Hint: Write down the probability of choosing \(k\) red and \(n-k\) blue balls in some fixed order.

c) Still assuming \(r=b=1\), what is the distribution of \(M_\infty \)? Hint: use part b) to compute \(\mathbb {P}(M_\infty \leq x)\).

Let \(X_n\) be a sequence of independent random variables with \(\mathbb {P}(X_n=1)=1/n=1-\mathbb {P}(X_n=0)\). Prove that \[ \left (\sum _{i=1}^{n}X_i-\log n\right )/\sqrt {\log n} \] converges weakly to a standard normal.

Fix \(\alpha \in (0,1)\). Let \(X_n\) be a sequence of independent random variables with \(\mathbb {P}(X_n=n)=\mathbb {P}(X_n=-n)=n^{-\alpha }/2\) and \(\mathbb {P}(X_n=0)=1-n^{-\alpha }\). Prove that \((X_1+\cdots +X_n)/n^{(3-\alpha )/2}\) converges weakly to a centered normal.

Let \(M_n\) be a martingale with \(M_0=0\) and increments \(X_n=M_n-M_{n-1}\) that satisfy \(\mathbb {E}[X_n^2]<\infty \) for all \(n\). Assume that \((b_n)_{n=1}^{\infty }\) is an increasing sequence of positive real numbers such that \(b_n\to \infty \) and \[ \sum _{n=1}^{\infty }\frac {\mathbb {E}[X_n^2]}{b_n^2}<\infty . \] Show that \(M_n/b_n\to 0\) almost surely as \(n\to \infty \). Hint: Since the sum of the \(X_k\) is a martingale so too should be a weighted sum; prove this idea and use it to produce a new martingale out of the \(X_k\) and \(b_k\). Derive convergence properties of this new martingale, and then use Kronecker’s Lemma: if \(a_n\) is a sequence of numbers such that \(\sum _{n=1}^{\infty }a_n\) exists and is finite and \(b_n\) is a sequence of the type above, then \[ \frac {1}{b_n}\sum _{k=1}^{n}a_kb_k\to 0 \qquad \text {as } n\to \infty . \]

Let \(X_1,X_2,\ldots \) be iid random variables with common pdf \[ f(x)= \begin {cases} 0, & |x|\leq 2,\\ \dfrac {c}{x^2\log |x|}, & |x|>2. \end {cases} \]

1. Prove that there is a finite constant \(c\) which makes \(f\) a pdf.
2. Let \(S_n=X_1+X_2+\cdots +X_n\). Prove that \(S_n/n\to 0\) in probability as \(n\to \infty \). Hint: Prove that \(S_n/n\) converges to 0 in distribution, then explain why you can upgrade to convergence in probability for free.
3. Does \(S_n/n\to 0\) almost surely as \(n\to \infty \)?

Let \(X\) and \(Y\) be two integer-valued random variables defined on the same probability space. The total variation distance between \(X\) and \(Y\) is defined as \[ d_{TV}(X,Y)=2\sup _{A\subseteq \mathbb {Z}} |\mathbb {P}(X\in A)-\mathbb {P}(Y\in A)|. \] Prove that \(\mathbb {P}(X=Y)\leq 1-\frac {1}{2}d_{TV}(X,Y)\).

Let \(g:\mathbb {R}\to \mathbb {R}\) be bounded and continuous and fix \(\lambda >0\). Use properties of the Poisson distribution to show that \[ \sum _{k=0}^{\infty }g(k/n)\frac {(n\lambda )^k}{k!}e^{-n\lambda } \to g(\lambda ) \qquad \text {as } n\to \infty . \]

Let \(X_1,X_2\) be iid with density \[ f(x;\theta )= \begin {cases} 3x^2/\theta ^3, & 0<x<\theta ,\\ 0, & \text {otherwise.} \end {cases} \] Find a complete, sufficient statistic for \(\theta \in [0,\infty )\) based on \(X_1\) and \(X_2\).

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with a Beta\((\theta ,\theta )\) distribution, i.e. their density is \[ f(x;\theta )=\frac {\Gamma (2\theta )}{\Gamma (\theta )^2} x^{\theta -1}(1-x)^{\theta -1}, \qquad 0<x<1. \] Show that a best critical region for testing \(H_0:\theta =1\) versus \(H_A:\theta =2\) is \[ \left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

A particular gene occurs as one of two alleles (\(A\) and \(a\)), with allele \(A\) having frequency \(\theta \) in the population. In other words, a random copy of the gene is \(A\) with probability \(\theta \) and \(a\) with probability \(1-\theta \). A diploid genotype is just two genes put together (in any particular order) so the probability of each diploid genotype is \(\theta ^2\) for \(AA\), \(2\theta (1-\theta )\) for \(Aa\), and \((1-\theta )^2\) for \(aa\). Suppose we test a random sample of people and find that \(k_1\) of them are \(AA\), \(k_2\) of them are \(Aa\), and \(k_3\) of them are \(aa\). Find the MLE of \(\theta \).

Let \(X_1,X_2,X_3\) be iid and distributed according to \(f(x)=2x\), \(0<x<1\). Let \(X_{(1)}<X_{(2)}<X_{(3)}\) be the order statistics. Compute the probability that the smallest of the \(X_i\) exceeds the median of the distribution.

Let \(X\) be discrete taking values in \(\{0,1\}\) and have the pdf \(f(x;\theta )=\theta ^x(1-\theta )^{1-x}\) for \(x\in \{0,1\}\). We test the simple hypothesis \(H_0:\theta =1/4\) agains the alternative composite hypothesis \(H_A:\theta <1/4\) by taking a random sample of size 10 and rejecting \(H_0\) if and only if the observed values \(x_1,\ldots ,x_{10}\) satisfy \[ \sum _{i=1}^{10}x_i\leq 1. \] Find the power function \(K(\theta )\) of this test for \(0\leq \theta \leq 1/4\).

Consider the density function \[ p(x;\alpha ,\lambda )=(2\pi x^3)^{-3/2} \exp \left \{(\alpha \lambda )^{1/2}-\frac {1}{2}\log \lambda -\frac {1}{2}\alpha x-\frac {\lambda }{2}x^{-1}\right \}, \qquad 0<x<\infty . \] Given \(X_1,X_2,\ldots ,X_n\) iid from this density find a sufficient statistic (that involves all of the \(X_i\)) for the parameters \((\alpha /2,\lambda /2)\).

Suppose that \(X_1,\ldots ,X_n\) form a random sample from a distribution for which the pdf \(f(x|\theta _1,\theta _2)\) is as follows: \[ f(x|\theta _1,\theta _2)= \begin {cases} \dfrac {1}{\theta _2-\theta _1}, & \theta _1<x<\theta _2,\\ 0, & \text {otherwise.} \end {cases} \] The values of \(\theta _1\) and \(\theta _2\) are unknown, other than \(-\infty <\theta _1<\theta _2<\infty \). Find their MLEs.

Let \(X_1,X_2,\ldots ,X_n\) be iid with a Poisson\((\theta )\) distribution, so that their common probability mass function is \[ \mathbb {P}(X_i=k)=e^{-\theta }\frac {\theta ^k}{k!}, \qquad k=0,1,2,\ldots ,\ \theta >0. \] Find the UMVUE of \(e^{-\theta }\).

A food processing company packages honey in glass jars. Each jar is supposed to contain 10 fluid ounces of honey. Previous experience suggests that the volume of a randomly selected jar of honey is normally distributed with a known variance of 2 ounces. At a significance level of \(\alpha =.05\) derive the likelihood ratio test for testing the null hypothesis \(H_0:\mu =10\) versus \(H_A:\mu \ne 10\).

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a \(N(\theta ,100)\) distribution. Find a uniformly most powerful critical region of size \(\alpha =.10\) for testing \(H_0:\theta =75\) against \(H_A:\theta >75\).

Let \(\mu \) be a translation invariant \(\sigma \)-additive measure on \((\mathbb {R},\mathcal {B}(\mathbb {R}))\) such that \(\mu ((0,1])<\infty \). Prove that there exists a \(c\in (0,\infty )\) such that \(c\mu \) is Lebesgue measure.

Let \(\{X_i:i\geq 1\}\) be a sequence of non-negative, i.i.d. random variables such \(\mathbb {E}[X_1]=\infty \). Prove that \[ \lim _{n\to \infty }\frac {X_1+\cdots +X_n}{n}=\infty \qquad \text {almost surely.} \]

Let \(\mu \) and \(\nu \) be probability measures on \((\mathbb {R},\mathcal {B}(\mathbb {R}))\). Let \(\lambda =\frac {1}{2}\mu +\frac {1}{2}\nu \). Let \(\mathcal {F}\) be a \(\sigma \)-algebra contained in \(\mathcal {B}(\mathbb {R})\). Let \(\mu _{\mathcal {F}}\), \(\nu _{\mathcal {F}}\), and \(\lambda _{\mathcal {F}}\) be the restrictions of \(\mu \), \(\nu \), and \(\lambda \) to \(\mathcal {F}\), respectively.

1. Prove that \(\mu _{\mathcal {F}}\ll \lambda _{\mathcal {F}}\) and \(\nu _{\mathcal {F}}\ll \lambda _{\mathcal {F}}\).
2. What is \(f_\mu =\frac {d\mu _{\mathcal {F}}}{d\lambda _{\mathcal {F}}}\) in terms of \(\frac {d\mu }{d\lambda }\)? Of course, a similar relationship holds for \(f_\nu =\frac {d\nu _{\mathcal {F}}}{d\lambda _{\mathcal {F}}}\).
3. For a bounded \(\mathcal {B}(\mathbb {R})\)-measurable function \(g\), compute \(\mathbb {E}^\lambda [g\mid \mathcal {F}]\) in terms of \(\mathbb {E}^\mu [g\mid \mathcal {F}]\), \(\mathbb {E}^\nu [g\mid \mathcal {F}]\), \(f_\mu \), and \(f_\nu \)?

Let \(\{X_i:i\geq 1\}\) be i.i.d. integrable random variables. Prove that \[ \lim _{n\to \infty } \frac {1}{\log n}\sum _{i=1}^{n}\frac {X_i}{i} = \mathbb {E}[X_1]. \]

Let \(\{X_i:i\geq 1\}\) be i.i.d. random variables distributed uniformly on \([0,1]\). Prove that the distribution of \[ \frac {4\sum _{i=1}^{n} iX_i-n^2}{n^{3/2}} \] converges weakly and identify the limiting distribution.

Let \(\{X_n:n\geq 1\}\) be a stochastic process adapted to a filtration \(\{\mathcal {F}_n:n\geq 1\}\) and satisfying \(\mathbb {E}[|X_n|]<\infty \) for all \(n\geq 1\). Prove that \(\{X_n:n\geq 1\}\) is a martingale if and only if \(\mathbb {E}[X_T]=\mathbb {E}[X_1]\) for all bounded stopping times \(T\).

Let \(\{X_n:n\geq 1\}\) be a martingale with respect to filtration \(\{\mathcal {F}_n:n\geq 1\}\), and let \(T\) be a stopping time with respect to \(\{\mathcal {F}_n:n\geq 1\}\). Prove the following two forms of the optional stopping theorem:

1. If \(T\) is almost surely bounded then \(\mathbb {E}[X_T]=\mathbb {E}[X_1]\),
2. If \(\mathbb {E}[\max _{1\leq n\leq T}|X_n|]<\infty \), then \(\mathbb {E}[X_T]=\mathbb {E}[X_1]\).

Prove the identity \[ \frac {\sin \theta }{\theta } = \int _0^1 e^{i\theta (2x-1)}\,dx. \] Then use this to show that \[ \frac {\sin \theta }{\theta } = \prod _{n=1}^{\infty }\cos (\theta /2^n). \] Hint: Use that a Uniform\((0,1)\) random variable can be written as an infinite weighted sum of iid Bernoulli\((1/2)\) random variables.

Let \(Z_1,Z_2,\ldots \) be iid \(N(0,1)\) random variables. First prove Mills’ ratio \[ \mathbb {P}(Z_i>\lambda ) \leq \frac {1}{\lambda \sqrt {2\pi }}e^{-\lambda ^2/2} \] for any \(\lambda >0\). Use this to show that for any \(\epsilon >0\) \[ \mathbb {P}\left ( \limsup _{n\to \infty } \frac {\max _{k\leq n}Z_k}{\sqrt {(2-\epsilon )\log n}} >1 \right )=1. \]

Show that for every \(\rho \in (-1,1)\) the function \[ f(x,y) = \frac {1}{2\pi \sqrt {1-\rho ^2}} \exp \left ( -\frac {x^2-2\rho xy+y^2}{2(1-\rho ^2)} \right ) \] is a pdf on \(\mathbb {R}^2\), and that if \((X,Y)\) is a pair of random variables with pdf \(f\) then \[ \mathbb {P}(XY<0)=\frac {1}{\pi }\arccos \rho . \]

Let \(X_1,X_2\) be iid with density \[ f(x;\theta )= \begin {cases} 3x^2/\theta ^3, & 0<x<\theta ,\\ 0, & \text {otherwise.} \end {cases} \] Find a complete, sufficient statistic for \(\theta \in [0,\infty )\) based on \(X_1\) and \(X_2\).

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with a Beta\((\theta ,\theta )\) distribution, i.e. their density is \[ f(x;\theta )=\frac {\Gamma (2\theta )}{\Gamma (\theta )^2} x^{\theta -1}(1-x)^{\theta -1}, \qquad 0<x<1. \] Show that a best critical region for testing \(H_0:\theta =1\) versus \(H_A:\theta =2\) is \[ \left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

A particular gene occurs as one of two alleles (\(A\) and \(a\)), with allele \(A\) having frequency \(\theta \) in the population. In other words, a random copy of the gene is \(A\) with probability \(\theta \) and \(a\) with probability \(1-\theta \). A diploid genotype is just two genes put together (in any particular order) so the probability of each diploid genotype is \(\theta ^2\) for \(AA\), \(2\theta (1-\theta )\) for \(Aa\), and \((1-\theta )^2\) for \(aa\). Suppose we test a random sample of people and find that \(k_1\) of them are \(AA\), \(k_2\) of them are \(Aa\), and \(k_3\) of them are \(aa\). Find the MLE of \(\theta \).

Let \(X_1,X_2,X_3\) be iid and distributed according to \(f(x)=2x\), \(0<x<1\). Let \(X_{(1)}<X_{(2)}<X_{(3)}\) be the order statistics. Compute the probability that the smallest of the \(X_i\) exceeds the median of the distribution.

Let \(X\) be discrete taking values in \(\{0,1\}\) and have the pdf \(f(x;\theta )=\theta ^x(1-\theta )^{1-x}\) for \(x\in \{0,1\}\). We test the simple hypothesis \(H_0:\theta =1/4\) agains the alternative composite hypothesis \(H_A:\theta <1/4\) by taking a random sample of size 10 and rejecting \(H_0\) if and only if the observed values \(x_1,\ldots ,x_{10}\) satisfy \[ \sum _{i=1}^{10}x_i\leq 1. \] Find the power function \(K(\theta )\) of this test for \(0\leq \theta \leq 1/4\).

Consider the density function \[ p(x;\alpha ,\lambda )=(2\pi x^3)^{-3/2} \exp \left \{(\alpha \lambda )^{1/2}-\frac {1}{2}\log \lambda -\frac {1}{2}\alpha x-\frac {\lambda }{2}x^{-1}\right \}, \qquad 0<x<\infty . \] Given \(X_1,X_2,\ldots ,X_n\) iid from this density find a sufficient statistic (that involves all of the \(X_i\)) for the parameters \((\alpha /2,\lambda /2)\).

Suppose that \(X_1,\ldots ,X_n\) form a random sample from a distribution for which the pdf \(f(x|\theta _1,\theta _2)\) is as follows: \[ f(x|\theta _1,\theta _2)= \begin {cases} \dfrac {1}{\theta _2-\theta _1}, & \theta _1<x<\theta _2,\\ 0, & \text {otherwise.} \end {cases} \] The values of \(\theta _1\) and \(\theta _2\) are unknown, other than \(-\infty <\theta _1<\theta _2<\infty \). Find their MLEs.

Let \(X_1,X_2,\ldots ,X_n\) be iid with a Poisson\((\theta )\) distribution, so that their common probability mass function is \[ \mathbb {P}(X_i=k)=e^{-\theta }\frac {\theta ^k}{k!}, \qquad k=0,1,2,\ldots ,\ \theta >0. \] Find the UMVUE of \(e^{-\theta }\).

A food processing company packages honey in glass jars. Each jar is supposed to contain 10 fluid ounces of honey. Previous experience suggests that the volume of a randomly selected jar of honey is normally distributed with a known variance of 2 ounces. At a significance level of \(\alpha =.05\) derive the likelihood ratio test for testing the null hypothesis \(H_0:\mu =10\) versus \(H_A:\mu \ne 10\).

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a \(N(\theta ,100)\) distribution. Find a uniformly most powerful critical region of size \(\alpha =.10\) for testing \(H_0:\theta =75\) against \(H_A:\theta >75\).

Let \((X_n)\) be a sequence of real-valued random variables and define the tail \(\sigma \)-algebra \(\mathcal {T}=\cap _{n\geq 1}\sigma (X_n,X_{n+1},\ldots )\). Let \(S_n=X_1+\cdots +X_n\). Prove or disprove the following statements.

1. \(\{\lim _{n\to \infty }S_n\text { exists}\}\in \mathcal {T}\).
2. \(\{\overline {\lim }_{n\to \infty }S_n>0\}\in \mathcal {T}\).
3. \(\{\overline {\lim }_{n\to \infty }S_n/c_n>1\}\in \mathcal {T}\) if \(c_n\to \infty \) as \(n\to \infty \).

Let \(\mu \) be a translation invariant \(\sigma \)-finite measure on \((\mathbb {R},\mathcal {B}(\mathbb {R}))\). Assume that there exist \(\alpha <\beta \) such that \(\mu ((\alpha ,\beta ])<\infty \). Prove that there exists a finite positive constant \(c\) such that \(c\mu \) is Lebesgue measure.

Let \((X_n)\) be i.i.d. random variables with mean \(\mu \).

1. Prove that \[ \lim _{n\to \infty } \frac {1}{\log n}\sum _{i=1}^{n}\frac {X_i}{i} = \mu \qquad \text {almost surely.} \]
2. If the random variables also have a finite variance \(\sigma ^2<\infty \), then find \(a_n\) such that \[ a_n\left (\sum _{i=1}^{n}\frac {X_i}{i}-\mu \log n\right ) \] has a weak limit. Identify this weak limit. (You need to prove the weak convergence, once you have identified \(a_n\) and the limiting distribution.)

1. Prove the Borel-Cantelli lemma. That is, prove that if \(\{A_n\}_{n\geq 1}\) is any sequence of events satisfying \(\sum _{n\geq 1}\mathbb {P}(A_n)<\infty \), then \(\mathbb {P}(A_n\text { occurs for infinitely many }n)=0\).
2. Prove that if \((X_n)\) is any sequence of real-valued random variables, then there are constants \(c_n\) so that \(X_n/c_n\to 0\) almost surely as \(n\to \infty \).

Prove or disprove the following statement. If \(X_n\) converges weakly to \(X\) as \(n\to \infty \) and \(Z_n-X_n\) converges weakly to 0 as \(n\to \infty \), then \(Z_n\) converges weakly to \(X\) as \(n\to \infty \).

Let \((X_n)\) be i.i.d. uniform random variables on \([0,1]\). For each \(n\geq 1\), define \[ Y_n=n\min \{X_k:1\leq k\leq n\}. \] Prove that as \(n\to \infty \), \(Y_n\) converges weakly to an exponential random variable with mean one.

Suppose \((M_n)\) is a martingale with \(M_0=0\) and \(|M_{n+1}-M_n|\leq 1\) for all \(n\geq 0\).

1. Prove that there exists a finite positive constant \(C\) such that \(e^x\leq 1+x+Cx^2\) for all \(x\in [-1,1]\).
2. Prove that \(\mathbb {E}[e^{\lambda M_n}]\leq (1+C\lambda ^2)^n\leq e^{C\lambda ^2n}\) for all \(n\geq 1\) and \(\lambda \in (0,1)\).
3. Prove that \(M_n/n^\delta \to 0\) almost surely for every \(\delta >1/2\).

Give an example of a sequence of random variables \(X_n\) with densities \(f_n(x)\) so that \(X_n\) converges weakly as \(n\to \infty \) to a uniform random variable on \([0,1]\), but \(f_n(x)\) does not converge to 1 for any \(x\in [0,1]\).

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with probability mass function \[ f(t;\theta )=\theta (1-\theta )^{t-1}, \qquad t=1,2,3,\ldots \] Find the uniformly minimum variance unbiased estimator of \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with density function \[ h(t;\eta )=e^{-(t-\eta )}I\{t\geq \eta \}, \] where \(I\) indicates the indicator function. We want to test \(H_0:\eta =\eta _0\) against \(H_A:\eta \ne \eta _0\). Find a test using the likelihood method and compute the power function.

Let \(X_1,X_2,\ldots ,X_n\) be iid random variables with density \[ h(t;\theta )=\theta t^{\theta -1}I\{0\leq t\leq 1\}, \qquad \theta >0. \] Find the maximum likelihood estimator of \(\theta \) and compute the asymptotic variance of the maximum likelihood estimator.

Let \(X_1,X_2,\ldots ,X_n\) be iid with density \[ h(t;\theta )=\frac {1}{2}I\{\theta -1\leq t\leq \theta +1\}. \] Find all maximum likelihood estimators of \(\theta \). Show that if \(\hat \theta _n\) is a maximum likelihood estimator then \(\mathbb {E}[\hat \theta _n]\to \theta \) as \(n\to \infty \).

Let \(X_1\) be \(N(\mu _1,\sigma _1^2)\) and \(X_2\) be \(N(\mu _2,\sigma _2^2)\) and assume \(X_1\) and \(X_2\) are independent. Show that \(Y_1=X_1+X_2\) and \(Y_2=X_1-X_2\) are independent if and only if \(\sigma _1=\sigma _2\).

Let \(X_1,X_2,\ldots ,X_n\) be iid with density \[ f(x)=2x/\theta ^2, \qquad 0<x<\theta . \] Let \(Y=X_{(n)}=X_{n:n}\) be the largest order statistic of the \(X_i\). Show that \(Y/\theta \) is a pivotal quantity for \(\theta \) and use it to construct a 99% confidence interval for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be iid with a Poisson\((\theta )\) distribution, so that their common probability mass function is \[ \mathbb {P}(X_i=k)=e^{-\theta }\frac {\theta ^k}{k!}, \qquad k=0,1,2,\ldots ,\ \theta >0. \] Find a sufficient statistic for \(\theta \).

Assume that the lifetimes of 100 lightbulbs are iid random variables with an Exponential(5) distribution, meaning that their common pdf is \[ f(x;\theta )=\frac {1}{5}e^{-x/5}, \qquad x>0. \] Use normal approximation to estimate the probability that in total the 100 lightbulbs last longer than 550 hours. You can express your answer as an integral.

Let \(X_1,X_2,\ldots ,X_n\) be iid with common pdf \[ f(x;\theta )=\frac {\theta }{(1+x)^{\theta +1}}, \qquad 0<x<\infty ,\ \theta >0. \] What is a lower bound for the variance of an unbiased estimator of \(\theta \)?

Let \(X_1,X_2,\ldots \) be independent random variables, and suppose that \(p_i:=\mathbb {P}\{X_i=0\}\) satisfies \(0<p_i<1\) for every \(i=1,2,\ldots \). For every \(n\geq 1\) let \[ N_n:=\sum _{i=1}^{n}\mathbf {1}_{\{X_i=0\}} \] denote the number of times the finite random sequence \(X_1,\ldots ,X_n\) enters zero. Prove that \[ \frac {N_n}{\sum _{i=1}^{n}p_i}\to 1 \qquad \text {in probability as }n\to \infty . \]

Let \(X_n\) be i.i.d. random variables with mean \(\mu \) and variance \(\sigma ^2<\infty \). Prove (without using the law of the iterated logarithm) that we have almost surely \[ \overline {\lim }_{n\to \infty }\frac {S_n-n\mu }{\sigma \sqrt {n}}=\infty \qquad \text {and}\qquad \underline {\lim }_{n\to \infty }\frac {S_n-n\mu }{\sigma \sqrt {n}}=-\infty . \]

Suppose \(X\) and \(Y\) are two independent, real-valued random variables, and suppose that \(\mathbb {P}\{X\in A\}=0\) for all Borel sets \(A\) of zero Lebesgue measure. Prove, carefully, that there exists a measurable function \(f:\mathbb {R}\to \mathbb {R}\) such that \[ \mathbb {P}\{X+Y\in B\}=\int _B f(x)\,dx \qquad \text {for all measurable sets }B\subseteq \mathbb {R}. \]

Let \(X_1,X_2,\ldots \) be a collection of i.i.d. random variables, taking values \(\pm 1\) with probability \(1/2\) each. Define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\), \(T_0:=0\), and then iteratively define \[ T_{k+1}:=\inf \{n>T_k:S_n=0\} \qquad [\inf \emptyset :=\infty ]\quad \text {for all }k\geq 0. \] That is, \(T_k\) denotes the \(k\)th return time to zero. Prove that \(\lim _{k\to \infty }(T_k/k)=\infty \) a.s.

Suppose \(X\) is a nonnegative random variable that satisfies the following: There exist two real numbers \(a,b>0\) such that \(a^n\leq \mathbb {E}[X^n]\leq b^n\) for every integer \(n\geq 1\). Prove that \(\mathbb {P}\{a\leq X\leq b\}=1\).

Let \(X_1,\ldots ,X_m\) be \(m\geq 2\) i.i.d. standard normal random variables, and define the random vector \[ \mathbf {X}:= \begin {bmatrix} X_1\\ \vdots \\ X_m \end {bmatrix}. \] Prove that the distribution of \(\mathbf {X}\) is the same as the distribution of \(A\mathbf {X}\) for every \(m\times m\) orthogonal matrix \(A\) that is non random.

Suppose \(X\) is a discrete, non-negative random variable with mass function \(p\). Prove that \[ \lim _{n\to \infty }\left (\mathbb {E}[X^n]\right )^{1/n} = \sup \{x\in \mathbb {R}:p(x)>0\}. \]

Fix \(0<\lambda <\rho \) and let \(X\), \(Y\), and \(Z\) be three independent Gamma distributed random variables with common scale parameter 1 and shape parameters \(\lambda \), \(\rho -\lambda \), and \(\rho \), respectively. (A Gamma random variable with scale parameter 1 and shape parameter \(\lambda \) has pdf \(\frac {1}{\Gamma (\lambda )}x^{\lambda -1}e^{-x}\mathbf {1}_{\{x>0\}}\).) Let \[ U=\frac {ZX}{X+Y},\qquad V=\frac {ZY}{X+Y},\qquad \text {and}\qquad W=X+Y. \] Prove that random vector \((U,V,W)\) has the same distribution as \((X,Y,Z)\). (It may be useful to note that \(Z=U+V\).)

Let \(M_n\) be an \(L^1\)-bounded martingale with respect to a filtration \(\mathcal {F}_n\). Prove that there exist two non-negative martingales \(Y_n\) and \(Z_n\) (in the same filtration) such that \(M_n=Y_n-Z_n\). Hint: Use \(Z_{n,j}=\mathbb {E}[|M_j|\mid \mathcal {F}_n]\) for \(j\geq n\).

Let \(M_n\) be a martingale such that \(\sup _n\mathbb {E}[M_n^2]<\infty \). Prove that there exists a random variable \(M_{\infty }\in L^2\) such that \(M_n\to M_{\infty }\) in \(L^2\). Hint: \(M_n=M_0+\sum _{j=1}^{n}(M_j-M_{j-1})\).

Consider an array \(\{X_{i,j}\}\) of random variables such that, for every \(n\geq 1\), the collection \(X_{1,n},\ldots ,X_{n,n}\) is i.i.d. with common distribution \[ \mathbb {P}\{X_{1,n}=1\} = \mathbb {P}\{X_{1,n}=-1\} = \frac {1}{2}\left (1-\frac {1}{n^2}\right ); \] and \(\mathbb {P}\{X_{1,n}=n^2\}=n^{-2}\). Let \(S_n:=X_{1,n}+\cdots +X_{n,n}\). Prove that \(S_n/\sqrt {n}\Rightarrow \mathrm {N}(0,\sigma ^2)\) for some \(\sigma \in (0,\infty )\), although \(\mathbb {E}(S_n/\sqrt {n})\to \infty \) as \(n\to \infty \).

Let \(X_1,X_2,\ldots \) be a collection of i.i.d. random variables, and define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\). Suppose \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2 k^2} \qquad \text {for all } k=\pm 1,\pm 2,\ldots . \] Prove that \(S_n/n\) converges weakly to a non-constant random variable as \(n\to \infty \).

Suppose \(X,Y\in L^1(\mathbb {P})\) satisfy \(\mathbb {E}[X\mid Y]=Y\) and \(\mathbb {E}[Y\mid X]=X\) almost surely. Prove that \(X=Y\) almost surely. (Hint: Prove first that \(\mathbb {E}[X-Y;X<q<Y]=0\) for all \(q\).)

Suppose \(X_1,X_2,\ldots \) are independent mean-zero random variables that satisfy the condition \[ \sum _{i=1}^{\infty }\operatorname {Var}(X_i)<\infty . \]

1. Prove that \(\sum _{n=1}^{\infty }X_n\) exists a.s. and is finite a.s.
2. Use part (a) to prove that if \(\varepsilon _1,\varepsilon _2,\ldots \) are i.i.d. with \(\mathbb {P}\{\varepsilon _1=\pm 1\}=1/2\), then the random harmonic series \(\sum _{i=1}^{\infty }(\varepsilon _i/i)\) converges a.s. and is finite a.s.
3. Is \(\sum _{i=1}^{\infty }(\varepsilon _i/i)\) in \(L^1(\mathbb {P})\)?

Suppose that \(X_1,X_2,\ldots \) are i.i.d. exponential random variables with parameter \(\lambda >0\). Prove that \[ \overline {\lim }_{n\to \infty }\left (\frac {X_n}{\log n}\right )=\lambda ^{-1} \qquad \text {and}\qquad \underline {\lim }_{n\to \infty }\left (\frac {\log X_n}{\log n}\right )=-1. \]

Let \(X_1,X_2,\ldots \) be i.i.d. with \(\mathbb {P}\{X_1=1\}=p\) and \(\mathbb {P}\{X_1=-1\}=q\), where \(p\notin \{0,\frac {1}{2},1\}\) and \(q=1-p\). Define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\) and consider the stopping time, \(T:=\inf \{n\geq 1:|S_n|=5\}\), where \(\inf \emptyset :=\infty \). Prove that \(T\in L^1(\mathbb {P})\) and compute \(\mathbb {E}(T)\). (HINT: Begin by proving that \(M_n:=(q/p)^{S_n}\) and \(N_n:=S_n-(p-q)n\) are martingales.)

Let \(X\) and \(Y\) be two real-valued random variables. Let \(f\) and \(g\) be two non-decreasing measurable functions such that \(f(X)\) and \(g(Y)\) are integrable. Prove that \[ \mathbb {E}[f(X)g(Y)]\geq \mathbb {E}[f(X)]\mathbb {E}[g(Y)]. \]

Let \(X_1,X_2,\ldots ,X_n\) be iid with common pdf \(f(x)=2x\), \(0<x<1\) and zero elsewhere. Find the limiting distribution of \(n(1-X_{(n)})\), where \(X_{(n)}=\max \{X_1,\ldots ,X_n\}\).

Let \(X_1\sim N(0,4\theta )\), \(X_2\sim N(0,9\theta )\), \(X_3\sim N(0,4\theta ^2)\), \(X_4\sim N(0,12\theta ^2)\), and assume that all \(X\)s are independent of each other. Using all four \(X\) variables and the \(t\) distribution, compute a \(100(1-\alpha )\%\) confidence interval for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be iid with common pdf \[ f(x;\theta )=\frac {1}{2\theta }e^{-|x|/\theta }, \qquad -\infty <x<\infty , \] with \(\theta \) unknown but positive. Compute the Cramer-Rao lower bound for the variance of an unbiased estimator of \(\theta \), and extend your calculation to finding the UMVUE by finding a linear combination of \[ \sum _{i=1}^n \frac {\partial }{\partial \theta }\log f(X_i;\theta ) \] that is unbiased.

If a random sample of size \(n\) is taken from a distribution having pdf \(f(x;\theta )=2x/\theta ^2\), \(0<x\leq \theta \) and zero elsewhere, find:

(a) the MLE \(\hat \theta \) for \(\theta \),

(b) the constant \(c\) so that \(\mathbb {E}[c\hat \theta ]=\theta \),

(c) the MLE for the median of the distribution.

If \(X_1,X_2,\ldots ,X_n\) is a random sample from a distribution having pdf of the form \(f(x;\theta )=\theta x^{\theta -1}\), \(0<x<1\) and zero elsewhere, show that a best critical region for testing \(H_0:\theta =1\) against \(H_1:\theta =2\) is \[ C=\left \{(x_1,x_2,\ldots ,x_n): c\leq \prod _{i=1}^n x_i\right \}. \]

Let \(X_1,X_2,\ldots ,X_{25}\) denote a random sample of size 25 from a normal distribution with mean \(\theta \) and variance 100. Find a uniformly most powerful critical region of size \(\alpha =0.10\) for testing \(H_0:\theta =75\) against \(H_1:\theta >75\).

Let \(X_1\) and \(X_2\) be two independent random variables. Suppose that \(X_1\) is \(\chi ^2(r_1)\) and \(X_1+X_2\) is \(\chi ^2(r)\), where \(r_1,r\) are the respective degrees of freedom and \(r_1<r\). Show that \(X_2\) is \(\chi ^2(r-r_1)\).

Let \(Y_1,Y_2,\ldots ,Y_n\) be independent random variables, each with the \(N(\beta x_i,\gamma ^2x_i^2)\) distribution, where the numbers \(x_1,x_2,\ldots ,x_n\) are known, not all are equal, and none of them is zero. Suppose \(\beta \) and \(\gamma \) are unknown. Find the maximum likelihood estimators of \(\beta \) and \(\gamma ^2\).

Let \(X_1,X_2,\ldots ,X_n\) be a random sample from a Gamma\((\alpha ,\theta )\) distribution, with \(\alpha \) known and \(\theta \) unknown. Find the Fisher information of this distribution and show that the MLE of \(\theta \) is an efficient estimator of \(\theta \).

Construct an example of a countable family \(\{\mathcal {F}_i\}_{i=1}^{\infty }\) of \(\sigma \)-algebras such that \[ \sigma \left (\bigcup _{i=1}^{\infty }\mathcal {F}_i\right ) \ne \bigcup _{i=1}^{\infty }\mathcal {F}_i. \] Can you do this so that \(\mathcal {F}_i\subset \mathcal {F}_{i+1}\) for all \(i\geq 1\)? Justify your reasoning.

Choose and fix two integers \(N\geq 1\) and \(x\in \{0,1,\ldots ,N\}\). Let \(X_0=x\) and suppose that, conditionally, \(X_{n+1}\) has a \(\operatorname {Binomial}(N,X_n/N)\) distribution given \(X_1,\ldots ,X_n\). More precisely, for all \(n\geq 0\) and \(k=0,\ldots ,N\), \[ \mathbb {P}(X_{n+1}=k\mid \mathcal {F}_n) = \binom {N}{k} \left (\frac {X_n}{N}\right )^k \left (1-\frac {X_n}{N}\right )^{N-k}, \] where \(\mathcal {F}_n=\sigma (\{X_1,\ldots ,X_n\})\) for all \(n\geq 1\). Prove that:

(a) \(X_{\infty }=\lim _{n\to \infty }X_n\) exists a.s. and in \(L^p(\mathbb {P})\) for all \(1\leq p<\infty \).

(b) Prove that \(\mathbb {E}(X_{\infty })=x\) and \(\mathbb {E}(X_{\infty }^2)=Nx\).

(c) Compute the distribution of \(X_{\infty }\).

Let \(n\geq 1\) be a fixed integer and recall that there are \(n!\) permutations of \([n]:=\{1,\ldots ,n\}\). Let \(\sigma :=\{\sigma (1),\ldots ,\sigma (n)\}\) denote an arbitrary permutation of \([n]\), and choose and fix some integer \(j\in [n]\). We say that \(\sigma \) fixes \(j\) if \(\sigma (j)=j\). Now suppose that \(\sigma \) is selected at random, all permutations of \([n]\) equally likely.

(a) What is the probability that \(\sigma \) fixed \(j\)?

(b) What is the expectation of the number of integers in \([n]\) that are fixed by \(\sigma \)?

Construct two uncorrelated random variables that are not independent.

Let \(\{X_n\}_{n=1}^{\infty }\) be i.i.d. random variables, and let \(p>0\) be fixed. Prove that the following are equivalent:

(a) \(\mathbb {E}(|X_1|^p)<\infty \);

(b) \(n^{-1/p}X_n\to 0\) almost surely; and

(c) \(n^{-1/p}\max _{1\leq j\leq n}|X_j|\to 0\) almost surely.

Suppose \(\{X_n\}_{n=1}^{\infty }\) are i.i.d. exponential random variables with mean one; that is, \(\mathbb {P}\{X_n>x\}=e^{-x}\) for all \(x>0\). Prove that \[ \mathbb {P}\left \{\limsup _{n\to \infty }\frac {X_n}{\log n}=1\right \}=1. \]

Suppose \(X_1,X_2,\ldots \) are i.i.d. strictly positive random variables. Compute for all \(n,m,k\geq 1\), \[ \mathbb {E}\left ( \frac {X_1^k+\cdots +X_m^k}{X_1^k+\cdots +X_n^k} \right ). \] You might wish to start by calculating \(\mathbb {E}(X_1^k\mid X_1^k+\cdots +X_n^k)\).

Let \(X\) and \(Y\) be two positive random variables. We say that \(X\) is stochastically dominated by \(Y\) if \(\mathbb {P}\{X>a\}\leq \mathbb {P}\{Y>a\}\) for all \(a>0\). Prove that, in this case, \(\mathbb {E}(X^k)\leq \mathbb {E}(Y^k)\) for all \(k\geq 1\).

Suppose \(X_1,N_1,X_2,N_2,\ldots \) are independent random variables such that the \(X\)’s are Bernoulli\((1/2)\)—that is, \(\mathbb {P}\{X_i=0\}=\mathbb {P}\{X_i=1\}=1/2\) for all \(i\geq 1\)—and each \(N_n\) is Poisson\((n)\); that is, \[ \mathbb {P}\{N_n=k\}=\frac {n^ke^{-n}}{k!} \qquad \text {for all integers }n\geq 1\text { and }k\geq 0. \] Find non-random sequences \(\{a_n\}_{n=1}^{\infty }\) and \(\{b_n\}_{n=1}^{\infty }\) such that, as \(n\to \infty \), \[ \frac {\sum _{i=1}^{N_n}X_i-a_n}{b_n}\Rightarrow \operatorname {N}(0,1). \]

Suppose \(\mathbb {P}\{X_1=0\}=\mathbb {P}\{X_1=1\}=1/2\) and for all integers \(n\geq 1\), \[ \mathbb {P}(X_{n+1}=1\mid \mathcal {F}_n) = 1-\mathbb {P}(X_{n+1}=0\mid \mathcal {F}_n) = \frac {S_n}{n}, \] where \(S_n:=X_1+\cdots +X_n\) and \(\mathcal {F}_n:=\sigma (\{X_1,\ldots ,X_n\})\). Prove that \(n^{-1}S_n=X_1\) almost surely for all \(n\geq 1\).

Let \(X_1,\ldots ,X_n\) be a random sample with density function \(f(x)=2e^{-2(x-\eta )}\) for \(x>\eta \), and zero otherwise. We wish to test \(H_0:\eta =4\) versus the alternative \(H_A:\eta >4\). We reject \(H_0\) if \(X_{(1)}=\min _{1\leq i\leq n}X_i\geq 4+2/n\).

(a) Compute the significance level of the test.

(b) Compute the power function.

Let \(X_1,\ldots ,X_n\) be iid exponential(1) random variables. Find the joint pdf of the pair \((X_{(1)},X_{(3)})\), where \(X_{(i)}\) denotes the \(i\)th order statistic, i.e. \[ X_{(1)}\leq X_{(2)}\leq \cdots \leq X_{(n)}. \]

Let \(X_1\) and \(X_2\) be independent with joint pdf \(f(x)=x^{-2}\) for \(1\leq x<\infty \) and 0 otherwise.

(a) Find the joint pdf of \(U=X_1X_2\) and \(V=X_2\).

(b) Find the marginal pdf of \(U\).

Assume that \(X_1,\ldots ,X_n\) are a random sample with common pdf \(f(x;\theta )=(\theta +1)x^{-\theta -2}\) for \(x>1\), and zero otherwise. Is the method of moments estimator for \(\theta \) the same as the maximum likelihood estimator for \(\theta \)?

Let \(X_1,\ldots ,X_n\) be iid with the Gamma\((\theta ,\kappa )\) distribution, meaning that their common pdf is \[ f(x;\theta ,\kappa )=\frac {1}{\theta ^\kappa \Gamma (\kappa )} x^{\kappa -1}e^{-x/\theta }, \qquad x>0, \] with \(\kappa \) known but \(\theta \) unknown. Derive a \(100(1-\alpha )\%\) equal-tailed confidence interval for \(\theta \) based on the sufficient statistic.

Suppose that \(X_1,\ldots ,X_n\) are iid from a \(\BETA (\theta _1,\theta _2)\) distribution, meaning that the pdf is \[ f(x;\theta _1,\theta _2)= \frac {\Gamma (\theta _1+\theta _2)}{\Gamma (\theta _1)\Gamma (\theta _2)} x^{\theta _1-1}(1-x)^{\theta _2-1}, \qquad 0<x<1. \] Find jointly sufficient statistics for \(\theta _1\) and \(\theta _2\), and use them to find a UMVUEs for \(\theta _1\) and \(\theta _2\).

Let \(Y_1<Y_2<\cdots <Y_5\) be the order statistics of a random sample of size \(n=5\) from a distribution with pdf \(f(x;\theta )=e^{-|x-\theta |}/2\), \(-\infty <x<\infty \), with \(\theta \) unknown. Find the likelihood ratio test \(\lambda \) for testing \(H_0:\theta =\theta _0\) versus \(H_1:\theta \ne \theta _0\).

Let \(X_1,X_2,\ldots ,X_n\) be iid from a distribution with CDF \(F(x)=1-1/x\) for \(1\leq x<\infty \), and zero otherwise.

(a) Derive the CDF of the smallest order statistic \(X_{(1)}\).

(b) Find the limiting distribution of \(X_{(1)}\).

(c) Find the limiting distribution of \(X_{(1)}^n\).

Let \(X_1,\ldots ,X_n\) be iid Uniform\((0,\theta )\) with \(\theta >0\), and \(X_{(n)}\) be the largest order statistic.

(a) Find the probability that the random interval \((X_{(n)},2X_{(n)})\) contains \(\theta \).

(b) Find the constant \(c\) such that \((X_{(n)},cX_{(n)})\) is a \(100(1-\alpha )\%\) confidence interval for \(\theta \).

Let \(X_1,\ldots ,X_n\) be a random sample from the \(N(\theta ,1)\) distribution, \(\theta \) unknown. We wish to test \(H_0:\theta =\theta _0\) versus \(H_A:\theta <\theta _0\). Find a test using the generalized likelihood ratio \(\lambda \).

The distance in feet by which a parachutist misses a target is \(D=\sqrt {X_1^2+X_2^2}\), where \(X_1\) and \(X_2\) are independent with \(X_i\sim N(0,25)\). Find \(\Prob (D\leq 12.25)\) (you can write down an integral for it, you don’t need to evaluate it).

If \(X\in L^2\), then it is easy to show that \(\mu \) is its mean if and only if \(\mathbb {E}[(X-a)^2]\) is uniquely minimized at \(a=\mu \). Assuming \(X\in L^1\), show that \(m\) is a median of \(X\) if and only if \(\mathbb {E}[|X-a|]\) is minimized (not necessarily uniquely) at \(a=m\).

(a) Find a simple asymptotic formula for \(\mathbb {P}(X>x)\), where \(X\) has the GAMMA\((\theta ,\alpha )\) density, \[ f(x)=\frac {1}{\theta ^{\alpha }\Gamma (\alpha )} x^{\alpha -1}e^{-x/\theta }, \qquad x>0. \]

(b) Let \(X_1,X_2,\ldots \) be i.i.d. GAMMA\((\theta ,\alpha )\). Find an increasing sequence \(a_n\) such that \(\limsup _{n\to \infty }X_n/a_n=1\) a.s., and prove it.

(a) Find the distribution of \(\int _0^1 B_t\,dt\), where \((B_t)\) is Brownian motion.

(b) Given an i.i.d. sequence \(X_1,X_2,\ldots \) with mean 0 and variance 1, what does Donsker’s invariance principle tell us about this sequence in connection with the distribution of \(\int _0^1 B_t\,dt\)?

Let \(X_1,X_2,\ldots \) be i.i.d. with common distribution \(\mathbb {P}(X_1=1)=\mathbb {P}(X_1=-1)=1/2\), and put \(S_0:=0\) and \(S_n:=X_1+\cdots +X_n\) for each \(n\geq 1\). Consider the stopping time \(N:=\min \{n\geq 1:S_n=1\}\).

(a) Let \(g(u):=\mathbb {E}[u^{X_1}]\) for all \(u>1\), and show that the optional stopping theorem applies to the martingale \(M_n:=u^{S_n}g(u)^{-n}\), where \(u>1\), and the stopping time \(N\).

(b) Use part (a) to show that \[ \mathbb {E}[v^N]=\frac {1-\sqrt {1-v^2}}{v}, \qquad 0<v<1. \]

(c) Use part (b) to show that \[ \mathbb {P}(N=2m+1)= \frac {1}{m+1}\binom {2m}{m}\frac {1}{2^{2m+1}}, \qquad m\geq 0. \]

Suppose \(g\) and \(h\) are continuous on \(\mathbf {R}\) with \(g>0\) and \(|h(x)|/g(x)\to 0\) as \(|x|\to \infty \). If \(X_n\Rightarrow X\) and \(\sup _n\mathbb {E}[g(X_n)]<\infty \), show that \(\mathbb {E}[h(X_n)]\to \mathbb {E}[h(X)]\).

Let \(U_0,U_1,\ldots \) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(N\) be a Poisson random variable with \(\mathbb {E}[N]=1\), independent of \(U_0,U_1,\ldots \). Compute \(\mathbb {E}[Y]\), where \(Y=\max _{0\leq k\leq N} U_k\).

Let \(\{e_i,-\infty <i<\infty \}\) be independent and identically distributed random variables with \(\mathbb {E}[e_1]=0\) and \(\mathbb {E}[e_1^2]=\sigma ^2\) and \(\rho \) be a real number satisfying \(|\rho |<1\). Show that \[ Y=\sum _{i=0}^{\infty }\rho ^i e_i \] is finite with probability one. Compute \(\mathbb {E}[Y]\) and \(\mathbb {E}[Y^2]\). (You need to justify your steps when you compute these two expected values.)

Compute \[ \begin {aligned} \lim _{n\to \infty }(b-a)^{-n} \int _a^b\int _a^b\cdots \int _a^b \frac {x_1+x_2+\cdots +x_n}{x_1^2+x_2^2+\cdots +x_n^2} \,dx_1\,dx_2\cdots dx_n \end {aligned} \] if \(0<a<b<\infty \).

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables with characteristic function \(\varphi \). Let \(N\) be a random variable with \[ \mathbb {P}(N=k)=\frac {1}{2^k}, \qquad k=1,2,\ldots . \] We assume that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent.

(a) Compute the characteristic function of \(Y=X_1+X_2+\cdots +X_N\).

(b) Can you weaken the condition that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent and the formula obtained in (a) remains valid?

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t;\theta )= \begin {cases} \dfrac {1}{\theta }, & \text {if } 0\leq t\leq \theta ,\\ 0, & \text {if } t\notin [0,\theta ]. \end {cases} \] We wish to test \(H_0:\theta =1\) against the alternative \(H_A:\theta <1\). We reject \(H_0\) if \[ X_{n,n}=\max _{1\leq i\leq n}X_i\leq 1-\frac {2}{n}. \] (a) Compute the significance level.

(b) Compute the power function.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t;\mu )=\frac {1}{(2\pi )^{1/2}}e^{-(t-\mu )^2/2}. \] We wish to test \(H_0:\mu \leq \mu _0\) against \(H_A:\mu >\mu _0\). Find the uniformly most powerful test of size \(\alpha \).

Let \(X_1,X_2\) be independent and identically distributed exponential\((\lambda )\) random variables, i.e. the common density is \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Find the joint density function of \[ Y_1=X_1+X_2 \quad \text {and}\quad Y_2=\frac {X_1}{X_1+X_2}. \]

We assume that the number of people entering a store follows a Poisson distribution with parameter \(\lambda \). Every customer spends money in the store, independently of each other according to a uniform distribution on \([10,100]\). We observe \(X\), the total amount of money spent in the store on a given day.

(a) Estimate \(\lambda \) using the method of moments.

(b) Let assume that we have the stores income \(X_1,X_2,\ldots ,X_n\) for \(n\) days and \(X_1,X_2,\ldots ,X_n\) are independent and distributed as \(X\). What would be the moment estimator for \(\lambda \) based on \(X_1,X_2,\ldots ,X_n\)?

Let \(X\) and \(Y\) be independent random variables with densities \(f\) and \(g\), where \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{2}\exp (-t/2), & \text {if } t\geq 0, \end {cases} \] and \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-1,2],\\ \dfrac {1}{3}, & \text {if } t\in [-1,2]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Construct an equal tail \(1-\alpha \) confidence interval for \(\lambda \). Compute the expected value of the length of the confidence interval.

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with density function \[ f(t)= \begin {cases} 0, & \text {if } t<0,\\ \dfrac {1}{\lambda }\exp (-t/\lambda ), & \text {if } t\geq 0. \end {cases} \] Find the uniformly minimum variance unbiased estimator for \(\theta =1/\lambda \).

Let \(U_1,U_2,\ldots ,U_n\) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) denote the order statistics.

(a) Compute the limit distribution of \(Z=n(U_{3,n}-U_{2,n})\).

(b) Show that \(Z_1=n(U_{3,n}-U_{2,n})\) and \(Z_2=n(U_{4,n}-U_{3,n})\) are asymptotically independent.

Let \(X_1,X_2,\ldots ,X_n\) be independent and identically distributed random variables with distribution function \(F\). We wish to test that \(F\) is a normal distribution function. Provide at least two different methods on to test for the normality of \(F\).

Let \(X_1,X_2,\ldots ,X_n\) be a random sample from the Poisson distribution with parameter \(\lambda >0\). Use the Rao–Blackwell theorem and/or the Lehmann–Scheffé theorem to find a UMVUE of \(\mathbb {P}\{X_1=0\}=e^{-\lambda }\).

Find the maximum likelihood estimators of \(\theta \) and \(\alpha \), based on a random sample \(X_1,X_2,\ldots ,X_n\) from the distribution with density \[ f(x;\theta ,\alpha )=\frac {\alpha }{\theta ^\alpha }x^{\alpha -1}, \qquad 0\leq x\leq \theta . \] Here \(\theta \) and \(\alpha \) are positive.

Let \(X_1,\ldots ,X_{2n+1}\) be independent lognormal\((\mu ,\sigma ^2)\). That is, \(X_1=e^{Y_1},\ldots ,X_{2n+1}=e^{Y_{2n+1}}\) with \(Y_1,\ldots ,Y_{2n+1}\) being independent \(N(\mu ,\sigma ^2)\). Use a theorem to show that the sample median \(X_{n+1:2n+1}\) is asymptotically normal and find its asymptotic mean and asymptotic variance.

Suppose we have two samples of sizes \(n_1=4\) and \(n_2=6\). We observe \[ (x_1,x_2,x_3,x_4)=(3.7,3.3,4.2,3.1) \] and \[ (y_1,y_2,y_3,y_4,y_5,y_6)=(3.6,5.0,4.5,3.9,5.1,4.7). \] The null hypothesis is that \(F_X=F_Y\) and the alternative hypothesis is that \(X\) is stochastically smaller than \(Y\). Use the Wilcoxon/Mann–Whitney test. Determine the value of the test statistic and evaluate the p-value of the data. Do not use a large-sample approximation.

Let \(X_1,X_2,\ldots ,X_n\) be a random sample from a \(N(\theta ,\sigma ^2)\) distribution (\(\sigma ^2\) known), and let the prior distribution of \(\theta \) be \(N(\mu _0,\sigma _0^2)\), where \(\mu _0\) and \(\sigma _0^2\) are known.

(a) Find the posterior distribution of \(\theta \).

(b) Find the Bayes estimator under squared error loss.

(a) Let \(X_1,X_2,\ldots ,X_n\) be a random sample from \(N(\theta ,\theta ^2)\), where \(\theta >0\). Find a pair of statistics that are jointly sufficient.

(b) Using some linear combination of \(\sum _{i=1}^n X_i^2\) and \(\left (\sum _{i=1}^n X_i\right )^2\), show that the statistics of part (a) are not jointly complete.

Let \(X_1\) and \(X_2\) be independent \(\operatorname {Exponential}(1)\) random variables (density \(e^{-x}\), \(x>0\)). Find the joint density of \(Y_1=X_1/X_2\) and \(Y_2=X_1+X_2\). Find the marginal densities of \(Y_1\) and \(Y_2\) as well.

Let \(X_1,\ldots ,X_n\) be a random sample from \(\operatorname {Uniform}[0,\theta ]\), where \(\theta \) is a positive unknown. Derive the generalized likelihood ratio (GLR) test of \(H_0:\theta =\theta _0\) vs. \(H_a:\theta \ne \theta _0\).

\(X_1,X_2,\ldots ,X_n\) be independent identically distributed positive random variables. Show that \[ \frac {1}{n}\max _{1\leq k\leq n} X_k \to 0 \qquad \text {a.s.} \] if and only if \(\mathbb {E} X_1<\infty \).

Let \(X\geq 0\) and \(\mathbb {E}|X|^p<\infty \) with some \(p>0\). Show that \[ \lim _{t\to \infty } t^p\mathbb {P}\{X>t\}=0. \]

Let \(X\) and \(Y\) be independent random variables. Show that if the distribution of \(X\) is absolutely continuous with respect to the Lebesgue measure, then the distribution of \(X+Y\) is also absolutely continuous with respect to the Lebesgue measure.

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables with \(\mathbb {E}\log (1+|X_1|)<\infty \). Show that \[ \sum _{i=1}^{\infty }\prod _{j=1}^{i}X_j \] is finite with probability one if and only if \(\mathbb {E}\log |X_1|<0\).

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables with distribution function \(F(x)=\mathbb {P}\{X_1\leq x\}\). Show that \[ \sup _{-\infty <x<\infty } |F_n(x)-F(x)| \to 0 \qquad \text {a.s.} \] as \(n\to \infty \), where \[ F_n(x)=\frac {1}{n}\sum _{i=1}^{n} I\{X_i\leq x\}. \]

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential(1) random variables. Let \(k\geq 1\). Show that \[ \mathbb {P}\{X_n>\phi _{\varepsilon }(n)\ \text {i.o.}\} = \begin {cases} 0, & \text {if }\varepsilon >0,\\ 1, & \text {if }\varepsilon \leq 0, \end {cases} \] where \[ \phi _{\varepsilon }(n)=\log n+\log _{(2)}(n)+\cdots + (1+\varepsilon )\log _{(k)}(n), \] and \(\log _{(1)}(n)=\log n\), \(\log _{(k)}(n)=\log (\log _{(k-1)}(n))\).

Let \(X_1,X_2,\ldots \) be independent identically distributed positive random variables with \(\mathbb {E} X_1=\mu \). Define \[ N(t)=\min \left \{k:\sum _{i=1}^{k} X_i>t\right \}. \] Show that \[ \lim _{t\to \infty }\frac {1}{t}N(t)=\frac {1}{\mu } \qquad \text {a.s.} \]

Let \(X\) and \(Y\) be independent standard normal random variables. Compute the density function of \(Z=X/Y\).

Compute \[ \lim _{n\to \infty } e^{-n} \sum _{i=0}^{n+\sqrt {n}}\frac {n^i}{i!}. \]

Let \(X_1,X_2,\ldots \) be independent random variables with 2 finite moments, and \[ \sum _{j=1}^{\infty }\operatorname {Var}(X_j)<\infty . \] Prove that \[ \lim _{n\to \infty }\sum _{j=1}^{n}(X_j-\mathbb {E} X_j) \] exists a.s. and in \(L^2(\mathbb {P})\).

Suppose \(\{X_n\}_{n=1}^{\infty }\) and \(\{Y_n\}_{n=1}^{\infty }\) are submartingales with respect to the same filtration \(\{\mathcal {F}_n\}_{n=1}^{\infty }\). Prove that \(Z_n:=\max (X_n,Y_n)\) is also a submartingale.

Suppose \(X=\operatorname {Poisson}(\lambda )\) for some \(\lambda >0\). Prove that \(\mathbb {P}\{X>e\lambda \}\leq e^{-\lambda }\).

Suppose \(\{X_j\}_{j=0}^{\infty }\) is a martingale such that \(X_0=0\) and \(|X_{i+1}-X_i|\leq 1\) for all \(i\geq 0\). Prove that \(\lim _{n\to \infty }n^{-\delta }X_n=0\), a.s. for every \(\delta >1/2\).

Let \(X_1,X_2,\ldots \) be independent, identically distributed random variables with the following common distribution: \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2k^2} \qquad \text {for } k=\pm 1,\pm 2,\ldots . \]

(a) Prove that \(\mathbb {E}\exp (itX_1)=1-(3|t|/\pi )+o(|t|)\) as \(t\to 0\). You may use, without proof, the fact that \[ \int _0^{\infty }(1-\cos \theta )\theta ^{-2}\,d\theta =\pi /2. \]

(b) Use the preceding in order to prove that \(n^{-1}(X_1+\cdots +X_n)\) converges weakly to a “Cauchy random variable \(Y\) with scale parameter \(3/\pi \).” That is, the characteristic function of \(Y\) is \[ \mathbb {E}\exp (itY)=\exp (-3|t|/\pi ) \qquad \text {for all }t\in \mathbb {R}. \] You may use, without proof, the fact that the latter is a characteristic function. This matter is the topic of Problem 8 below.

Suppose \(f:[0,\infty )\to \mathbb {R}\) is bounded and measurable, and \(f(x)=0\) for all \(x\geq 2\).

(a) Prove that if \(X_1,X_2,\ldots \) are independent, all distributed uniformly in the interval \((0,1)\), then \[ \frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+X_j}{n}\right ) \xrightarrow {\mathbb {P}} \int _0^{\infty } f(x)\,dx \qquad \text {as } n\to \infty . \]

(b) (Extra credit) Deduce from this the following version of Riemann sums for Lebesgue integrals [Chaterjee’s theorem]: There exists a sequence \(\{\delta _{j,n};\ 0\leq j\leq n,\ n\geq 1\}\) of real numbers in \((0,1)\) such that \[ \lim _{n\to \infty }\frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+\delta _{j,n}}{n}\right ) = \int _0^{\infty } f(x)\,dx. \]

Let \(f\) be a continuous element of \(L^1(\mathbb {R})\), and recall that its Fourier transform is \[ \hat {f}(t):=\int _{-\infty }^{\infty }\exp (itx)f(x)\,dx. \]

(a) Suppose also that \(\hat {f}\in L^1(\mathbb {R})\). Prove the following [inversion formula]: \[ f(x)=\frac {1}{2\pi }\int _{-\infty }^{\infty }e^{-ixt}\hat {f}(t)\,dt \qquad \text {for all }x\in \mathbb {R}. \] In other words, \(\widehat {\widehat {f}}(-x)=2\pi f(x)\) for all \(x\in \mathbb {R}\).

(b) Choose and fix some \(\theta >0\). Use the preceding to prove that the Fourier transform of the probability density function \[ f(x):=\frac {\theta }{\pi (\theta ^2+x^2)} \qquad [-\infty <x<\infty ] \] is \(\hat {f}(t)=\exp (-\theta |t|)\).

Suppose \(X\) and \(Y\) are i.i.d. random variables and \(f\) and \(g\) are nondecreasing, bounded and measurable functions on \(\mathbb {R}\).

(a) Prove that \[ \mathbb {E}[(f(X)-f(Y))(g(X)-g(Y))]\geq 0. \]

(b) Conclude that \(f(X)\) and \(g(X)\) are always positively correlated; i.e., \[ \mathbb {E}[f(X)g(X)]\geq \mathbb {E}[f(X)]\cdot \mathbb {E}[g(X)]. \]

Let \(Z=\operatorname {N}(0,1)\) denote a standard normal random variable. Suppose \(f:\mathbb {R}\to \mathbb {R}\) is continuously differentiable and there exists \(c>1\) such that \[ |f(x)|+|f'(x)|\leq c\exp (c|x|) \qquad \text {for all }x\in \mathbb {R}. \] Derive the following [Stein’s differential equation]: \[ \mathbb {E}[f'(Z)]=\mathbb {E}[Zf(Z)]. \]

Let \(X_1,X_2,\ldots \) be i.i.d. with \(\mathbb {P}\{X_1=1\}=1/4\) and \(\mathbb {P}\{X_1=0\}=3/4\). Let \(N\) denote the smallest integer \(k\geq 2\) such that \(X_{k-1}=0,=X_k=1\); that is, \(N\) is the first time the pattern “01” occurs in the infinite sequence \(X_1,X_2,\ldots \). Compute \(\mathbb {E}(N)\).

Suppose \(X_1,X_2,\ldots ,X_n\) are i.i.d. random variables, and \(1\leq X_1\leq 2\) a.s. Compute, for every \(1\leq i\leq j\leq n\), the quantity \[ \mathbb {E}\left (\frac {X_i+\cdots +X_j}{X_1+\cdots +X_n}\right ). \]

Suppose \(\mu \) is a probability measure on the Borel subsets of \((0,1)\) such that \[ \int |\ln (x)|\,\mu (dx)<\infty . \] Prove that \[ I(n):=\left (\int x^{1/n}\,\mu (dx)\right )^n \to \exp \left (\int \ln (x)\,\mu (dx)\right ) \qquad \text {as } n\to \infty . \]

[Hint: \(I(n)=\mathbb {E}\left (\prod _{j=1}^{n}X_j^{1/n}\right )\), where \(X_1,\ldots ,X_n\) are i.i.d.]

Let \(\mathbf {X}_n:=(X_{1,n},\ldots ,X_{d,n})\) be a random variable in \(\mathbf {R}^d\) for every \(n\geq 1\). Let \(\mathbf {Y}:=(Y_1,\ldots ,Y_d)\) be another random variable in \(\mathbf {R}^d\). Prove that \(\mathbf {X}_n\Rightarrow \mathbf {Y}\) if and only if the 1-dimensional random variable \(\sum _{i=1}^{d}\lambda _iX_{i,n}\) converges weakly to \(\sum _{i=1}^{d}\lambda _iY_i\) for every \(\lambda :=(\lambda _1,\ldots ,\lambda _d)\in \mathbf {R}^d\).

For every random vector \((X,Y)\) in \(\mathbf {R}^2\) we can define a function \(\mathcal {Q}\) of two variables as follows: \[ \mathcal {Q}(a,b):=\Prob \{X>a,\ Y>b\}-\Prob \{X>a\}\Prob \{Y>b\}. \]

(a) Suppose \(f,g:\mathbf {R}\to \mathbf {R}\) are bounded and have integrable, bounded, and continuous derivatives. Then prove that \[ \E [f(X)g(Y)]-\E [f(X)]\E [g(Y)] = \iint _{\mathbf {R}^2}\mathcal {Q}(a,b)f'(a)g'(b)\,da\,db. \]

(b) Suppose \(\mathcal {Q}(a,b)\geq 0\) for all \(a,b\in \mathbf {R}\). Prove that if \(X\) and \(Y\) have two finite moments, then \(\operatorname {Cov}(X,Y)\geq 0\) and there exists a finite constant \(\alpha \), depending only on \(f\) and \(g\), such that \[ |\E [f(X)g(Y)]-\E [f(X)]\E [g(Y)]|\leq \alpha \operatorname {Cov}(X,Y). \]

[In other words, if \(\mathcal {Q}\geq 0\) and \(X\) and \(Y\) have a small covariance, then \(X\) and \(Y\) are almost independent.]

Suppose that, for every integer \(n\geq 1\), \(X(n)\) has a Gamma distribution with parameters \(n+1\) and 1; that is, the probability density function of \(X(n)\) is \[ f_n(x):=\frac {x^ne^{-x}}{n!}\mathbf {1}_{(0,\infty )}(x). \]

(a) Compute \(\mathbb {E}(X(n))\) and \(\operatorname {Var}(X(n))\) for every integer \(n\geq 1\).

(b) Prove that, as \(n\to \infty \), \[ \frac {X(n)-n}{\sqrt {n}}\Rightarrow \mathrm {N}(0,1). \]

Let \(X\) be an integer-valued random variable that is non-degenerate; this means that there exist at least two integers \(n,m\) such that \(\mathbb {P}\{X=n\}\) and \(\mathbb {P}\{X=m\}\) are both strictly positive. Prove that the set \[ \mathcal {V}:=\{t\in \mathbf {R}:|\mathbb {E}[e^{itX}]|=1\} \] has zero Lebesgue measure. [Hint: If \(t\in \mathcal {V}\) then \(Z:=e^{it(X-Y)}\) has mean one, where \(Y\) is an independent copy of \(X\). Proceed by proving that \(\mathbb {E}(|Z-1|^2)=0\).]

Consider i.i.d. \(X_1,X_2,\ldots \) that are strictly positive, and define \[ S_n:=\sqrt {X_1+\sqrt {X_2+\sqrt {X_3+\sqrt {\cdots +\sqrt {X_n}}}}}, \qquad \text {for all }n\geq 1. \] In other words, if \(f(a,b):=\sqrt {a+\sqrt {b}}\) for all \(a,b>0\), then \(S_1:=f(X_1,0)\) and \(S_{k+1}:=f(S_k,X_{k+1})\) for all \(k\geq 0\).

Prove that \(\lim _{n\to \infty }S_n\) exists a.s. and is finite a.s.

Let \(X_1,X_2,\ldots \) be independent identically distributed random variables with characteristic function \(\varphi \). Let \(N\) be a random variable with distribution \[ \mathbb {P}\{N=k\}=\frac {1}{2^k},\qquad k=1,2,\ldots . \] It is assumed that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent.

(a) Compute the characteristic function of \(Y=X_1+\cdots X_N\).

(b) Can you weaken the condition that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent so that the formula obtained in part (a) remains true?

Let \(X_1,\ldots ,X_n\) be independent identically distributed random variables with \(\mathbb {E} X_1=\mu \) and \(0<\operatorname {Var}(X_1)=\sigma ^2<\infty \). Let \[ Y_n=\sum _{1\leq i<j\leq n}X_iX_j. \] Find numerical sequences \(a_n\) and \(b_n\) such that \((Y_n-a_n)/b_n\) has a non-degenerate limit distribution.

Let \(f\) be a continuous and bounded function on \([0,\infty )\). Compute \[ \lim _{n\to \infty }\int _0^\infty \cdots \int _0^\infty f\!\left (\frac {x_1+\cdots +x_n}{n}\right ) e^{-(x_1+\cdots +x_n)}\,dx_1\cdots x_n. \]

Let \(X_1,\ldots ,X_n\) be independent identically distributed Poisson random variables for each \(n\) with parameter \(\lambda _n\).

(a) Show that \(X_1+\cdots +X_n\) is asymptotically normal if and only if \(n\lambda _n\to \infty \).

(b) Can you weaken the condition that the X’s are identically distributed for each \(n\)?

(a) Let \(X\) be a random variable with characteristic function \(\varphi \). Show that \(X\) is symmetric if and only if \(\varphi (t)\) is real for all \(t\).

(b) Give three distinct examples of real characteristic functions.

Let \(X\) and \(Y\) be integrable random variables on \((\Omega ,\mathcal {F},P)\), and let \(\mathcal {G}\) be a sub-\(\sigma \)-algebra of \(\mathcal {F}\). If \(X=Y\) on \(G\in \mathcal {G}\), show that \(\mathbb {E}[X\mid \mathcal {G}]=\mathbb {E}[Y\mid \mathcal {G}]\) a.s. on \(G\).

Let \(Z_n\) be a Galton–Watson branching process with offspring distribution \(\{p_k,\ k=0,1,2,\ldots \}\) and \(Z_0=x\) (with \(x\) a positive integer), and let \[ f(\theta )=\sum p_k\theta ^k \] be the associated pgf. Suppose that \(\rho \in (0,1)\) satisfies \(f(\rho )=\rho \). Show that \(\rho ^{Z_n}\) is a martingale, and use this to conclude that \(\mathbb {P}(Z_n=0\text { for some }n\geq 0)=\rho ^x\).

Let \(X\geq 0\), \(\mathbb {E} X^2<\infty \), and \(0\leq a<\mathbb {E} X\). Apply the Cauchy–Schwarz inequality to prove that \[ \mathbb {P}(X>a)\geq \frac {(\mathbb {E} X-a)^2}{\mathbb {E} X^2}. \]

By considering the Poisson distribution, show that \[ e^{-n}\left (1+n+\frac {n^2}{2!}+\cdots +\frac {n^n}{n!}\right )\to \frac {1}{2} \] as \(n\to \infty \).

Let \(X_1,X_2,\ldots \) be independent identically distributed random variables with characteristic function \(\varphi \). Let \(N\) be a random variable with distribution \[ \mathbb {P}\{N=k\}=\frac {1}{2^k},\qquad k=1,2,\ldots . \] It is assumed that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent.

(a) Compute the characteristic function of \(Y=X_1+\cdots X_N\).

(b) Can you weaken the condition that \(\{X_i,\ i\geq 1\}\) and \(N\) are independent so that the formula obtained in part (a) remains true?

Let \(X_1,X_2,\ldots \) be independent identically distributed standard normal random variables. Show that \[ \limsup _{n\to \infty }\frac {1}{\sqrt {\log n}}|X_n|=c \quad \text {almost surely} \] and compute the value of \(c\).

Let \(X_1,\ldots ,X_n\) be independent identically distributed random variables with \(\mathbb {E} X_1=\mu \) and \(0<\operatorname {Var}(X_1)=\sigma ^2<\infty \). Let \[ Y_n=\sum _{1\leq i<j\leq n}X_iX_j. \] Find numerical sequences \(a_n\) and \(b_n\) such that \((Y_n-a_n)/b_n\) has a non-degenerate limit distribution.

Let \(f\) be a continuous and bounded function on \([0,\infty )\). Compute \[ \lim _{n\to \infty }\int _0^\infty \cdots \int _0^\infty f\!\left (\frac {x_1+\cdots +x_n}{n}\right ) e^{-(x_1+\cdots +x_n)}\,dx_1\cdots x_n. \]

Let \(X_1,\ldots ,X_n\) be independent identically distributed Poisson random variables for each \(n\) with parameter \(\lambda _n\).

(a) Show that \(X_1+\cdots +X_n\) is asymptotically normal if and only if \(n\lambda _n\to \infty \).

(b) Can you weaken the condition that the X’s are identically distributed for each \(n\)?

Let \(X\) be a random variable with characteristic function \(\varphi \). Show that \(X\) is symmetric if and only if \(\varphi (t)\) is real for all \(t\).

Let \(X\) and \(Y\) be integrable random variables on \((\Omega ,\mathcal {F},P)\), and let \(\mathcal {G}\) be a sub-\(\sigma \)-algebra of \(\mathcal {F}\). If \(X=Y\) on \(G\in \mathcal {G}\), show that \(\mathbb {E}[X\mid \mathcal {G}]=\mathbb {E}[Y\mid \mathcal {G}]\) a.s. on \(G\).

Let \(Z_n\) be a Galton–Watson branching process with offspring distribution \(\{p_k,\ k=0,1,2,\ldots \}\) and \(Z_0=x\) (with \(x\) a positive integer), and let \[ f(\theta )=\sum p_k\theta ^k \] be the associated pgf. Suppose that \(\rho \in (0,1)\) satisfies \(f(\rho )=\rho \). Show that \(\rho ^{Z_n}\) is a martingale, and use this to conclude that \(\mathbb {P}(Z_n=0\text { for some }n\geq 0)=\rho ^x\).

Let \(X\geq 0\), \(\mathbb {E} X^2<\infty \), and \(0\leq a<\mathbb {E} X\). Apply the Cauchy–Schwarz inequality to prove that \[ \mathbb {P}(X>a)\geq \frac {(\mathbb {E} X-a)^2}{\mathbb {E} X^2}. \]

By considering the Poisson distribution, show that \[ e^{-n}\left (1+n+\frac {n^2}{2!}+\cdots +\frac {n^n}{n!}\right )\to \frac {1}{2} \] as \(n\to \infty \).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with distribution function \[ F(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ t^3, & \text {if } 0\leq t\leq 1,\\ 1, & \text {if } t>0. \end {cases} \] Show that \[ Y_n=n^{1/3}X_{1,n} \] converges in distribution, where \(X_{1,n}=\min \{X_1,X_2,\ldots ,X_n\}\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ h(t,\theta )= \begin {cases} 0, & \text {if } t\notin [-\theta ,\theta ],\\ \dfrac {5}{2}\theta ^{-5}t^4, & \text {if } -\theta \leq t\leq \theta . \end {cases} \] Find a moment estimator for \(\theta \).

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ h(t,\theta )= \begin {cases} 0, & \text {if } -\infty <t<0,\\ \theta (t+1)^{-\theta -1}, & \text {if } 0\leq t<\infty , \end {cases} \] \(\theta >0\).

(a) Find the maximum likelihood estimator for \(\theta \).

(b) Is the estimator unbiased?

(c) Find the asymptotic variance of the maximum likelihood estimator for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \[ h(t,\theta )= \begin {cases} 0, & \text {if } t\notin [0,\theta ],\\ 2\theta ^{-2}t, & \text {if } 0\leq t\leq \theta . \end {cases} \] Find the uniformly minimum variance unbiased estimator for \(\theta \). Explain your answer. You need to prove directly that the sufficient statistic is also complete in this case.

Let \(X\) and \(Y\) be two independent random variables with density functions \[ f(t)= \begin {cases} 0, & \text {if } t\notin [0,4],\\ 1/4, & \text {if } 0\leq t\leq 4, \end {cases} \] and \[ h(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ 2e^{-2t}, & \text {if } 0\leq t<\infty . \end {cases} \] Compute the density of \(X-Y\).

Let \(X_1\) and \(X_2\) be independent random variables. The density function function of \(X_1\) is \[ f(t)= \begin {cases} 0, & \text {if } t\notin [0,1],\\ \dfrac {e^t}{e-1}, & \text {if } 0\leq t\leq 1. \end {cases} \] The distribution of \(X_2\) is \[ \mathbb {P}\{X_2=1\}=p \quad \text {and}\quad \mathbb {P}\{X_2=-1\}=q,\qquad p+q=1. \] Compute the moment generating function of \(X_1X_2\).

Let \(X_1,\ldots ,X_n\) be an i.i.d. sample from a \(\operatorname {Uniform}(0,\theta )\) distribution, where \(\theta >0\) is unknown. Find a 95% confidence interval for \(\theta \).

Let \(X_1,\ldots ,X_n\) denote an independent sample from an exponential distribution with [unknown] mean \(\theta >0\). What does the Neyman–Pearson lemma say about \(H_0:\theta =1\) versus \(H_a:\theta =2\)? Explain carefully, and identify explicitly the rejection region.

Let \(m\) denote the median distance [in 1000 miles] required for a certain brand of automobile tires to wear out. Test to see whether or not \(m\leq 29\), based on the following random sample: \[ 23\quad 20\quad 26\quad 25\quad 48\quad 26\quad 25\quad 24\quad 15\quad 20 \]

The following data are times (in hours) between failures of air conditioning equipment in a particular airplane: \[ 74\quad 57\quad 48\quad 29\quad 502\quad 12\quad 70\quad 21\quad 29\quad 386\quad 59\quad 27\quad 153\quad 26\quad 326. \] Assume that the data are observed values of an i.i.d. random sample from an exponential distribution, \(X_i\sim \operatorname {Exponential}(\theta )\). Test \(H_0:\theta =125\) versus \(H_a:\theta \ne 125\). (A chi-square table is provided.)

A sample of 400 people was asked their degree of support of a balanced budget and their degree of support of public education, with the following results:

Test the hypothesis of independence at \(\alpha =0.05\). (A chi-square table is provided.)

In this problem, all random variables are nonnegative. We say that \(X\) is stochastically dominated by \(Y\) if \(\mathbb {P}\{X>a\}\leq \mathbb {P}\{Y>a\}\) for all \(a>0\). Prove that if \(X\) is stochastically dominated by \(Y\), then \(\mathbb {E}\Phi (X)\leq \mathbb {E}\Phi (Y)\) for all increasing functions \(\Phi :\mathbf {R}_+\to \mathbf {R}_+\).

Suppose \(X_1,X_2,\ldots \) are independent, identically-distributed exponential random variables with mean \(\lambda >0\). Prove that \[ \max (X_1\ldots ,X_n)-\frac {1}{\lambda }\ln n\Rightarrow X, \] and compute \(\mathbb {P}\{X>x\}\) for all \(x>0\).

Give a rigorous proof that \(\mathbb {E}[XY]=\mathbb {E}[X]\mathbb {E}[Y]\) if \(X\) and \(Y\) are independent random variables belonging to \(L^1(\mathbb {P})\). In particular, show that \(XY\in L^1(\mathbb {P})\).

Fix \(n\geq 2\) and let \(X,Y_1,\ldots ,Y_n\) be jointly distributed random variables. We say that \(Y_1,\ldots ,Y_n\) are conditionally i.i.d. given \(X\) if \[ \mathbb {P}(Y_1\leq y_1,\ldots ,Y_n\leq y_n\mid X) =\mathbb {P}(Y_1\leq y_1\mid X)\cdots \mathbb {P}(Y_1\leq y_n\mid X) \]

for all \(y_1,\ldots ,y_n\). Show that, if \(Y_1,\ldots ,Y_n\) are conditionally i.i.d. given \(X\), then \[ \operatorname {Var}(Y_1+\cdots +Y_n)=n^2\operatorname {Var}(Y_1)-n(n-1)\mathbb {E}[\operatorname {Var}(Y_1\mid X)]. \]

A random experiment has exactly three possible outcomes, referred to as outcomes 1, 2, and 3, with probabilities \(p_1>0\), \(p_2>0\), and \(p_3>0\), where \(p_1+p_2+p_3=1\). We consider a sequence of independent trials, at each of which the specified random experiment is performed. For \(i=1,2\), let \(N_i\) be the number of trials needed for outcome \(i\) to occur, and put \(N:=\min (N_1,N_2)\).

(a) Show that \(N\) is independent of \(1_{\{N_1<N_2\}}\).

(b) Evaluate \(\mathbb {E}[N_1\mid N_1<N_2]\).

(c) Roll a pair of dice until a total of 6 or 7 appears. Given that 6 appears before 7, what is the (conditional) expected number of rolls?

If \(X\) is either (a) Poisson\((\lambda )\) or (b) gamma\((\lambda ,1)\) (density proportional to \(x^{\lambda -1}e^{-x}\), \(x>0\)), show that \((X-\mathbb {E}[X])/\sqrt {\operatorname {Var}(X)}\) converges in distribution to \(N(0,1)\) as \(\lambda \to \infty \) (\(\lambda \) need not be an integer).

Let \(X_1,X_2,\ldots \) be i.i.d. with mean \(\mu \) and finite variance. Show that \[ U_n:=\binom {n}{2}^{-1}\sum _{1\leq i<j\leq n}X_iX_j \] converges in probability to \(\mu ^2\) as \(n\to \infty \).

Let \(X_1,X_2,\ldots \) be i.i.d. with mean \(\mu \). Define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\). For fixed \(n\geq 2\), define \[ M_1:=\frac {S_n}{n},\qquad M_2:=\frac {S_{n-1}}{n-1},\qquad \ldots \qquad M_n:=\frac {X_1}{1}. \] (a) Show that \(\mathbb {E}[X_k\mid S_n]=S_n/n\) for \(1\leq k\leq n\).

(b) Show that \(M_1,M_2,\ldots ,M_n\) is a martingale.

Let \(Z\) be a random variable with all moments finite. Choose \(X\) and \(Y\) appropriately as in the Cauchy–Schwarz inequality or the Hölder inequality to prove that \(f(p):=\ln \mathbb {E}[|Z|^p]\) is convex on \((0,\infty )\).

Let \(X\) be an exponential random variable with mean 1, that is, \(\mathbb {P}(X>u)=e^{-u}\) for all \(u>0\). Evaluate \(\mathbb {E}[X\mid X\wedge t]\) and \(\mathbb {E}[X\mid X\vee t]\) for all \(t>0\). Here \(a\wedge b:=\min (a,b)\) and \(a\vee b:=\max (a,b)\).

Let \(X\) be a random variable with values in an interval \(I\), and suppose that \(f\) and \(g\) are nondecreasing functions on \(I\) such that \(f(X)\) and \(g(X)\) have finite variance. Show that \(\operatorname {Cov}(f(X),g(X))\geq 0\).

Hint: If \(X_1\) and \(X_2\) are i.i.d. as \(X\), then \((f(X_1)-f(X_2))(g(X_1)-g(X_2))\geq 0\).

Let \(X_1,X_2,\ldots \) be independent with \[ \mathbb {P}(X_n=n^2-1)=n^{-2}=1-\mathbb {P}(X_n=-1),\qquad n\geq 1. \] Notice that \(\mathbb {E}[X_n]=0\) for all \(n\geq 1\), which might lead one to expect that \(S_n:=X_1+\cdots +X_n\) satisfies \(S_n/n\to 0\) a.s. Show that in fact \(S_n/n\to -1\) a.s.

Let \(X_1,X_2,\ldots \) be an i.i.d. sequence with \(\mathbb {P}(X_1=1)=p\) and \(\mathbb {P}(X_1=0)=1-p\), where \(0<p<1\). For each \(m\geq 1\), define \[ N_m:=\min \{n\geq m:(X_{n-m+1},\ldots ,X_n)=(1,1,\ldots ,1)\} \] and show that \[ \mathbb {E}[N_m]=\frac {1}{p}+\frac {1}{p^2}+\cdots +\frac {1}{p^m}. \] Hint: Evaluate \(\mathbb {E}[N_m]\) by conditioning on \(N_{m-1}\).

Consider a deck of \(n\) cards labeled \(1,2,\ldots ,n\). Assume it is well shuffled with every possible arrangement equally likely. Let \(E_n\) be the event that card \(j\) is in position \(j\) in the shuffled deck for some \(j\in \{1,2,\ldots ,n\}\). Evaluate \(\mathbb {P}(E_n)\) using inclusion-exclusion and show that \(\lim _{n\to \infty }\mathbb {P}(E_n)=1-e^{-1}\).

Let \(X_1,X_2,\ldots \) be i.i.d. uniform\((0,2)\), and put \(M_n:=X_1X_2\cdots X_n\) for each \(n\geq 1\). Show that \(\{M_n\}_{n=1}^{\infty }\) is a martingale. What does the martingale convergence theorem tell us about \(\lim _{n\to \infty }M_n\)? (In particular, evaluate this limit.)

Suppose \(X_1\) and \(X_2\) are independent random variables that satisfy the following three properties: (i) \(Y_1:=(X_1+X_2)/\sqrt {2}\) is standard normal; (ii) \(Y_2:=(X_1-X_2)/\sqrt {2}\) is standard normal; and (iii) \(Y_1\) and \(Y_2\) are independent. Prove that \(X_1\) and \(X_2\) are normally distributed. Can you compute their respective means and variances?

Prove that if \(\mathbb {P}\{X\geq 0\}=1\) and \(0<\mathbb {E}(X^2)<\infty \), then \[ \mathbb {P}\{X=0\}\leq \frac {\operatorname {Var} X}{\mathbb {E}(X^2)}. \]

Let \(X\) and \(Y\) be independent, standard-normal random variables. Find the distribution of \(X/Y\)?

Let \(X_1,X_2,\ldots \) be a sequence of i.i.d. random variables, each taking the values 0 and 1 with equal probabilities \(1/2\). Define \[ U:=\sum _{n=1}^{\infty }\frac {X_n}{2^n}. \] Prove that \(U\) is distributed uniformly on \((0,a)\) and compute \(a\).

Let \(X_1\) and \(X_2\) be independent, each with probability density function \(f(x)=1/x^2\), \(x>1\). Find the joint probability density function of \(Y_1=X_1X_2\) and \(Y_2=X_1/X_2\). It is important to state where your formula is valid. (If you have time, check that your joint pdf integrates to 1; if it doesn’t, find the error.)

Let \(X_1,\ldots ,X_n\) be independent \(\operatorname {LogNormal}(\mu ,\sigma ^2)\) (i.e., \(X_1=e^{Y_1},\ldots ,X_n=e^{Y_n}\) with \(Y_1,\ldots ,Y_n\) being independent \(N(\mu ,\sigma ^2)\)). Argue that the sample median is asymptotically normal and find the asymptotic mean and variance.

Let \(X_1,\ldots ,X_n\) be a random sample from \(N(\theta ,\theta )\), where \(\theta >0\). In particular, the population mean and the population variance are equal but unknown.

(a) Find the method-of-moments estimator of \(\theta \) based on the second sample moment.

(b) Find the maximum likelihood estimator of \(\theta \).

Let \(X_1,\ldots ,X_n\) be a random sample from a \(N(\theta ,1)\) distribution, and let the prior distribution of \(\theta \) be \(N(\mu _0,1)\), where \(\mu _0\) is known.

(a) Find the posterior distribution of \(\theta \).

(b) Find the Bayes estimator of \(\theta \) under squared error loss, and show that it is a weighted average of the prior mean \(\mu _0\) and the sample mean \(\bar X\).

Let \(X_1,\ldots ,X_n\) be a random sample from \(\operatorname {Poisson}(\theta )\).

(a) Find a complete, sufficient statistic.

(b) Noting that \(\Prob _\theta (X_1=1)=\theta e^{-\theta }\), use the Rao–Blackwell and Lehmann–Scheffé theorems to find a UMVUE of \(\tau (\theta ):=\theta e^{-\theta }\).

Assume a sample of size \(n\) from \(\operatorname {Uniform}[\eta -\theta ,\eta +\theta ]\), which is a location-scale family. Here \(\eta \) is real and \(\theta >0\).

(a) Find a \(100(1-\alpha )\%\) confidence interval for \(\theta \).

(b) Find a \(100(1-\alpha )\%\) confidence interval for \(\eta \).

It is not necessary to find the explicit distribution of the pivotal quantities; just denote the needed quantiles by \(x_p\) and \(y_p\) for an appropriate \(p\).

Let \(X_1,\ldots ,X_n\) be a random sample from \(\operatorname {Uniform}[0,\theta ]\). Find the UMP test of size \(\alpha \) of \(H_0:\theta \geq \theta _0\) vs. \(H_a:\theta <\theta _0\) by first deriving a most powerful test of simple hypotheses and then extending it to composite hypotheses.

Let \(X_1,\ldots ,X_n\) be a random sample from \(N(\mu ,\sigma _0^2)\), where \(\sigma _0^2\) is known. Find the generalized likelihood ratio test of size \(\alpha \) of \(H_0:\mu \geq \mu _0\) vs. \(H_a:\mu <\mu _0\).

According to a genetic model the proportions of individuals having the four blood types should be given by O: \(q^2\); A: \(p^2+2pq\); B: \(r^2+2qr\); AB: \(2pr\). Here \(p>0\), \(q>0\), \(r>0\), and \(p+q+r=1\), but the three parameters are unknown. We observe \(n_O\), \(n_A\), \(n_B\), \(n_{AB}\) individuals of the four blood types in a random sample of size \(n\). Describe a goodness-of-fit test of size \(\alpha \) of the stated hypothesis.

Let \(X_1,X_2,\ldots \) be i.i.d. with mean \(\mu \) and variance \(\sigma ^2\), and put \(S_0:=0\) and \(S_n:=X_1+\cdots +X_n\) for each \(n\geq 1\). Let \(N\) be a nonnegative-integer-valued random variable independent of \(X_1,X_2,\ldots \). Find the mean and variance of \(S_N\) by conditioning on \(N\).

(a) Find a closed-form expression (i.e., one with no summation sign) for \(\mathbb {E}[(X+1)^{-1}]\) when \(X\) is binomial\((n,p)\) with \(n\geq 1\) and \(0<p<1\).

(b) Find a closed-form expression for \(\mathbb {E}[(X+1)^{-1}]\) when \(X\) is Poisson\((\lambda )\) with \(\lambda >0\). You may use the result of part (a) here.

For \(n=1,2,\ldots \), let \(X_n\) have density \[ f_n(x):=\frac {n}{\pi (1+n^2x^2)},\qquad -\infty <x<\infty . \] (a) Does \(X_n\) converge to 0 in probability?

(b) Does \(X_n\) converge to 0 in \(L^1(\mathbb {P})\)?

(c) If \(X_1,X_2,\ldots \) are independent, does \(X_n\) converge to 0 a.s.?

Let \(X_1,X_2,X_3,Y_1,Y_2,Y_3\) be independent \(N(0,1)\) random variables. Let \(P_1\), \(P_2\), and \(P_3\) be the points in the plane with coordinates \((X_1,Y_1)\), \((X_2,Y_2)\), and \((X_3,Y_3)\). Let \(M\) be the midpoint of the segment \(P_1P_2\) and let \(r\) be half its length. Show that the probability that the point \(P_3\) lies within the circle of radius \(r\) centered at \(M\) is equal to \(1/4\).

Suppose \(X_n\Rightarrow X\) as \(n\to \infty \), and \(\sup _{n\geq 1}\mathbb {E}(X_n^2)<\infty \). Prove that \(X\in L^1(\mathbb {P})\) and \(\lim _{n\to \infty }\mathbb {E} X_n=\mathbb {E} X\).

Choose and fix a sequence of real numbers \(a_1,a_2,\ldots \) such that: (i) \(0<a_k<1\) for all \(k\geq 1\); and (ii) \(\sum _{k=1}^{\infty }a_k^2=\infty \). Let \(X_1,X_2,\ldots \) be independent random variables, where each \(X_k\) is distributed uniformly on \((-a_k,a_k)\). Prove that \(n^{-1/2}(X_1+\cdots +X_n)\) converges weakly to \(N(0,\sigma ^2)\) and compute \(\sigma \).

Suppose \(f\) is a probability density function on \(\mathbf {R}\), and define for all \(x,y\in \mathbf {R}\), \[ g(x,y):= \begin {cases} 2f(x)f(y), & \text {if } x\leq y,\\ 0, & \text {otherwise.} \end {cases} \] (a) Prove that \(g\) is the probability density function of some random variable \((X,Y)\) which takes values in \(\mathbf {R}^2\).

(b) Are \(X\) and \(Y\) independent? Prove or disprove.

Suppose \(X_1,X_2,\ldots \) are independent, identically-distributed random variables such that \[ \mathbb {P}\{X_1>x\}= \begin {cases} x^{-\alpha }, & \text {if } x\geq 1,\\ 1, & \text {if } x<1, \end {cases} \] where \(\alpha >0\) is fixed. Prove that \[ n^{-1/\alpha }\max (X_1,\ldots ,X_n)\Rightarrow X, \] and compute \(\mathbb {P}\{X>x\}\) for all \(x\in \mathbf {R}\).

Let \(\alpha ,\lambda \) be fixed, strictly positive, and finite. The Gamma\((\alpha ,\lambda )\) density function is \[ f(x):= \begin {cases} \dfrac {\lambda ^\alpha }{\Gamma (\alpha )}x^{\alpha -1}e^{-\lambda x}, & \text {if } x>0,\\ 0, & \text {if } x\leq 0, \end {cases} \] where \(\Gamma (\alpha ):=\int _0^\infty t^{\alpha -1}e^{-t}\,dt\). This defines a probability density function.

(a) Compute the characteristic function of \(f\).

(b) Suppose \(X_1,\ldots ,X_n\) are independent with common density Gamma\((1,\lambda )\). Prove that \(S_n:=X_1+\cdots +X_n\) has a Gamma density; also compute its parameters.

(c) Compute \(\mathbb {P}\{S_n<k<S_{n+1}\}\) for all integers \(k\geq 1\).

Let \(U_1,U_2,\ldots ,U_n\) be independent random variables uniform on \([0,1]\) and let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) be the corresponding order statistics.

a. Compute the asymptotic distribution of \(nU_{2,n}\). (5 points)

b. Show that \(nU_{1,n}\) and \(n(1-U_{n,n})\) are asymptotically independent. (5 points)

Let \(X\) and \(Y\) be two independent random variables with distribution functions \(F\) and \(G\) and density functions \(f\) and \(g\).

a. Determine the distribution function of \(XY\). (5 points)

b. Does \(XY\) always have a density function? (5 points)

c. Assuming that \(XY\) has a density function, compute it. (5 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ f(t;\theta )= \begin {cases} 0, & \text {if } t<\theta ,\\ e^{-(t-\theta )}, & \text {if } t>\theta . \end {cases} \]

a. Find a moment estimator for \(\theta \). (5 points)

b. Find the maximum likelihood estimator for \(\theta \). (5 points)

c. Determine the asymptotic efficiency between the two estimators. (5 points)

The number of customers entering a store on a given day is a Poisson random variable with parameter \(\theta \). The money spent by a customer is uniformly distributed on \([0,\eta ]\). The number of customers and the money spent in the store are independent. We observe \(Z_1,Z_2,\ldots ,Z_n\), the total spending (the money collected by the store) on \(n\) days. We can assume that \(Z_1,Z_2,\ldots ,Z_n\) are independent.

a. Provide estimators for \(\theta \) and \(\eta \) using the method of moments. (10 points)

b. Provide estimators for \(\theta \) and \(\eta \) using the likelihood method. (10 points)

Let \(X_1,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We assume that \(F\) is strictly increasing and continuous on its support. Let \(x_{1/2}\) denote the median.

a. Find a \(1-\alpha \) confidence interval for \(x_{1/2}\). (5 points)

b. Provide a large sample approximation for the confidence interval. (5 points)

Let \(X_1,X_2,\ldots \) be random variables such that the partial sums \(S_n:=X_1+X_2+\cdots +X_n\) determine a martingale. Show that \(\mathbb {E}[X_iX_j]=0\) if \(i\neq j\).

Let \(\{X_n,n\geq 1\}\) be independent \(N(0,1)\) random variables. Show that \[ \Prob \left (\limsup \frac {|X_n|}{\sqrt {\log n}}=\sqrt {2}\right )=1. \] Hint: \(1-\Phi (x)\) is asymptotic to \(\phi (x)/x\) as \(x\to \infty \).

Let \(X\) and \(Y\) be independent exponential random variables with parameters \(\lambda \) and \(\mu \). (For example, \(\mathbb {P}(X>x)=e^{-\lambda x}\) for \(x>0\).)

(a) Show that \(Z:=\min (X,Y)\) is independent of the event \(\{X<Y\}\).

(b) Find \(\mathbb {P}(X=Z)\).

(c) Find the distribution of \(U:=(X-Y)^+=\max (X-Y,0)\).

(d) Find the distribution of \(V:=\max (X,Y)-\min (X,Y)\).

Suppose \(W:=(X,Y)\) is a two-dimensional random variable and \(QW^T\) is \(N(0,1)\) for all \(2\times 2\) rotation matrices \[ Q:= \begin {pmatrix} \cos \theta & \sin \theta \\ -\sin \theta & \cos \theta \end {pmatrix}. \] Prove that \(X\) and \(Y\) are independent and \(N(0,1)\).

Let \(X_1,X_2,\ldots \) and \(Y_1,Y_2,\ldots \) denote random variables on a common probability space. Suppose \(X_n\) converges to 0 in probability as \(n\to \infty \), and \(Y_n\) converges to \(Y\) weakly. Prove that \((X_n,Y_n)\) converges weakly to \((0,Y)\).

Construct two random variables \(X\) and \(Y\) such that:

(a) \(X\) is \(N(0,1)\),

(b) \(Y\) is \(N(0,1)\), and

(c) \((X,Y)\) is not a Gaussian random vector.

Suppose \(X\) and \(Y\) are independent bounded random variables. Prove in complete detail the following assertions:

(a) \(\mathbb {E}(XY)=\mathbb {E}(X)\mathbb {E}(Y)\),

(b) \(\mathbb {E}(X\mid Y)=\mathbb {E}(X)\).

Let \(X_i\) be a random sample of size \(n\) with pdf \(f(x;\eta )=e^{-(x-\eta )}\) (\(x>\eta \), \(\eta >0\)).

1a) (5 points) Find the MLE \(\hat \eta \) and compute its pdf. Do you recognize it?

1b) (5 points) Is \(\hat \eta \) unbiased, asymptotically unbiased, MSE consistent, simple consistent?

Let \(X_i\) be a random sample of size \(n\) with pdf \(f(x;\theta )=\frac {1}{\theta }e^{-(x-\theta )/\theta }\) (\(x>\theta \), \(\theta >0\)).

2a) (5 points) What is the MME of \(\theta \)? Is it a reasonable estimator?

2b) (5 points) What is the MLE of \(\theta \)?

(10 points) Let \(X_i\sim \operatorname {POI}(\mu )\) be a random sample of size \(n\). Find the MLE of \(e^{-\mu }\). Is it unbiased, asymptotically unbiased, MSE consistent, simple consistent? [Hint: The MGF of \(\operatorname {POI}(\mu )\) is \(e^{\mu (e^t-1)}\).]

Let \(X_i\) be a random sample of size \(n\) with pdf \(f(x;\theta )=\theta x^{\theta -1}\) (\(0<x<1\), \(\theta >0\).)

4a) (5 points) Find a complete and sufficient statistic.

4b) (5 points) Find a uniformly minimum variance unbiased estimator (UMVUE) of \(1/\theta \).

Let \(X\) and \(Y\) be independent standard normals and consider \((U,V)=(aX+bY,cX+dY)\). Assume that at least one of \(a\) and \(b\) is nonzero, and at least one of \(c\) and \(d\) is nonzero. (Otherwise either \(U\) or \(V\) would be 0 and the problem is not interesting.)

5a) (5 points) Show that if \(ad=bc\), then \(U\) and \(V\) cannot be independent random variables.

5b) (5 points) Assume \(ad\ne bc\) and find the joint pdf of \((U,V)\).

5c) (5 points) Find a necessary and sufficient condition for \(U\) and \(V\) to be independent.

Let \(X_i\sim \operatorname {EXP}(\theta )\) be a random sample of size \(n\).

7a) (5 points) Find the MGF of \(\frac {2n\bar X}{\theta }\) and identify its distribution.

7b) (10 points) Derive the generalized likelihood ratio test of \(H_0:\theta =\theta _0\) against \(H_a:\theta \ne \theta _0\).

7c) (5 points) The following data are times (in hours) between failures of air conditioning equipment in a particular airplane: 74, 57, 48, 29, 502, 12, 70, 21, 29, 386, 59, 27, 153, 26, 326. Test \(H_0:\theta =125\) against \(H_a:\theta \ne 125\).

(5 points) Let \(\zeta _n\) and \(\eta _n\) be two sequences of random variables. Prove or give counterexamples to the following statements:

9a) If \(\zeta _n\xrightarrow {(d)}\zeta \) and \(\eta _n\xrightarrow {(d)}\eta \), then \(\zeta _n+\eta _n\xrightarrow {(d)}\zeta +\eta \).

9b) If \(\zeta _n\xrightarrow {P}\zeta \) and \(\eta _n\xrightarrow {P}\eta \), then \(\zeta _n+\eta _n\xrightarrow {P}\zeta +\eta \).

9c) If \(\zeta _n\xrightarrow {(d)}\zeta \) and \(\eta _n\xrightarrow {(d)}\eta \), then \(\zeta _n+\eta _n\xrightarrow {P}\zeta +\eta \).

(\(\xrightarrow {(d)}\) and \(\xrightarrow {P}\) denote convergence in distribution and in probability, respectively.)

Let \(X\) and \(Y\) have joint density \[ f(x,y)=g\left (\sqrt {x^2+y^2}\right ),\qquad (x,y)\in \mathbb {R}^2, \] for some function \(g\). Show that \(Z=X/Y\) has a Cauchy density.

(a) State the central limit theorem as well as Kolmogorov’s 0-1 law.

(b) For arbitrary events \(A_n\), \(n\geq 1\), show that \(\inf _{n\geq 1}\mathbb {P}(A_n)>0\) implies \(\mathbb {P}(A_n\ \text {i.o.})>0\).

(c) Let \(X_1,X_2,\ldots \) be i.i.d. mean 0, variance 1 random variables, and put \(S_n=X_1+\cdots +X_n\) for each \(n\geq 1\). Show that \(\limsup _{n\to \infty }S_n=\infty \) a.s.

Construct three random variables \(X,Y,Z\) such that \[ \mathbb {E}[\mathbb {E}[X\mid Y]\mid Z]\neq \mathbb {E}[\mathbb {E}[X\mid Z]\mid Y] \] with positive probability.

Let \(X_1,X_2,\ldots \) be a sequence of integrable random variables defined on the same probability space. Show that \(\{X_n\}_{n\geq 1}\) is a submartingale if and only if \(\mathbb {E}[X_T]\geq \mathbb {E}[X_1]\) for all bounded stopping times \(T\). (Take the filtration to be the natural one, \(\mathcal {F}_n=\sigma (X_1,\ldots ,X_n)\).)

Let \(X_1,X_2,\ldots ,X_n\) be i.i.d. positive random variables. Show that \[ \mathbb {E}\left [ \frac {\sum _{i=1}^m X_i}{\sum _{i=1}^n X_i} \right ] = \frac {m}{n},\qquad 1\leq m\leq n. \]

Suppose \(X_{1,n},X_{2,n},\ldots \) are i.i.d. with \(\mathbb {P}\{X_{1,n}=j\}=1/n\) for \(j=1,\ldots ,n\). Prove that if \[ T_n:=\inf \{k\geq 1:X_{1,n}+\cdots +X_{k,n}>n\}, \] then \(\lim _{n\to \infty }\mathbb {P}\{T_n=k\}\) exists for all \(k\geq 1\). Compute that limit.

Let \(X_1,X_2,\ldots \) be i.i.d. mean 0, variance 1 random variables and \(S_n=X_1+\cdots +X_n\) for each \(n\geq 1\). Show directly that the characteristic function of \(S_n/\sqrt {n}\) converges pointwise to the characteristic function of the standard normal distribution. Notice that this is a key step in the proof of the central limit theorem, so you are not allowed to use the central limit theorem in your derivation.

Let \(X_1,X_2,\ldots \) be i.i.d. with \(\mathbb {P}(X_1>x)=e^{-x}\), \(x>0\). Prove that \[ \limsup _{n\to \infty } \frac {X_n}{\ln n}=1 \] a.s.

Show that the number of fixed points in a random permutation of length \(n\) is asymptotically Poisson with mean 1. Hint: Use characteristic functions. Write \(e^{it1_A}\) as \(1+(e^{it}-1)1_A\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We use \[ \hat f_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n} K\left (\frac {t-X_i}{h}\right ) \] to estimate \(f=F'\).

We assume that the following conditions are satisfied: \[ \int _{-1}^{1}K(u)\,du=1, \qquad K(t)=0,\quad \text {if } t\notin [-1,1]. \] \(F\) is three times differentiable and the derivatives are uniformly bounded on the real line \[ h=h(n)\to 0. \]

(i) Compute the bias of \(\hat f_n(t)\) for any \(t\) and any \(n\). (7 points)

(ii) Prove that \(\hat f_n(t)\) is asymptotically unbiased for any \(t\) as \(n\to \infty \). (7 points)

(iii) Compute the mean square error of \(\hat f_n(t)\) (7 points)

Let \(X\) be a random variable with distribution function \(F\). Prove that \(X\) is symmetric around 0 (i.e. \(F(x)=1-F(-x)\) for all \(x\)) if and only if the characteristic function of \(X\) is real. (7 points)

Let \(X_1\) and \(X_2\) be independent standard normal random variables. Find the necessary and sufficient condition for the independence of \(Y_1=a_1X_1+a_2X_2\) and \(Y_2=b_1X_1+b_2X_2\). (7 points)

Let Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \[ F(t)=\left (\frac {t}{\theta }\right )^2 \quad \text {if } 0\leq t\leq \theta . \]

(i) Find the maximum likelihood estimator for \(\theta \). (7 points)

(ii) Compute the limit distribution of the maximum likelihood estimator. (7 points)

Let Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with \(\E X_1=\mu \), \(\operatorname {Var} X_1=\sigma ^2\). We assume that \(\E X_4<\infty \). The random variable \[ \hat \sigma _n^2=\frac {1}{2n}\sum _{2\leq i\leq n}(X_i-X_{i-1})^2 \] is suggested to estimate \(\sigma ^2\).

(i) Establish some basic properties of \(\hat \sigma _n^2\). (7 points)

(ii) Is \(\hat \sigma _n^2\) a better estimator than the sample variance? Justify your answer. (7 points)

Let \((X_1,\ldots ,X_m)\) be a multinomial random vector with parameters \(N\) and \(p_1,p_2,\ldots ,p_m\). Note that \(N=X_1+\cdots +X_m\) and \(p_1+p_2+\cdots +p_m=1\). Find the maximum likelihood estimators for \(p_1,p_2,\ldots ,p_m\). (7 points)

Let \(X\) and \(Y\) be independent random variables with density function \(f\) and \(g\). Assume that \(Y>0\). Show that the function \[ h(t)=\int _{0}^{\infty } z f(zt)g(z)\,dz \] can be chosen as the density of \(Z=X/Y\). (7 points)

Let \(X_1, X_2, \ldots \) be an i.i.d. sample from the uniform distribution on \([\theta , 1+\theta ]\), where \(\theta > 0\) is unknown. Is there an unique maximum likelihood estimator of \(\theta \) that is based on \(\{X_i\}_{i=1}^{\infty }\)? Prove or disprove.

Suppose \(X_1, \ldots , X_n\) is an i.i.d. sample from the exponential density with mean \(\theta \), where \(\theta > 0\) is unknown. Find an estimator of \(\theta \) based on \(X_1, \ldots , X_n\). Is your estimator complete? Is it sufficient? Justify your reasoning.

Let \(X\) have the binomial distribution with parameters \(n=10\) and \(p=\theta \) unknown. Suppose we impose a uniform-\((0,1)\) prior on \(\theta \). Compute the posterior mean of \(\theta \). Is it consistent?

Let \(X_1, X_2, \ldots \) denote an i.i.d. sample with common distribution function \(F\). Define \[ F_n(x) = n^{-1}\sum _{j=1}^{n} I\{X_j \leq x\} \] for all \(x \in (-\infty , \infty )\); i.e., \(F_n\) is the empirical distribution function based on the \(X_i\)’s. Prove that for all \(x \in (-\infty , \infty )\) fixed, \[ \operatorname {Var}\bigl (F_n(x)-F(x)\bigr ) \leq \frac {1}{4n}. \] Use this to conclude that \(F_n(x)\) is a consistent estimator for \(F(x)\).

From here on let \(X_1, X_2, \ldots \) denote an i.i.d. sample from an exponential density with mean \(1/\theta \).

Is the MLE a consistent estimator of \(\theta \)? Prove or disprove.

Find a confidence interval for \(\theta \) that has asymptotic level \((1-\alpha )\%\).

Develop carefully the likelihood ratio test for \(H_0:\theta =1\) versus \(H_1:\theta =2\). Discuss the various optimality properties—if any—of your test.

Compute the distributions of: \[ \begin {array}{ll} \text {(a)} & \max _{1\leq i\leq n} X_i,\\ \text {(b)} & \min _{1\leq i\leq n} X_i,\\ \text {(c)} & X_1+\cdots +X_n. \end {array} \]

If \(X\) is distributed uniformly on \([0,1]\), then compute \(\mathbb {E}[X\mid \mathcal {G}]\), where \(\mathcal {G}\) is the \(\sigma \)-algebra generated by \(\{X\le 1/2\}\).

Let \(\{X_i\}_{i=1}^{\infty }\) be a sequence of i.i.d. random variables, and define \[ S(r):=\sum _{i=1}^{\infty }(1-r)^iX_i \qquad \text {for all } 0<r<1. \]

(a) Prove that for every fixed \(r\in (0,1)\), \(S(r)\) is a.s. absolutely convergent if and only if \(\mathbb {E}\{\log (1+|X_1|)\}<\infty \).

(b) Suppose that \(\mathbb {E}[X_1]=\mu \) and \(\operatorname {Var}(X_1)=\sigma ^2<\infty \). Then prove that there exists a constant \(c\) such that \(rS(r)\to c\) in probability, as \(r\to 0\). Compute \(c\).

Let \(\{X_n\}_{n=1}^{\infty }\) denote a martingale such that \(X_1\in L^2(\mathbb {P})\), and \[ \sum _{n=1}^{\infty }\mathbb {E}\{(X_{n+1}-X_n)^2\}<\infty . \] Prove that \(\lim _{n\to \infty }X_n\) exists a.s. and in \(L^2(\mathbb {P})\).

Let \(X_1,X_2,\ldots \) be i.i.d., and \(S_n:=X_1+\cdots +X_n\). Assume also that \(\mathbb {P}\{X_1=1\}=\mathbb {P}\{X_1=-1\}=1/2\), so that \(\{S_n\}_{n=1}^{\infty }\) is a simple symmetric random walk. Define \[ T:=\inf \{k\ge 1: |S_k|\ge 2\}. \] As usual, \(\inf \emptyset :=\infty \). Prove that \(\mathbb {E}[T^2]<\infty \).

Let \(\{X_n\}_{n=1}^{\infty }\) be a sequence of independent (but not identically distributed) random variables, such that for all \(n\ge 1\), \[ \mathbb {P}\{X_n=1\}=\mathbb {P}\{X_n=-1\}=\frac {1}{2}-\frac {1}{2n^2}, \qquad \mathbb {P}\{X_n=n\}=\mathbb {P}\{X_n=-n\}=\frac {1}{2n^2}. \] Prove that \(n^{-1/2}\sum _{i=1}^{n}X_i\Rightarrow N(0,\sigma ^2)\), and compute \(\sigma ^2\) explicitly.

Let \(\{X_i\}_{i=1}^{\infty }\) denote a sequence of i.i.d. random variables with \(\mathbb {P}\{X_1=0\}=p\) and \(\mathbb {P}\{X_1=1\}=q:=1-p\). Consider the random sequence \(X_1X_2\cdots \), and compute \(\mathbb {E}[T]\), where \(T\) denotes the first time that the pattern “001” appears in the said sequence.

Construct an example of a sequence \(\{A_n\}_{n=1}^{\infty }\) such that:

(a) (i) \(\sum _n\mathbb {P}(A_n)=\infty \); and (ii) only finitely-many of the \(A_n\)’s occur with positive probability.

(b) (i) \(\sum _n\mathbb {P}(A_n)=\infty \); and (ii) infinitely-many of the \(A_n\)’s occur with positive probability.

Let \(X\) and \(Y\) be two standard-normal random variables.

(a) Prove that if \((X,Y)\) is Gaussian then \(X\) and \(Y\) are independent.

(b) Construct an example wherein \((X,Y)\) is not a (two-dimensional) Gaussian random variable.

Let \(\{X_i\}_{i=1}^{\infty }\) be i.i.d. with the “standard Cauchy distribution.” That is, the density function of \(X_1\) is \[ f(x)=\frac {1}{\pi (1+x^2)} \qquad -\infty <x<\infty . \] Prove that \(\max _{1\le i\le n}X_i/n\) converges weakly. Identify the limit.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ g(t;\theta )= \begin {cases} 0, & \text {if } -\infty <t<\theta ,\\ e^{-(t-\theta )}, & \text {if } \theta \leq t<\infty . \end {cases} \] We wish to test \(H_0:\theta \leq \theta _0\) against \(H_A:\theta >\theta _0\).

(a) Find the uniformly most powerful test of size \(\alpha \).

(b) Compute the power function.

Let \(X_1,\ldots ,X_n\) be independent, identically distributed random variables with a continuous distribution function \(F\). Let \(x_{1/4}\) be defined by \(F(x_{1/4})=1/4\).

(a) Find a \(1-\alpha \) lower confidence bound for \(x_{1/4}\).

(b) Provide a large sample approximation.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with distribution function \[ F(t)=\frac {1}{1+e^{-t}},\qquad -\infty <t<\infty . \] Show that \[ Y_n=X_{n,n}-\log n \] converges in distribution, where \(X_{n,n}=\max \{X_1,X_2,\ldots ,X_n\}\).

Let \(X\) and \(Y\) be independent random variables with densities, \[ g(t)= \begin {cases} 0, & \text {if } t\notin [-2,2],\\ 1/4, & \text {if } t\in [-2,2], \end {cases} \] and \[ f(t)= \begin {cases} 0, & \text {if } t\notin [-1,1],\\ 1/2, & \text {if } t\in [-1,1]. \end {cases} \] Compute the density of \(X+Y\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with probability mass function \[ P\{X_i=1\}=\theta \quad \text {and}\quad P\{X_i=0\}=1-\theta . \] Let: \[ \bar {X}_n=\frac {1}{n}\sum _{1\leq i\leq n} X_i \] and \(h(t)=t^3-t\). Find the sequences \(a_n\) and \(b_n\) such that \(a_n\bigl (h(\bar {X}_n)-h(\theta )-b_n\bigr )\) has a non-degenerate limit distribution.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with a continuous probability distribution function \(F\) and \(Y_1,Y_2,\ldots ,Y_m\) be independent, identically distributed random variables with a continuous probability distribution function \(H\). The two samples are independent and \(F=H\). Let \[ T=\sum _{1\leq i\leq n}\sum _{1\leq j\leq m} I\{X_i\leq Y_j\}. \] Compute \(E T\) and \(\operatorname {var}T\).

Prove that for all submartingales \(\{X_n\}_{n=1}^{\infty }\), \[ \mathbb {P}\left \{\max _{n\le m}|X_n|\ge \lambda \right \} \le \frac {3}{\lambda }\max _{n\le m}\mathbb {E}\{|X_n|\}, \qquad \lambda >0,\; m=1,2,\ldots . \]

Suppose \(X_1,X_2,\ldots \) are independent with distribution, \[ \mathbb {P}\{X_n=n^\alpha \}=\mathbb {P}\{X_n=-n^{-\alpha }\}=\frac {1}{2}, \qquad n=1,2,\ldots . \] Prove that \(\sum _{i=1}^{n}X_n/n\to 0\) a.s. if \(0<\alpha <\frac {1}{2}\).

Suppose \(X_{\alpha ,\beta }\) has a Gamma distribution with parameters \((\alpha ,\beta )\). That is, the density function of \(X_{\alpha ,\beta }\) is \[ f(x)=\frac {\beta ^\alpha }{\Gamma (\alpha )}x^{\alpha -1}e^{-\beta x}, \qquad x>0, \] where \(\Gamma (\alpha )=\int _0^\infty x^{\alpha -1}e^{-x}\,dx\) for all \(\alpha >0\). Prove that \((\beta X_{\alpha ,\beta }-\alpha )/\sqrt {\alpha }\Rightarrow N(0,1)\).

Let \(X\) be a mean-zero random variable with \(\mathbb {E}(X^2)=\sigma ^2<\infty \). Prove Cantelli’s inequality: For all \(\lambda >0\), \[ \mathbb {P}\{X>\lambda \}\le \inf _{t>0}\left [\frac {t^2+\sigma ^2}{(t+\lambda )^2}\right ] =\frac {\sigma ^2}{\sigma ^2+\lambda ^2}. \] Prove that this is a better bound than Chebyshev’s inequality.

If \(X\in L^2(\Omega ,\mathcal {F},\mathbb {P})\), then prove that for all sub-\(\sigma \)-algebras \(\mathcal {G}\) of \(\mathcal {F}\), \[ \operatorname {Var}(X)=\mathbb {E}[\operatorname {Var}(X\mid \mathcal {F})]+\operatorname {Var}(\mathbb {E}\{X\mid \mathcal {F}\}) \qquad \text {a.s.} \]

Suppose \(X_1,X_2,\ldots \) have common mean \(\mu \) and variances \(\sigma _1^2,\sigma _2^2,\ldots \). Prove that if \(\sup _n\sigma _n^2<\infty \) and \(\lim _{|i-j|\to \infty }\mathbb {E}[X_iX_j]=0\), then \((X_1+\cdots +X_n)/n\) converges to \(\mu \) in probability.

Prove that whenever \(X_n\Rightarrow X\) and \(Y_n\to c\) in probability for a non-random \(c>0\), then \(X_n/Y_n\Rightarrow c^{-1}X\). Use this to prove the following: If \(\xi _1,\xi _2,\ldots \) are i.i.d. with \(\mathbb {E}[\xi _1]=\mu \) and \(\operatorname {Var}(\xi _1)=\sigma ^2\) then \[ \frac {S_n-n\mu }{\sqrt {\sum _{i=1}^{n}(\xi _i-\bar {\xi }_n)^2}} \Rightarrow N(0,1). \] where \(S_n=\xi _1+\cdots +\xi _n\), and \(\bar {\xi }_n=S_n/n\).

Suppose \(\{X_n\}_{n=1}^{\infty }\) is a martingale that is bounded in \(L^1(\mathbb {P})\); i.e., \(\sup _n\|X_n\|_1<\infty \). Use Doob’s decomposition to prove that \(X_T\in L^1(\mathbb {P})\) for all stopping times \(T\).

Suppose \(\{X_n\}_{n=1}^{\infty }\) is a sequence of random variables that has the property that \(\sup _n|X_n|\le 1\) a.s. Then use Doob’s decomposition to prove that \(\sum _n X_n\) converges a.s. iff \(\sum _n\mathbb {E}[X_n\mid X_1,\ldots ,X_{n-1}]\) converges a.s.

Let \(U_1,U_2,\ldots ,U_n\) be independent identically distributed random variables, uniformly distributed on \([0,1]\). Compute the covariance between \(U_{1,n}=\min _{1\leq i\leq n}U_i\) and \(U_{n,n}=\max _{1\leq i\leq n}U_i\).

Let \(X_1\) and \(X_2\) be two independent, identically distributed exponential (1) random variables. Compute the joint density of \(Y_1=X_1+X_2\) and \(Y_2=X_1/(X_1+X_2)\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed uniform on \([0,\theta ]\), \(\theta >0\), random variables. We wish to test \(H_0:\theta \leq \theta _0\) against \(H_A:\theta >\theta _0\). Find the uniformly most powerful test of size \(\alpha \) and compute the power function. (You must explain why your test has this property.)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\notin [0,\theta ],\\ \dfrac {1}{\theta }, & \text {if } t\in [0,\theta ]. \end {cases} \] We use \(\hat {\theta }=cX_{1,n}\) and \(\hat {\eta }=\alpha \bar {X}\) to estimate \(\theta \), where \[ X_{1,n}=\min _{1\leq i\leq n}X_i\quad \text {and}\quad \bar {X}=\frac {1}{n}\sum _{1\leq i\leq n}X_i. \] (a) Find \(c\) and \(\alpha \) such that the estimators are unbiased.

(b) Compute the relative efficiency.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed Poisson \((\theta )\) random variables. Find the uniformly minimum unbiased estimator for \(\eta =\theta ^2 e^{-\theta }\). (You must explain your answer.)

Let \(X_1,X_2,\ldots ,X_n\) and \(Y_1,Y_2,\ldots ,Y_m\) be independent, identically distributed random variables with a common continuous distribution function. Compute the expected value and the variance of \[ Z=\sum _{1\leq i\leq n}\sum _{1\leq j\leq n} I\{X_i\leq Y_j\}. \]

where \(I\{A\}\) denotes the indicator function of the event \(A\).

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with density function \[ f(t,\theta )= \begin {cases} 0, & \text {if } t\leq \theta ,\\ e^{-(t-\theta )}, & \text {if } t>\theta . \end {cases} \] (a) Find two \(1-\alpha \) equal-tail confidence intervals for \(\theta \). One should be based on the smallest order statistic and the other should be based on the sample mean.

(b) Compare the expected lengths of the confidence intervals.

Let \(\xi _n\) and \(\eta _m\) be two sequences of random variables. Prove or give counterexamples to the following statements:

(a) If \(\xi _n\overset {d}{\to }\xi \) and \(\eta _m\overset {d}{\to }\eta \), then \(\xi _n+\eta _m\overset {d}{\to }\xi +\eta \).

(b) \(\xi _n\overset {P}{\to }\xi \) and \(\eta _m\overset {P}{\to }\eta \), then \(\xi _n+\eta _m\overset {P}{\to }\xi +\eta \).

(c) \((\xi _n,\eta _n)\overset {P}{\to }(\xi ,\eta )\) then \(\xi _n+\eta _n\overset {P}{\to }\xi +\eta \).

(\(\overset {d}{\to }\) and \(\overset {P}{\to }\) denote convergence in distribution and convergence in probability.)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with \(P\{X_i=0\}=\theta \) and \(P\{X_i=1\}=1-\theta \) \((0<\theta <1)\). Find the numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n=a_n\hat {X}_n(1-\hat {X}_n)-b_n \] converges in distribution to a nondegenerate random variables, where \[ \hat {X}_n=\frac {1}{n}\sum _{1\leq i\leq n}X_i. \]

A sequence \(X_1,X_2,\ldots \) of random variables is said to be uniformly integrable if \[ \lim _{c\to \infty }\sup _{n\ge 1}\mathbb {E}\{|X_n|;\ |X_n|>c\}=0. \] Prove that \(X_n\to X\) in \(L^1(\mathbb {P})\) if and only if both of the following happen: (i) \(X_n\to X\) in probability; and (ii) \(\{X_n\}_{n=1}^{\infty }\) is uniformly integrable.

Construct three random variables \(X,Y\), and \(Z\), such that any distinct pair of them are independent but \((X,Y,Z)\) are not independent.

Suppose \(\{X_n\}_{n=1}^{\infty }\) are i.i.d. standard normal random variables. Prove that \[ M_n:=(n+1)^{-1/2}\exp \left (\frac {S_n^2}{2n+2}\right ) \] defines a martingale, where \(S_n=\sum _{i=1}^{n}X_i\).

Choose and fix an integer \(n\ge 1\). For all continuous functions \(f:[0,1]\to \mathbb {R}\) define \[ B_nf(p)=\sum _{k=0}^{n}\binom {n}{k}p^k(1-p)^{n-k}f(k/n), \qquad \text {for all }p\in [0,1]. \] [ \(B_nf\) is the Bernstein polynomial of \(f\).] Prove that if \(f\) is increasing, then so is \(B_nf\).

Let \(\{X_n\}_{n=1}^{\infty }\) be i.i.d. random variables with distribution, \(\mathbb {P}\{X_1=2\}=\mathbb {P}\{X_1=-2\}=1/2\). Define \(S_n=\sum _{i=1}^{n}X_i\) and \(T=\inf \{n\ge 1:S_n\in \{-2,6\}\}\). Prove that \(T<\infty \) a.s., and then compute \(\mathbb {P}\{S_T=-2\}\).

Suppose \(\{X_n\}_{n=1}^{\infty }\) and \(\{Y_n\}_{n=1}^{\infty }\) are random variables on the same probability space \((\Omega ,\mathcal {F},\mathbb {P})\). Suppose also that \(X_n\Rightarrow X\) and \(Y_n\Rightarrow Y\). True or false: \((X_n,Y_n)\Rightarrow (X,Y)\). [If true, then prove it. If false, then construct a counter-example.]

Let \(\{X_n\}_{n=1}^{\infty }\) be a sequence of i.i.d. random variables with \(\mathbb {E}[X_1]=0\) and \(\mathbb {E}[X_1^2]=1\). Define \(S_k=X_1+\cdots +X_k\) for all \(k\ge 1\). Let \(N_m\) be another independent random variable with a mean-\(m\) Poisson distribution. Prove that there exist non-random \(\alpha _m\) and \(\beta _m\) such that \((S_{N_m}-\alpha _m)/\beta _m\) converges weakly to a standard-normal distribution as \(m\to \infty \). Describe an explicit example of \(\alpha _m\) and \(\beta _m\).

If \(X\ge 0\) and \(X\in L^p(\mathbb {P})\) for all \(p>1\), then prove that \(\lim _{p\to \infty }\|X\|_p\) exists.

Consider two random variables \(X,Y\in L^2(\mathbb {P})\). Prove that if \(\mathbb {E}[X\mid Y]=Y\) and \(\mathbb {E}[X^2\mid Y]=Y^2\), then \(X=Y\) a.s.

Define \(X_0=1\). Then iteratively define \(X_n\) so that for all Borel sets \(A\subseteq \mathbb {R}\), \[ \mathbb {P}\left (X_n\in A\mid X_0,\ldots ,X_{n-1}\right ) = \frac {\text {The lebesgue measure of }A\cap [0,X_{n-1}]}{X_{n-1}}. \] Prove that \(\lim _{n\to \infty }2^nX_n=0\) a.s. You may use–without proof–the following version of the Borel–Cantelli lemma [due to Paul Lévy]: If \(\{\mathcal {F}_n\}_{n=1}^{\infty }\) is a filtration and \(A_n\)’s are events such that \(A_n\in \mathcal {F}_n\) and \(\sum _{n=1}^{\infty }\mathbb {P}(A_n\mid \mathcal {F}_{n-1})=\infty \) a.s., then \(\mathbb {P}(A_n\text { infinitely often})=1\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential (1) random variables with common density function \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ e^{-t}, & \text {if } 0\leq t<\infty . \end {cases} \] (a) Show that \(\left (\sum _{1\leq i\leq n}X_i\right )^{1/2}-n^{1/2}\) converges in distribution to a normal random variable.

(b) Compute the mean and the variance of the limiting normal random variable.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed uniform \((0,\theta )\) random variables with common density function \[ f(t)= \begin {cases} 0, & \text {if } t\notin [0,\theta ],\\ 1/\theta , & \text {if } t\in [0,\theta ]. \end {cases} \] (a) Find the maximum likelihood estimator for \(\theta \).

(b) Find a \(1-\alpha \) equal-tail confidence interval for \(\theta \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with common density function \(f\). Let \[ \hat {f}_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ), \] where \(K\) is a symmetric density function and \(K(t)=0\) if \(|t|>1\). We assume that \(h=h(n)\to 0\) as \(n\to \infty \). We assume that the second derivative of \(f\) exists and continuous in a neighborhood of \(t\). We use \(\hat {f}\) to estimate \(f\). Compute the bias of \(\hat {f}_n(t)\).

\(X\) and \(Y\) are two independent identically distributed normal \(N(0,\sigma ^2)\) random variables. Let \[ Z=X^2+Y^2 \] and \[ W=\frac {X}{(X^2+Y^2)^{1/2}}. \] (a) Show that \(Z\) and \(W\) are independent.

(b) Compute the marginal distributions of \(Z\) and \(W\).

\(Z_1\) and \(Z_2\) are two independent identically distributed exponential \((\lambda )\) random variables, i.e. the common density is \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ \dfrac {1}{\lambda }e^{-t/\lambda }, & \text {if } 0\leq t<\infty . \end {cases} \] Let \(X=Z_1\) and \(Y=Z_1+Z_1Z_2\). Compute

(a) \(E(Y\mid X=x)\)

(b) \(\operatorname {var}(E(Y\mid X))\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential \((\lambda )\) random variables. Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential \((\mu )\) random variables. The two samples are independent. We wish to test \(H_0:\theta =\mu \) against the alternative that \(H_0\) is not true.

(a) Find the likelihood ratio test.

(b) Find the asymptotic distribution of the likelihood ratio under \(H_0\).

(c) Find an equal-tail confidence interval for \(\tau =\lambda /\mu \).

Let \(X_1,X_2,X_3,\ldots \) be independent and exponential with mean 1. Use the Borel–Cantelli lemmas to show that \(\limsup _{n\to \infty }X_n/\log n=1\) a.s.

Let \(X\) be such that \(\mathbb {E}[X^+]=\infty \) and \(\mathbb {E}[X^-]<\infty \), and let \(X_1,X_2,X_3,\ldots \) be i.i.d. with common distribution that of \(X\). With \(S_n=X_1+X_2+\cdots +X_n\), show that \(\lim _{n\to \infty }S_n/n=\infty \) a.s.

Let \(X_1,X_2,X_3,\ldots \) be i.i.d. nonnegative random variables with mean 1 and variance 1, and put \(S_n=X_1+X_2+\cdots +X_n\). Show that \(2(\sqrt {S_n}-\sqrt {n})\) converges in distribution to a standard normal as \(n\to \infty \).

Let \(X_1,X_2,X_3,\ldots \) be a sequence of random variables, each of which has all moments finite, and suppose that \(\lim _{n\to \infty }\mathbb {E}[(X_n)^k]=k!\) for each \(k\ge 1\). Show that \(X_n\) converges in distribution.

Let \(X_1,X_2,X_3,\ldots \) be i.i.d. with finite mean and variance, and put \(S_n=X_1+X_2+\cdots +X_n\). Show that \(S_n-n\mathbb {E}[X_1]\) and \((S_n-n\mathbb {E}[X_1])^2-n\operatorname {Var}(X_1)\) are martingales.

Let \(\mathbb {P}\{X=1\}=\mathbb {P}\{X=-1\}=1/2\), let \(X_1,X_2,X_3,\ldots \) be i.i.d. with common distribution that of \(X\), and put \(S_n=X_1+X_2+\cdots +X_n\). Define \(T=\min \{n\ge 0:S_n=-A\text { or }S_n=B\}\), where \(A\) and \(B\) are positive integers. Using the martingales of problem 5, find \(\mathbb {P}\{S_T=B\}\) and \(\mathbb {E}[T]\).

Let \(P\) be the transition matrix for a finite irreducible aperiodic Markov chain, and assume that \(P\) is doubly stochastic (row and column sums are 1). Find the limit \(\lim _{n\to \infty }P_{ij}^n\) for all \(i\) and \(j\), and provide justification.

Consider the Markov chain in the set of nonnegative integers with transitions \(P(n,0)=1-P(n,n+1)=p_n>0\) for each \(n\ge 0\). Give necessary and sufficient conditions on the sequence \(\{p_n\}\) for the chain to be (a) positive recurrent, (b) null recurrent, and (c) transient.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed normal \(N(0,\sigma ^2)\) random variables. We wish to test \[ H_0:\sigma \leq \sigma _0 \] against \[ H_A:\sigma >\sigma _0. \]

1. Find the uniformly most powerful test of size \(\alpha \). (10 points)
2. Compute the rejection region as a function of \(\alpha \). (10 points)

Consider the following model: \[ y_i=f(x_i)+\epsilon _i,\qquad 1\leq i\leq n, \] where \[ f(t)=a_0+a_1t+\cdots +a_pt^p. \] The parameters \(a_0,a_1,\ldots ,a_p\) are unknown. The errors \(\epsilon _1,\epsilon _2,\ldots ,\epsilon _n\) are independent normal \(N(0,\sigma ^2)\) random variables. It is assumed that \(x_i\in [0,1]\) and known. We observe \((y_i,x_i)\), \(1\leq i\leq n\).

1. Estimate the parameters \(a_0,a_1,\ldots ,a_p\) and \(\sigma ^2\). (15 points)
2. Get the distribution of the estimator for \(\sigma ^2\). (5 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with distribution function \(F\). We assume that \(f=F'\) has a bounded second derivative. \(K(t)\) is a non-negative function satisfying \[ K(t)=K(-t)\quad \text {(symmetric),} \] \[ \int _{-\infty }^{\infty }K(t)\,dt=1 \quad \text {and}\quad K(t)=0\text { if }t\notin [-1,1]. \] The sequence \(h=h(n)\) is positive and goes to 0. We estimate \(f\) with \[ \hat {f}_n=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ). \]

1. Compute \(B_n\), the bias of \(\hat {f}_n\). (10 points)
2. Compute the limit of \(B_n/h^2(n)\) as \(n\to \infty \). (10 points)

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed normal \(N(\mu ,\sigma ^2)\) random variables. Let \[ U_n=\frac {2}{n(n-1)} \sum _{1\leq i<j\leq n}I\{|X_i-X_j|>1\}. \]

1. Compute \(EU_n\). (10 points)
2. Assume that \(\sigma \) is known. We wish to test \(H_0:\mu =\mu _0\) (\(\mu _0\) is given) against the alternative that \(H_0\) is not true. Can we use \(U_n\) to test \(H_0\) against this alternative? (5 points)
3. Now \(\mu \) and also \(\sigma \) are unknown. We wish to test the null-hypothesis that \(\sigma =\sigma _0\) against the alternative that \(H_0\) is not true. Can we use \(U_n\) now? (5 points)

Let \(X_1,X_2,X_3\) be i.i.d., each with an exponential distribution with mean 1. Find the joint distribution of \[ Y_1=\frac {X_1}{X_1+X_2},\qquad Y_2=\frac {X_1+X_2}{X_1+X_2+X_3},\qquad Y_3=X_1+X_2+X_3. \] (7 points) In particular, are they independent? (3 points)

Suppose \(\{E_1,E_2,\ldots \}\) is a sequence of measurable (otherwise arbitrary) events. Suppose \(\{F_1,F_2,\ldots \}\) is another sequence of measurable events also totally independent of all of the \(E_i\)’s. Assume the following:

(a) \(\mathbb {P}(E_k\text {, infinitely often})=1\);

(b) there exists some \(p>0\), such that for all \(k\ge 1\), \(\mathbb {P}(F_k)\ge p\).

Prove: \(\mathbb {P}(E_k\cap F_k\text {, infinitely often})\ge p>0\). (10 points)

Let \(X_1,X_2,\ldots \) be independent random variables with distribution functions \(F_1,F_2,F_3,\ldots \). Let \(Y=\sup _{1\le i<\infty }X_i\).

(a) Show that \(\mathbb {P}\{Y<\infty \}\) is 0 or 1 depending on whether \(\sum _{1\le i<\infty }(1-F_i(x))\) converges for some \(x\). (5 points)

(b) Show that if \(\mathbb {P}\{Y<\infty \}=1\), then \(\prod _{1\le i<\infty }F_i(x)\) converges for all \(x\) and it is the distribution function of \(Y\).

Let \(X_1,X_2,X_3\) be i.i.d., each with an exponential distribution with mean 1. Find the joint distribution of \[ Y_1=\frac {X_1}{X_1+X_2},\qquad Y_2=\frac {X_1+X_2}{X_1+X_2+X_3},\qquad Y_3=X_1+X_2+X_3. \] (7 points) In particular, are they independent? (3 points)

Suppose \(N\) is a positive random variable.

(a) Show that \[ \mathbb {P}(N>0)\ge \frac {(\mathbb {E} N)^2}{\mathbb {E} N^2}. \] (5 points)

(b) Let \(X_1,X_2,\ldots \) be i.i.d. random variables. Define \(S_n=X_1+\cdots +X_n\). Use part (a) to show that for any positive integer \(n\), \[ \mathbb {P}(S_j=0,\text { for some }1\le j\le n) \ge \frac {\sum _{j=1}^n\mathbb {P}(S_j=0)} {2\left (1+\sum _{j=1}^n\mathbb {P}(S_j=0)\right )}. \] (5 points)

Let \(X_1,X_2,\ldots ,X_n\) denote a random sample from a Poisson distribution with parameter \(\theta >0\).

(a) Show that \((-1)^{X_1}\) is unbiased for \(e^{-2\theta }\), and use the Lehmann–Scheffé theorem to deduce a UMVUE of \(e^{-2\theta }\) based on the sample. (Hint: The conditional distribution of \(X_1\) given \(X_1+\cdots +X_n\) has a simple form.) (5 points)

(b) Argue that the UMVUE in part (a) is a consistent estimator of \(e^{-2\theta }\). (5 points)

Let \(X_1,\ldots ,X_n\) be a random sample from the \(N(\mu ,\sigma ^2)\) distribution, with both \(\mu \) and \(\sigma ^2\) unknown.

(a) Derive the likelihood ratio test for \(H:\sigma ^2=\sigma _0^2\) against all alternatives. Here \(\sigma _0^2\) is a known positive constant. (5 points)

(b) Do the same for \(H:\sigma ^2\ge \sigma _0^2\). (5 points)

Prove Cantelli’s theorem: If \(X_1,X_2,\ldots \) are independent (but not necessarily identically distributed), mean zero, random variables with \(\sup _n E[X_n^4]<\infty \), then \(S_n/n\to 0\) a.s. as \(n\to \infty \).

Let \((\Omega ,\mathcal {F},P)\) be a probability space, let \(\mathcal {G}\) be a sub-\(\sigma \)-field of \(\mathcal {F}\), and let \(M\) be the closed subspace of \(L^2(\Omega ,\mathcal {F},P)\) consisting of the \(\mathcal {G}\)-measurable functions in \(L^2\). Show that \(T(X)=E[X\mid \mathcal {G}]\) coincides with the orthogonal projection of \(L^2(\Omega ,\mathcal {F},P)\) onto \(M\).

Consider the Markov chain in the state space \(S=\{\ldots ,-2,-1,0,1,2,\ldots \}\) with transitions \(P(i,i+2)=p\) and \(P(i,i-1)=1-p\), where \(0<p<1\). Determine for which \(p\) this chain is recurrent and for which \(p\) it is transient.

Let \(\{B(t),t\geq 0\}\) be a standard Brownian motion. Prove directly, using \[ V_n=\sum _{i=1}^{2^n}\left |B(i/2^n)-B((i-1)/2^n)\right |; \] that \(B(\cdot ,\omega )\) is of unbounded variation on \([0,1]\) for a.e. \(\omega \).

Prove Cantelli’s theorem: If \(X_1,X_2,\ldots \) are independent (but not necessarily identically distributed), mean zero, random variables with \(\sup _n \mathbb {E}[X_n^4]<\infty \), then \(S_n/n\to 0\) a.s. as \(n\to \infty \).

(a) Let \(X_1,X_2,\ldots \) be i.i.d. Poisson random variables with mean 1, and let \(Z\) be \(N(0,1)\). Denote \(a^-=-\min \{a,0\}\), and prove that \[ \text {(*)}\qquad \mathbb {E}\left [\left (\frac {S_n-n}{\sqrt n}\right )^-\right ]\to \mathbb {E}[Z^-]. \]

(b) Evaluate both expectations in (*) explicitly, and, noting the telescoping sum, deduce Stirling’s formula, for \(n!\).

Let \((\Omega ,\mathcal F,P)\) be a probability space, let \(\mathcal G\) be a sub-\(\sigma \)-field of \(\mathcal F\), and let \(M\) be the closed subspace of \(L^2(\Omega ,\mathcal F,P)\) consisting of the \(\mathcal G\)-measurable functions in \(L^2\). Show that \(T(X)=\mathbb {E}[X\mid \mathcal G]\) coincides with the orthogonal projection of \(L^2(\Omega ,\mathcal F,P)\) onto \(M\).

Consider the Markov chain in the state space \(S=\{\ldots ,-2,-1,0,1,2,\ldots \}\) with transitions \(P(i,i+2)=p\) and \(P(i,i-1)=1-p\), where \(0<p<1\). Determine for which \(p\) this chain is recurrent and for which \(p\) it is transient.

Let \(\{B(t),\, t\ge 0\}\) be a standard Brownian motion. Prove directly, using \[ V_n=\sum _{i=1}^{2^n}\left |B(i/2^n)-B((i-1)/2^n)\right |, \] that \(B(\cdot ,\omega )\) is of unbounded variation on \([0,1]\) for a.e. \(\omega \).

Suppose \(X_1,\ldots ,X_n\) is an i.i.d. sample from a normal population with \[ \mathbb {E} X_1=\operatorname {Var}(X_1)=\mu >0. \]

(a) Compute the maximum likelihood estimator \(\hat {\mu }\) of \(\mu \);

(b) Is \(\hat {\mu }\) consistent?

(c) Is \(\hat {\mu }\) asymptotically normal?

Suppose \(X_1,\ldots ,X_n\) is an i.i.d. sample from a normal distribution with mean \(\mu \) and variance 1.

(a) Find the UMVU estimator for \(\mu \). (Prove the optimality criterion.)

(b) Put a \(N(\theta ,\tau ^2)\) prior on \(\mu \) and find the minimax estimator of \(\mu \).

Let \(X_1,\ldots ,X_n\) be an i.i.d. sample which is uniformly chosen from the interval \((\theta _1,\theta _2)\).

(a) Prove that there are no one-dimensional (i.e., not vector-valued) sufficient statistics for \(h(\theta _1,\theta _2)\) where \(h\) is a one-to-one measurable function.

(b) Is there a one-dimensional (i.e., not vector-valued) sufficient statistics for \(\mu =(\theta _1+\theta _2)/2\)?

Let \(X_1,\ldots ,X_n\) be an i.i.d. sample from a uniform \((0,\theta )\) distribution, where \(\theta >0\).

(a) Find the maximum likelihood estimator \(\hat {\theta }\) for \(\theta \);

(b) Construct a \(100(1-\alpha )\%\) confidence interval for \(\theta \) based on \(\hat {\theta }\).

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \((\Omega ,\mathcal {A},P)\) be a probability space and \(\{B(t),0\leq t\leq 1\}\) be a Brownian motion on it. Since almost all sample paths of \(B\) are continuous, \(\int _0^t B(t)\,dt\) can be defined as a usual Riemann integral. Compute the distribution of \(\int _0^1 B(t)\,dt\).

Let \(\{S_n,\mathcal {F}_n,n\geq 1\}\) be a nonnegative martingale with \(ES_n=1\). Show that for all \(\lambda >0\), \[ P\{S_n\geq \lambda ,\text { for some }n\geq 1\}\leq \frac {1}{\lambda }. \]

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.

Suppose \(X\sim \operatorname {POISS}(\theta )\). Put a \(\operatorname {GAMMA}(\alpha ,\beta )\) prior on \(\theta \) and suppose we have the following loss function: \[ \ell (\theta ,a)=\frac {(a-\theta )^2}{\theta }. \] Find the Bayes’ estimator of \(\theta \).

Suppose \(\theta \in \{\theta _0,\theta _1\}\) is an unknown (\(\theta _0\) and \(\theta _1\) are, however, known.) Put some prior, \(\pi \), on \(\theta \). We are to test \(H_0:\theta =\theta _0\) versus \(H_1:\theta =\theta _1\). Our actions are 0 (accept) and 1 (reject). Find the Bayes’ procedure for doing this test, if the loss function, \(\ell (\theta ,a)\), is the 0–1 loss given by: \[ \ell (0,1)=0,\qquad \ell (0,0)=1, \] \[ \ell (1,0)=0,\qquad \ell (1,1)=1. \] Is this procedure minimax?

Construct a \((1-\alpha )\) two-sided confidence interval for the correlation coefficient of a bivariate normal distribution. (HINT. This is an exponential family.)

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \((\Omega ,\mathcal A,P)\) be a probability space and \(\{B(t),0\le t\le 1\}\) be a Brownian motion on it. Since almost all sample paths of \(B\) are continuous, \(\int _0^t B(t)dt\) can be defined as a usual Rieman integral. Compute the distribution of \(\int _0^1 B(t)dt\).

Let \(\{S_n,\mathcal F_n,\,n\ge 1\}\) be a nonnegative martingale with \(\mathbb {E} S_n=1\). Show that for all \(\lambda >0\), \[ \mathbb {P}\{S_n\ge \lambda ,\text { for some } n\ge 1\}\le \frac {1}{\lambda }. \]

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables. Show that the following statements are equivalent:

(a) \(\mathbb {E}|X_1|^\nu <\infty \);

(b) \(X_n/n^{1/\nu }\to 0\), almost surely;

(c) \(\max _{1\le i\le n}|X_i|/n^{1/\nu }\to 0\), almost surely.

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.

Let \(0\le X_n\le 1\) be adapted to \(\mathcal F_n\). Let \(\alpha ,\beta >0\) with \(\alpha +\beta =1\) and suppose \[ \mathbb {P}(X_{n+1}=\alpha +\beta X_n\mid \mathcal F_n)=X_n \] \[ \mathbb {P}(X_{n+1}=\beta X_n\mid \mathcal F_n)=1-X_n. \]

(a) Show that \(\mathbb {P}\{\lim _{n\to \infty }X_n=0\text { or }1\}=1\).

(b) Show that if \(X_0=\theta \), then \(\mathbb {P}\{\lim _{n\to \infty }X_n=1\}=\theta \).

Suppose \(X\sim \operatorname {Poisson}(\theta )\). Put a \(\operatorname {Gamma}(\alpha ,\beta )\) prior on \(\theta \) and suppose we have the following loss function: \(\ell (\theta ,a)=(a-\theta )^2/\theta \). Find the Bayes’ estimator of \(\theta \).

Suppose \(\theta \in \{\theta _0,\theta _1\}\) is an unknown (\(\theta _0\) and \(\theta _1\) are, however, known.) Put some prior, \(\pi \), on \(\theta \). We are to test \(H_0:\theta =\theta _0\) versus \(H_1:\theta =\theta _1\). Our actions are 0 (accept) and 1 (reject). Find the Bayes’ procedure for doing this test, if the loss function, \(\ell (\theta ,a)\), is the 0-1 loss given by: \[ \begin {alignedat}{2} \ell (0,1)&=0,\qquad & \ell (0,0)&=1,\\ \ell (1,0)&=0,\qquad & \ell (1,1)&=1. \end {alignedat} \] Is this procedure minimax?

Construct a \((1-\alpha )\) two-sided confidence interval for the correlation coefficient of a bivariate normal distribution. (HINT. This is an exponential family.)

Let \(X_0=1\) and define \(X_n\) inductively by declaring that \(X_{n+1}\) is uniformly distributed over \((0,X_n)\). Prove that \(n^{-1}\log X_n\to c\) a.s. and compute \(c\).

Let \(X_1,X_2,\ldots \) be independent Poisson random variables with \(EX_n=\lambda _n\) and let \(S_n=X_1+\cdots +X_n\). Show that if \(\sum \lambda _n=\infty \), then \(S_n/ES_n\to 1\) a.s.

Hint: Show that (*) \(Y_n/c_n\to 1\) a.s., provided that \(Y_n\geq 0\) is nondecreasing in \(n\), and (*) holds for a subsequence \(n(k)\) that has \(c_{n(k+1)}/c_{n(k)}\to 1\).

Suppose that \(X\) and \(Y\) are independent. Let \(f\) be a Borel function on \(\mathbb {R}^2\) with \(E[|f(X,Y)|]<\infty \) and let \(g(x)=E[f(x,Y)]\). Show that \(E[f(X,Y)\mid X]=g(X)\).

A thinker who owns \(r\) umbrellas travels back and forth between home and office, taking along an umbrella (if there is one at hand) in rain (probability \(p\)) but not otherwise (probability \(q=1-p\)). Let the state be the number of umbrellas at hand, irrespective of whether the thinker is at home or at work. Set up the transition matrix, and show that the Markov chain approaches equilibrium (i.e., the ergodic theorem is applicable). Find the steady-state probability of his getting wet, and show that five umbrellas will protect him at the \(5\%\) level against any climate (any \(p\)).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \(f\). We assume that \(f'\) is bounded. Let \(K\) be a function satisfying \(\int _{-\infty }^{\infty }K(u)\,du=1\), \(K'(u)\) is bounded and \(K(u)=0\), if \(|u|\geq a\) where \(a\) is a constant. The density \(f\) is estimated by \[ \hat f_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ). \]

Show that \(\hat f_n(t)\) is an almost surely uniformly consistent estimator for \(f\) on \([\alpha ,\beta ]\), \(-\infty <\alpha <\beta <\infty \).

Let \(X_0=1\) and define \(X_n\) inductively by declaring that \(X_{n+1}\) is uniformly distributed over \((0,X_n)\). Prove that \(n^{-1}\log X_n\to c\) a.s. and compute \(c\).

Let \(X_1,X_2,\ldots \) be independent Poisson random variables with \(\mathbb {E} X_n=\lambda _n\), and let \(S_n=X_1+\cdots +X_n\). Show that if \(\sum \lambda _n=\infty \), then \(S_n/\mathbb {E} S_n\to 1\) a.s.

Hint: Show that (*) \(Y_n/c_n\to 1\) a.s., provided that \(Y_n\ge 0\) is nondecreasing in \(n\), and (*) holds for a subsequence \(n(k)\) that has \(c_{n(k+1)}/c_{n(k)}\to 1\).

Let \(X_1,X_2,\ldots \) be i.i.d. mean 0, variance \(\sigma ^2\in (0,\infty )\).

(a) Use the central limit theorem as well as Kolmogorov’s 0-1 law to conclude that \(\limsup _{n\to \infty }S_n/\sqrt n=\infty \) a.s.

(b) Show that \(S_n/\sqrt n\) does not converge in probability.

Suppose that \(X\) and \(Y\) are independent. Let \(f\) be a Borel function on \(\mathbb R^2\) with \(\mathbb {E}[|f(X,Y)|]<\infty \) and let \(g(x)=\mathbb {E}[f(x,Y)]\). Show that \(\mathbb {E}[f(X,Y)\mid X]=g(X)\).

A thinker who owns \(r\) umbrellas travels back and forth between home and office, taking along an umbrella (if there is one at hand) in rain (probability \(p\)) but not otherwise (probability \(q=1-p\)). Let the state be the number of umbrellas at hand, irrespective of whether the thinker is at home or at work. Set up the transition matrix, and show that the Markov chain approaches equilibrium (i.e., the ergodic theorem is applicable). Find the steady-state probability of his getting wet, and show that five umbrellas will protect him at the \(5\%\) level against any climate (any \(p\)).

Let \(U_1,U_2,\ldots ,U_n\) be independent identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\le U_{2,n}\le \cdots \le U_{n,n}\) denote the order statistics. Let \(X_1,X_2,\ldots ,X_{n+1}\) be independent identically distributed exponential random variables with \(\mathbb {E} X_1=1\). Define \(S(i)=X_1+\cdots +X_i\).

(a) Prove that the random vectors \[ \{U_{1,n},\ldots ,U_{n,n}\} \quad \text {and}\quad \left \{\frac {S(1)}{S(n+1)},\ldots ,\frac {S(n)}{S(n+1)}\right \} \] have the same distribution.

(b) Compute the asymptotic distribution of \(n(U_{i+3,n}-U_{i,n})\), as \(n\to \infty \) when \(i\) is fixed.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with density function \(f\). We assume that \(f'\) is bounded. Let \(K\) be a function satisfying \(\int _{-\infty }^{\infty }K(u)du=1\), \(K'(u)\) is bounded and \(K(u)=0\), if \(|u|\ge a\) where \(a\) is a constant. The density \(f\) is estimated by \[ \hat f_n(t)=\frac {1}{nh}\sum _{1\le i\le n} K\left (\frac {t-X_i}{h}\right ). \] Show that \(\hat f_n(t)\) is an almost surely uniformly consistent estimator for \(f\) on \([\alpha ,\beta ]\), \(-\infty <\alpha <\beta <\infty \).