Probability Prelim Exam 1.1 January 2017

Consider an array \(\{X_{i,j}\}\) of random variables such that, for every \(n\geq 1\), the collection \(X_{1,n},\ldots ,X_{n,n}\) is i.i.d. with common distribution \[ \mathbb {P}\{X_{1,n}=1\} = \mathbb {P}\{X_{1,n}=-1\} = \frac {1}{2}\left (1-\frac {1}{n^2}\right ); \] and \(\mathbb {P}\{X_{1,n}=n^2\}=n^{-2}\). Let \(S_n:=X_{1,n}+\cdots +X_{n,n}\). Prove that \(S_n/\sqrt {n}\Rightarrow \mathrm {N}(0,\sigma ^2)\) for some \(\sigma \in (0,\infty )\), although \(\mathbb {E}(S_n/\sqrt {n})\to \infty \) as \(n\to \infty \).

Consider a one-dimensional simple symmetric random walk: \(S_n=x+\sum _{k=1}^{n}X_k\), where \(x\) is where the walk starts and \(\{X_n:n\geq 1\}\) are i.i.d. with \[ \mathbb {P}(X_n=1)=\mathbb {P}(X_n=-1)=1/2. \]

1. Compute the probability that starting at \(x=0\) the random walk will hit \(-1\) before it hits \(b\in \mathbb {N}\). Deduce from this that the random walk is recurrent, i.e. starting at 0 the random walk will return to 0 with probability 1.
2. Compute the mean time it takes the random walk to go from \(x=0\) to \(-1\). Deduce from this that the random walk is null recurrent, i.e. starting at 0, the mean time it takes to return to 0 is infinite.

(Hint: Prove first that when \(x=0\) both \(S_n\) and \(S_n^2-n\) are martingales.)

Let \(X_1,X_2,\ldots \) be a collection of i.i.d. random variables, and define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\). Suppose \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2 k^2} \qquad \text {for all } k=\pm 1,\pm 2,\ldots . \] Prove that \(S_n/n\) converges weakly to a non-constant random variable as \(n\to \infty \).

Suppose \(X,Y\in L^1(\mathbb {P})\) satisfy \(\mathbb {E}[X\mid Y]=Y\) and \(\mathbb {E}[Y\mid X]=X\) almost surely. Prove that \(X=Y\) almost surely. (Hint: Prove first that \(\mathbb {E}[X-Y;X<q<Y]=0\) for all \(q\).)

Let \(p_n\) and \(q_n\) denote two probability mass functions on \(\mathbb {Z}^n\), for every integer \(n\geq 1\). That is, for all \(n\geq 1\):

1. \(p_n(\mathbf {x})\geq 0\) and \(q_n(\mathbf {x})\geq 0\) for all \(\mathbf {x}\in \mathbb {Z}^n\); and
2. \(\sum _{\mathbf {x}\in \mathbb {Z}^n}p_n(\mathbf {x}) =\sum _{\mathbf {x}\in \mathbb {Z}^n}q_n(\mathbf {x})=1\).

Now suppose \(\{X_i\}_{i=1}^{\infty }\) is a collection of random variables such that \((X_1,\ldots ,X_n)\) has mass function \(p_n\) for every \(n\geq 1\). That is, \[ \mathbb {P}\{X_1=x_1,\ldots ,X_n=x_n\} = p_n(x_1,\ldots ,x_n) \qquad \text {for all }(x_1,\ldots ,x_n)\in \mathbb {Z}^n. \]

1. Prove that \(p_n(X_1,\ldots ,X_n)>0\) a.s. for all \(n\geq 1\).
2. Prove that \(\{Z_n\}_{n=1}^{\infty }\) is a martingale, where, for all \(n\geq 1\), \[ Z_n:= \frac {q_n(X_1,\ldots ,X_n)}{p_n(X_1,\ldots ,X_n)}, \] almost surely on the \(\mathbb {P}\)-measure-one event \(\{p_n(X_1,\ldots ,X_n)>0\}\), and \(Z_n:=0\) otherwise.
3. Prove that \(\lim _{n\to \infty }Z_n\) exists a.s. and is finite a.s. (This is a starting point for likelihood-ratio testing in classical statistics.)

If \(X_1,X_2,\ldots \) are independent standard normal random variables, then find non-random sequences \(a_n,b_n\to \infty \) such that \(a_n\min _{1\leq i\leq n}X_i+b_n\) converges weakly. Identify the limiting distribution.

Suppose \(X_1,X_2,\ldots \) are independent mean-zero random variables that satisfy the condition \[ \sum _{i=1}^{\infty }\operatorname {Var}(X_i)<\infty . \]

1. Prove that \(\sum _{n=1}^{\infty }X_n\) exists a.s. and is finite a.s.
2. Use part (a) to prove that if \(\varepsilon _1,\varepsilon _2,\ldots \) are i.i.d. with \(\mathbb {P}\{\varepsilon _1=\pm 1\}=1/2\), then the random harmonic series \(\sum _{i=1}^{\infty }(\varepsilon _i/i)\) converges a.s. and is finite a.s.
3. Is \(\sum _{i=1}^{\infty }(\varepsilon _i/i)\) in \(L^1(\mathbb {P})\)?

Suppose that \(X_1,X_2,\ldots \) are i.i.d. exponential random variables with parameter \(\lambda >0\). Prove that \[ \overline {\lim }_{n\to \infty }\left (\frac {X_n}{\log n}\right )=\lambda ^{-1} \qquad \text {and}\qquad \underline {\lim }_{n\to \infty }\left (\frac {\log X_n}{\log n}\right )=-1. \]

Let \(X_1,X_2,\ldots \) be i.i.d. with \(\mathbb {P}\{X_1=1\}=p\) and \(\mathbb {P}\{X_1=-1\}=q\), where \(p\notin \{0,\frac {1}{2},1\}\) and \(q=1-p\). Define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\) and consider the stopping time, \(T:=\inf \{n\geq 1:|S_n|=5\}\), where \(\inf \emptyset :=\infty \). Prove that \(T\in L^1(\mathbb {P})\) and compute \(\mathbb {E}(T)\). (HINT: Begin by proving that \(M_n:=(q/p)^{S_n}\) and \(N_n:=S_n-(p-q)n\) are martingales.)

Let \(X\) and \(Y\) be two real-valued random variables. Let \(f\) and \(g\) be two non-decreasing measurable functions such that \(f(X)\) and \(g(Y)\) are integrable. Prove that \[ \mathbb {E}[f(X)g(Y)]\geq \mathbb {E}[f(X)]\mathbb {E}[g(Y)]. \]