Probability Prelim Exam 1.1 January 2020

Let \(X_1,X_2,\ldots \), and \(X\) be real-valued random variables defined on a common probability space. Note they do NOT have to be independent or identically distributed. Prove that \[ \sum _{n=1}^{\infty }\mathbb {E}[|X_n-X|]<\infty \Longrightarrow X_n\to X \quad \text {almost surely.} \]

Let \(A_1,A_2,\ldots \), be a sequence of events (not necessarily independent) such that \[ \sum _{n=1}^{\infty }\mathbb {P}(A_n)=\infty \qquad \text {and}\qquad \mathbb {P}(A_n\cap A_m)\leq \mathbb {P}(A_n)\mathbb {P}(A_m) \quad \text {for } m\ne n. \] Prove that \(\mathbb {P}(A_n\text { i.o.})=1\) or provide a counterexample. Hint: Consider the mean and variance of \[ \sum _{i=1}^{n}\mathbf {1}_{A_i}. \]

Let \(X\) be a non-negative, integer-valued random variable.

a) Prove that \(\mathbb {E}[X]=\sum _{n=1}^{\infty }\mathbb {P}(X\geq n)\).

b) A dresser has \(k\) distinct pairs of socks (so \(2k\) socks total) and the socks are unmatched. We select, at random and without replacement, one sock at a time until a pair has been drawn. Compute the expectation of the total number of draws needed.

Let \(X\) and \(Y\) be any two random variables. Suppose \(\mathbb {E}[X^2]<\infty \). The conditional variance of \(X\) given \(Y\) is defined to be \[ \operatorname {Var}(X\mid Y)=\mathbb {E}\bigl [(X-\mathbb {E}[X\mid Y])^2\mid Y\bigr ]. \] Prove the conditional variance formula: \[ \operatorname {Var}(X)=\mathbb {E}[\operatorname {Var}(X\mid Y)]+\operatorname {Var}(\mathbb {E}[X\mid Y]). \]

Let \(\{X_n:n\in \mathbb {N}\}\) be a sequence of i.i.d. positive integrable random variables. Define \(T_n=X_1+\cdots +X_n\), \(n\in \mathbb {N}\), and \[ N(t)=\max \{n\in \mathbb {N}:T_n\leq t\}, \qquad t\geq 0. \] Show that with probability one, \[ \frac {N(t)}{t}\to \frac {1}{\mathbb {E}[X_1]} \qquad \text {as } t\to \infty . \] (Hint: Sketch what the process \(t\mapsto N(t)\) looks like for a fixed realization of the \(X_n\). What do you know about the asymptotic behavior of the jump times?)

Fix numbers \(p_x,q_x\in (0,1)\), \(x\in \mathbb {N}\), such that \(p_x+q_x\leq 1\). Let \(\{Z_{n,x}:n,x\in \mathbb {N}\}\) be independent random variables that take values in \(\{-1,0,1\}\), with common distribution \[ \mathbb {P}(Z_{n,x}=1)=p_x,\qquad \mathbb {P}(Z_{n,x}=-1)=q_x,\qquad \mathbb {P}(Z_{n,x}=0)=1-p_x-q_x. \] Let \(X_0=1\) and for \(n\in \mathbb {Z}_+\) define inductively \(X_{n+1}=X_n+Z_{n,X_n}\) if \(X_n\in \mathbb {N}\) and \(X_{n+1}=0\) if \(X_n=0\). In particular, \(X_n\in \mathbb {Z}_+\) almost surely. These \(X_n\) satisfy \[ \begin {aligned} \mathbb {P}(X_{n+1}=x+1\mid X_n=x)&=p_x,\\ \mathbb {P}(X_{n+1}=x-1\mid X_n=x)&=q_x,\\ \mathbb {P}(X_{n+1}=x\mid X_n=x)&=1-p_x-q_x, \end {aligned} \] if \(x\in \mathbb {N}\) and \(\mathbb {P}(X_{n+1}=x\mid X_n=x)=1\) if \(x=0\). The process \(X_n\) is called a birth and death process, absorbed at 0. Now define the function \(\phi :\mathbb {Z}_+\to \mathbb {R}\) by \(\phi (0)=0\), \(\phi (1)=1\), and for an integer \(x\geq 2\), \[ \phi (x)=1+\sum _{i=1}^{x-1}\prod _{j=1}^{i}\frac {q_j}{p_j}. \] Prove that \(M_n=\phi (X_n)\) is a martingale in the natural filtration \(\mathcal {F}_n=\sigma (X_0,\ldots ,X_n)\).

Assume the same setting as in the previous problem. You can assume that the claim in that problem is true and solve this problem independently.

a) For an integer \(a\geq 0\) let \(T_a=\inf \{n\geq 0:X_n=a\}\). Fix \(a\geq 2\) and prove that \[ \mathbb {P}(T_0<\infty \text { or }T_a<\infty )=1. \] (Hint: As long as the process is not at 0 it has a positive probability to go up \(a\) times in a row. How do you turn this observation into a proof?)

b) Calculate \(\mathbb {P}(T_a<T_0\mid X_0=1)\). (Carefully explain why the conditions of the theorems you use are satisfied.)

Use your limit theorems to prove that the following limit exists and to compute its value: \[ \lim _{n\to \infty }\int _{-\sqrt {n}}^{\sqrt {n}} \left (1-\frac {x^2}{2n}\right )^n\,dx. \] You need to fully justify the use of whatever limit theorem you choose.

Recall that the characteristic function of a random variable \(X\) is defined as \(\phi _X(t)=\mathbb {E}[e^{itX}]\) for \(t\in \mathbb {R}\). Prove that if \(X\in L^1\) then \(\phi '_X(0)=\mathbb {E}[X]\). Make sure to justify all of your steps.

Consider the symmetric random variable \(X\) taking values in \(\mathbb {Z}\setminus \{-1,0,1\}\) with distribution \[ \mathbb {P}(X=k)=\mathbb {P}(X=-k)=\frac {C}{k^2\log k}, \qquad k=2,3,\ldots \]

a) Prove that there is a finite \(C<\infty \) such that the above is a probability distribution.

b) Show that \(\mathbb {E}[|X|]=\infty \), hence \(\mathbb {E}[X]\) is not defined.

c) However, prove that \(\phi '_X(0)=0\), which shows that the converse direction of the previous problem is not always true. This requires showing that the limit defining \(\phi '_X(0)\) exists and equals zero. (Hint: Prove then use the inequality \(|\cos x-1|\leq \min (x^2/2,1)\).)