Probability Prelim Exam 1.3 August 2014

Let \(X_1,X_2,\ldots \) be independent random variables with 2 finite moments, and \[ \sum _{j=1}^{\infty }\operatorname {Var}(X_j)<\infty . \] Prove that \[ \lim _{n\to \infty }\sum _{j=1}^{n}(X_j-\mathbb {E} X_j) \] exists a.s. and in \(L^2(\mathbb {P})\).

Suppose \(\{X_n\}_{n=1}^{\infty }\) and \(\{Y_n\}_{n=1}^{\infty }\) are submartingales with respect to the same filtration \(\{\mathcal {F}_n\}_{n=1}^{\infty }\). Prove that \(Z_n:=\max (X_n,Y_n)\) is also a submartingale.

Suppose \(X=\operatorname {Poisson}(\lambda )\) for some \(\lambda >0\). Prove that \(\mathbb {P}\{X>e\lambda \}\leq e^{-\lambda }\).

Suppose \(\{X_j\}_{j=0}^{\infty }\) is a martingale such that \(X_0=0\) and \(|X_{i+1}-X_i|\leq 1\) for all \(i\geq 0\). Prove that \(\lim _{n\to \infty }n^{-\delta }X_n=0\), a.s. for every \(\delta >1/2\).

Let \(X_1,X_2,\ldots \) be independent, identically distributed random variables with the following common distribution: \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2k^2} \qquad \text {for } k=\pm 1,\pm 2,\ldots . \]

(a) Prove that \(\mathbb {E}\exp (itX_1)=1-(3|t|/\pi )+o(|t|)\) as \(t\to 0\). You may use, without proof, the fact that \[ \int _0^{\infty }(1-\cos \theta )\theta ^{-2}\,d\theta =\pi /2. \]

(b) Use the preceding in order to prove that \(n^{-1}(X_1+\cdots +X_n)\) converges weakly to a “Cauchy random variable \(Y\) with scale parameter \(3/\pi \).” That is, the characteristic function of \(Y\) is \[ \mathbb {E}\exp (itY)=\exp (-3|t|/\pi ) \qquad \text {for all }t\in \mathbb {R}. \] You may use, without proof, the fact that the latter is a characteristic function. This matter is the topic of Problem 8 below.

Let \(X_1,X_2,\ldots \) be i.i.d. integer-valued random variables with probability mass function \(f(a):=\Prob \{X_1=a\}\) for all \(a\in \mathbb {Z}\), and suppose \(f(a)>0\) for all \(a\in \mathbb {Z}\). Now let \(g\) be another probability mass function on \(\mathbb {Z}\) and define the likelihood ratio, \[ \Lambda _n:=\prod _{j=1}^{n}\frac {g(X_j)}{f(X_j)}. \]

(a) Consider the log-likelihood \(\log \Lambda _n\), where “log” denotes the natural logarithm. Prove that \[ \lim _{n\to \infty }n^{-1}\log \Lambda _n =\mathbb {D}(g\,\|\,f) \qquad \text {a.s.,} \] where \[ \mathbb {D}(g\,\|\,f):= \sum _{a=-\infty }^{\infty } f(a)\log \left [\frac {g(a)}{f(a)}\right ], \] provided that \[ \mathbb {E}(g\,\|\,f):= \sum _{a=-\infty }^{\infty } \left |\log \left [\frac {g(a)}{f(a)}\right ]\right |f(a)<\infty . \]

(b) Verify that \(\log x\leq x-1\) for all \(x>0\), and that the inequality is strict except when \(x=1\). Use this to prove the following [Gibbs’ inequality]: \(\mathbb {D}(g\,\|\,f)<0\).

(c) Conclude from the preceding that if \(\mathbb {E}(g\,\|\,f)<\infty \), then \(\Lambda _n\to 0\) a.s. very rapidly as \(n\to \infty \).

This fact plays an important role in statistics.

Suppose \(f:[0,\infty )\to \mathbb {R}\) is bounded and measurable, and \(f(x)=0\) for all \(x\geq 2\).

(a) Prove that if \(X_1,X_2,\ldots \) are independent, all distributed uniformly in the interval \((0,1)\), then \[ \frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+X_j}{n}\right ) \xrightarrow {\mathbb {P}} \int _0^{\infty } f(x)\,dx \qquad \text {as } n\to \infty . \]

(b) (Extra credit) Deduce from this the following version of Riemann sums for Lebesgue integrals [Chaterjee’s theorem]: There exists a sequence \(\{\delta _{j,n};\ 0\leq j\leq n,\ n\geq 1\}\) of real numbers in \((0,1)\) such that \[ \lim _{n\to \infty }\frac {1}{n}\sum _{j=0}^{n} f\left (\frac {j+\delta _{j,n}}{n}\right ) = \int _0^{\infty } f(x)\,dx. \]

Let \(f\) be a continuous element of \(L^1(\mathbb {R})\), and recall that its Fourier transform is \[ \hat {f}(t):=\int _{-\infty }^{\infty }\exp (itx)f(x)\,dx. \]

(a) Suppose also that \(\hat {f}\in L^1(\mathbb {R})\). Prove the following [inversion formula]: \[ f(x)=\frac {1}{2\pi }\int _{-\infty }^{\infty }e^{-ixt}\hat {f}(t)\,dt \qquad \text {for all }x\in \mathbb {R}. \] In other words, \(\widehat {\widehat {f}}(-x)=2\pi f(x)\) for all \(x\in \mathbb {R}\).

(b) Choose and fix some \(\theta >0\). Use the preceding to prove that the Fourier transform of the probability density function \[ f(x):=\frac {\theta }{\pi (\theta ^2+x^2)} \qquad [-\infty <x<\infty ] \] is \(\hat {f}(t)=\exp (-\theta |t|)\).

Suppose \(X\) and \(Y\) are i.i.d. random variables and \(f\) and \(g\) are nondecreasing, bounded and measurable functions on \(\mathbb {R}\).

(a) Prove that \[ \mathbb {E}[(f(X)-f(Y))(g(X)-g(Y))]\geq 0. \]

(b) Conclude that \(f(X)\) and \(g(X)\) are always positively correlated; i.e., \[ \mathbb {E}[f(X)g(X)]\geq \mathbb {E}[f(X)]\cdot \mathbb {E}[g(X)]. \]

Let \(Z=\operatorname {N}(0,1)\) denote a standard normal random variable. Suppose \(f:\mathbb {R}\to \mathbb {R}\) is continuously differentiable and there exists \(c>1\) such that \[ |f(x)|+|f'(x)|\leq c\exp (c|x|) \qquad \text {for all }x\in \mathbb {R}. \] Derive the following [Stein’s differential equation]: \[ \mathbb {E}[f'(Z)]=\mathbb {E}[Zf(Z)]. \]