Weak Convergence and Distributional Limits

Let \((a_{n})_{n \ge 1}\) be a nondecreasing sequence of positive numbers such that \[ \lim _{n \to \infty }\frac {a_{n}^{2}}{\sum _{k=1}^{n}a_{k}^{2}} = 0. \]

Let \((X_{n})_{n \ge 1}\) be a sequence of i.i.d. random variables with mean 0 and variance 1. Prove that \[ \frac {\sum _{k=1}^{n} a_{k}X_{k}}{\sqrt {\sum _{k=1}^{n}a_{k}^{2}}} \]

converges weakly to the standard normal distribution.

Without using the Central Limit Theorem or the Law of the Iterated Logarithm, prove the Weak Law of Large Numbers when \(X_{i} \in L^{1}\), i.e. show that when \(X_{1}, X_{2}, \ldots \) is a sequence of i.i.d. variables with \(\mathbb {E}[X_{1}] = \mu \) and \(X_{1} \in L^{1}\), then

Hint: first, prove the claim when \(X_{1} \in L^{2}\). Then for \(a > 0\), consider

Show that if \(X_{n} \to X\) in probability, then \(X_{n} \Rightarrow X,\) i.e. \(X_{n}\) converges weakly to \(X\).

Prove that if \(\{X_{i}\}_{i=1}^{\infty }\) are i.i.d. Uniform-\([0,1]\) random variables, then \[ \frac {4\sum _{i=1}^{n} iX_{i} - n^{2}}{n^{3/2}} \] converges weakly, and identify the limiting distribution. (Recall \(\sum _{i=1}^{n} i = \frac {n(n+1)}{2}\) and \(\sum _{i=1}^{n} i^{2} = \frac {n(n+1)(2n+1)}{6}\).)

Fix an integer \(n \ge 1\). Let \(\{Y_{k}\}_{k \in \mathbb {N}}\) be i.i.d. uniform on \(\{1, 2, \ldots , n\}\). Let

be the first time we get a repeated sample. Find an exponent \(\gamma \) and a nondegenerate distribution \(\mu \) such that, as \(n \to \infty \), \(n^{-\gamma }\sigma _{n}\) converges weakly to the distribution \(\mu \). (Hint: look at tail probabilities \(P(n^{-\gamma }\sigma _{n} \ge x)\).)

Give an example of a sequence of independent random variables \(\{Z_{k}\}_{k \in \mathbb {N}}\) satisfying \(\mathbb {E}[Z_{k}] = 0\), \(\operatorname {Var}(Z_{k}) = 1\), for each \(k\), and such that \((Z_{1} + \cdots + Z_{n})/\sqrt {n}\) does not converge in distribution to a standard normal. Justify any claims you make.

Let \(\{X_{n}\}_{n \in \mathbb {N}}\) be independent random variables such that for each \(n\), \(X_{n}\) takes the values \(+1\) and \(-1\) with probability \((1 - 2^{-n})/2\) each and the value \(2^{k}\) with probability \(2^{-k}\), for integers \(k > n\).

1. Show that \(\mathbb {E}[|X_{n}|] = \infty \) for all \(n \in \mathbb {N}\).
2. Show that \((X_{1} + \cdots + X_{n})/\sqrt {n}\) converges in distribution to a standard normal. (Hint: Check that we can write \(X_{n} = (1 - B_{n})Y_{n} + 2^{n}B_{n}Z_{n}\), where \(B_{n}\) is \(\operatorname {Bernoulli}(2^{-n})\), \(Y_{n}\) takes values \(\pm 1\) equally likely, \(Z_{n}\) takes the value \(2^{k}\) with probability \(2^{-k}\), for \(k \in \mathbb {N}\), and \(\{B_{n}, Z_{n}, Y_{n} : n \ge 1\}\) are mutually independent. Now, \(X_{n}\) is a small perturbation of \((1 - B_{n})Y_{n}\). Work out the details.)

Let \(\{X_{n}\}_{n \ge 1}\) be i.i.d. with mean 0 and variance 1. Let \(\{a_{n}\}_{n \ge 1}\) be a sequence of nonzero real numbers and \[ A_{n} = \left (\sum _{k=1}^{n} a_{k}^{2}\right )^{1/2}. \] Assume that \[ A_{n}^{-1}\left (\max _{1 \le k \le n} |a_{k}|\right ) \to 0 \qquad \text {as } n \to \infty . \] Find (and prove) a distributional limit for \[ A_{n}^{-1}\sum _{k=1}^{n} a_{k}X_{k} \qquad \text {as } n \to \infty . \]

Let \(U_1,U_2,\ldots ,U_n\) be independent and identically distributed random variables, uniform on \([0,1]\). Let \(U_{1,n}\leq U_{2,n}\leq \cdots \leq U_{n,n}\) be the order statistics. Show that \(nU_{3,n}\) converges in distribution and compute the limit.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables that have the uniform distribution on \([0,1]\). Let \(X_{1,n}\leq X_{2,n}\leq \cdots \leq X_{n,n}\) be the order statistics. Let \(k\geq 1\) be a fixed integer. Show that \[ Y_n=nX_{k,n} \] converges in distribution and compute the limiting distribution.

(Pólya’s Urn) An urn initially contains \(r\) red and \(b\) blue balls. A ball is chosen uniformly at random (i.e. with probability \(1/(r+b)\) each). If it comes up red (resp. blue), then it is returned and another red (resp. blue) ball is added to the urn. The process is repeated indefinitely. Let \(R_n\) be the number of red balls in the urn after \(n\) draws.

1. A vector \((X_1,X_2,\ldots ,X_n)\) of random variables is said to be exchangeable if \[ (X_1,\ldots ,X_n)\sim (X_{\pi (1)},\ldots ,X_{\pi (n)}) \qquad \text {for all permutations }\pi , \] where \(\sim \) denotes equality in distribution. If \(X_i\) is the indicator that a red ball is drawn from the urn at time \(i\), prove that \((X_1,\ldots ,X_n)\) is exchangeable.
2. Find the mean and variance of \(S_n=X_1+\cdots +X_n\), the total number of red balls added to the urn up to time \(n\).

Let \(X_n\) be a sequence of independent random variables with \(\mathbb {P}(X_n=1)=1/n=1-\mathbb {P}(X_n=0)\). Prove that \[ \left (\sum _{i=1}^{n}X_i-\log n\right )/\sqrt {\log n} \] converges weakly to a standard normal.

Fix \(\alpha \in (0,1)\). Let \(X_n\) be a sequence of independent random variables with \(\mathbb {P}(X_n=n)=\mathbb {P}(X_n=-n)=n^{-\alpha }/2\) and \(\mathbb {P}(X_n=0)=1-n^{-\alpha }\). Prove that \((X_1+\cdots +X_n)/n^{(3-\alpha )/2}\) converges weakly to a centered normal.

Let \(X_1,X_2,\ldots \) be iid random variables with common pdf \[ f(x)= \begin {cases} 0, & |x|\leq 2,\\ \dfrac {c}{x^2\log |x|}, & |x|>2. \end {cases} \]

1. Prove that there is a finite constant \(c\) which makes \(f\) a pdf.
2. Let \(S_n=X_1+X_2+\cdots +X_n\). Prove that \(S_n/n\to 0\) in probability as \(n\to \infty \). Hint: Prove that \(S_n/n\) converges to 0 in distribution, then explain why you can upgrade to convergence in probability for free.
3. Does \(S_n/n\to 0\) almost surely as \(n\to \infty \)?

Let \(\{X_i:i\geq 1\}\) be i.i.d. random variables distributed uniformly on \([0,1]\). Prove that the distribution of \[ \frac {4\sum _{i=1}^{n} iX_i-n^2}{n^{3/2}} \] converges weakly and identify the limiting distribution.

Let \((X_n)\) be i.i.d. random variables with mean \(\mu \).

1. Prove that \[ \lim _{n\to \infty } \frac {1}{\log n}\sum _{i=1}^{n}\frac {X_i}{i} = \mu \qquad \text {almost surely.} \]
2. If the random variables also have a finite variance \(\sigma ^2<\infty \), then find \(a_n\) such that \[ a_n\left (\sum _{i=1}^{n}\frac {X_i}{i}-\mu \log n\right ) \] has a weak limit. Identify this weak limit. (You need to prove the weak convergence, once you have identified \(a_n\) and the limiting distribution.)

Prove or disprove the following statement. If \(X_n\) converges weakly to \(X\) as \(n\to \infty \) and \(Z_n-X_n\) converges weakly to 0 as \(n\to \infty \), then \(Z_n\) converges weakly to \(X\) as \(n\to \infty \).

Let \((X_n)\) be i.i.d. uniform random variables on \([0,1]\). For each \(n\geq 1\), define \[ Y_n=n\min \{X_k:1\leq k\leq n\}. \] Prove that as \(n\to \infty \), \(Y_n\) converges weakly to an exponential random variable with mean one.

Give an example of a sequence of random variables \(X_n\) with densities \(f_n(x)\) so that \(X_n\) converges weakly as \(n\to \infty \) to a uniform random variable on \([0,1]\), but \(f_n(x)\) does not converge to 1 for any \(x\in [0,1]\).

Let \(X_1,X_2,\ldots \) be a collection of i.i.d. random variables, and define \(S_n:=X_1+\cdots +X_n\) for all \(n\geq 1\). Suppose \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2 k^2} \qquad \text {for all } k=\pm 1,\pm 2,\ldots . \] Prove that \(S_n/n\) converges weakly to a non-constant random variable as \(n\to \infty \).

If \(X_1,X_2,\ldots \) are independent standard normal random variables, then find non-random sequences \(a_n,b_n\to \infty \) such that \(a_n\min _{1\leq i\leq n}X_i+b_n\) converges weakly. Identify the limiting distribution.

Let \(X_1,X_2,\ldots ,X_n\) be iid with common pdf \(f(x)=2x\), \(0<x<1\) and zero elsewhere. Find the limiting distribution of \(n(1-X_{(n)})\), where \(X_{(n)}=\max \{X_1,\ldots ,X_n\}\).

Let \(X_1,X_2,\ldots ,X_n\) be iid exponential(1) random variables and \[ \bar X_n=\frac {1}{n}\sum _{i=1}^n X_i \] be the sample mean. Find two sequences of numbers \(a_n\) and \(b_n\) such that \[ \frac {\log \bar X_n-a_n}{b_n} \] converges in distribution to a standard normal random variable.

Let \(X_1,X_2,\ldots ,X_n\) be iid from a distribution with CDF \(F(x)=1-1/x\) for \(1\leq x<\infty \), and zero otherwise.

(a) Derive the CDF of the smallest order statistic \(X_{(1)}\).

(b) Find the limiting distribution of \(X_{(1)}\).

(c) Find the limiting distribution of \(X_{(1)}^n\).

(a) Find the distribution of \(\int _0^1 B_t\,dt\), where \((B_t)\) is Brownian motion.

(b) Given an i.i.d. sequence \(X_1,X_2,\ldots \) with mean 0 and variance 1, what does Donsker’s invariance principle tell us about this sequence in connection with the distribution of \(\int _0^1 B_t\,dt\)?

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables uniform on \([0,1]\) and let \(\alpha >0\). Show that there are numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n=\frac {\sum _{k=1}^{n}k^{\alpha }X_k-a_n}{b_n} \] converges in distribution to a standard normal random variable.

Let \(X_1,X_2,\ldots \) be independent and identically distributed random variables uniform on \([0,1]\). Show that for all \(\alpha >-1\) there are numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n= \frac {\sum _{1\leq k\leq n} k^{\alpha }X_k-a_n}{b_n} \] converges in distribution to a standard normal random variable.

Let \(X_1,X_2,\ldots \) be independent, identically distributed random variables with the following common distribution: \[ \mathbb {P}\{X_1=k\}=\frac {3}{\pi ^2k^2} \qquad \text {for } k=\pm 1,\pm 2,\ldots . \]

(a) Prove that \(\mathbb {E}\exp (itX_1)=1-(3|t|/\pi )+o(|t|)\) as \(t\to 0\). You may use, without proof, the fact that \[ \int _0^{\infty }(1-\cos \theta )\theta ^{-2}\,d\theta =\pi /2. \]

(b) Use the preceding in order to prove that \(n^{-1}(X_1+\cdots +X_n)\) converges weakly to a “Cauchy random variable \(Y\) with scale parameter \(3/\pi \).” That is, the characteristic function of \(Y\) is \[ \mathbb {E}\exp (itY)=\exp (-3|t|/\pi ) \qquad \text {for all }t\in \mathbb {R}. \] You may use, without proof, the fact that the latter is a characteristic function. This matter is the topic of Problem 8 below.

Let \(\mathbf {X}_n:=(X_{1,n},\ldots ,X_{d,n})\) be a random variable in \(\mathbf {R}^d\) for every \(n\geq 1\). Let \(\mathbf {Y}:=(Y_1,\ldots ,Y_d)\) be another random variable in \(\mathbf {R}^d\). Prove that \(\mathbf {X}_n\Rightarrow \mathbf {Y}\) if and only if the 1-dimensional random variable \(\sum _{i=1}^{d}\lambda _iX_{i,n}\) converges weakly to \(\sum _{i=1}^{d}\lambda _iY_i\) for every \(\lambda :=(\lambda _1,\ldots ,\lambda _d)\in \mathbf {R}^d\).

Let \(\{W(t)\}_{t\geq 0}\) be a standard Brownian motion, and define \[ U(t):=e^{-t/2}W(e^t)\qquad \text {for all }t\geq 0. \]

(a) Prove that \(\{U(t)\}_{t\geq 0}\) is Gaussian process; compute its mean and covariance functions.

(b) Prove that \[ \limsup _{t\to \infty }\frac {U(t)}{\sqrt {2\ln t}} = -\liminf _{t\to \infty }\frac {U(t)}{\sqrt {2\ln t}} =1 \qquad \text {a.s.} \]

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with distribution function \[ F(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ t^3, & \text {if } 0\leq t\leq 1,\\ 1, & \text {if } t>0. \end {cases} \] Show that \[ Y_n=n^{1/3}X_{1,n} \] converges in distribution, where \(X_{1,n}=\min \{X_1,X_2,\ldots ,X_n\}\).

If \(X\) is either (a) Poisson\((\lambda )\) or (b) gamma\((\lambda ,1)\) (density proportional to \(x^{\lambda -1}e^{-x}\), \(x>0\)), show that \((X-\mathbb {E}[X])/\sqrt {\operatorname {Var}(X)}\) converges in distribution to \(N(0,1)\) as \(\lambda \to \infty \) (\(\lambda \) need not be an integer).

Let \(X_1,X_2,\ldots \) be i.i.d. uniform\((0,2)\), and put \(M_n:=X_1X_2\cdots X_n\) for each \(n\geq 1\). Show that \(\{M_n\}_{n=1}^{\infty }\) is a martingale. What does the martingale convergence theorem tell us about \(\lim _{n\to \infty }M_n\)? (In particular, evaluate this limit.)

Choose and fix a sequence of real numbers \(a_1,a_2,\ldots \) such that: (i) \(0<a_k<1\) for all \(k\geq 1\); and (ii) \(\sum _{k=1}^{\infty }a_k^2=\infty \). Let \(X_1,X_2,\ldots \) be independent random variables, where each \(X_k\) is distributed uniformly on \((-a_k,a_k)\). Prove that \(n^{-1/2}(X_1+\cdots +X_n)\) converges weakly to \(N(0,\sigma ^2)\) and compute \(\sigma \).

Let \(X_1,X_2,\ldots \) and \(Y_1,Y_2,\ldots \) denote random variables on a common probability space. Suppose \(X_n\) converges to 0 in probability as \(n\to \infty \), and \(Y_n\) converges to \(Y\) weakly. Prove that \((X_n,Y_n)\) converges weakly to \((0,Y)\).

(5 points) Let \(\zeta _n\) and \(\eta _n\) be two sequences of random variables. Prove or give counterexamples to the following statements:

9a) If \(\zeta _n\xrightarrow {(d)}\zeta \) and \(\eta _n\xrightarrow {(d)}\eta \), then \(\zeta _n+\eta _n\xrightarrow {(d)}\zeta +\eta \).

9b) If \(\zeta _n\xrightarrow {P}\zeta \) and \(\eta _n\xrightarrow {P}\eta \), then \(\zeta _n+\eta _n\xrightarrow {P}\zeta +\eta \).

9c) If \(\zeta _n\xrightarrow {(d)}\zeta \) and \(\eta _n\xrightarrow {(d)}\eta \), then \(\zeta _n+\eta _n\xrightarrow {P}\zeta +\eta \).

(\(\xrightarrow {(d)}\) and \(\xrightarrow {P}\) denote convergence in distribution and in probability, respectively.)

Let \(X_1,X_2,\ldots ,X_n\) and \(Y_1,Y_2,\ldots ,Y_m\) be two independent random samples. We assume that \(X_1,X_2,\ldots ,X_n\) are independent identically distributed exponential \((\lambda )\) random variables and \(Y_1,Y_2,\ldots ,Y_m\) are independent identically distributed exponential \((\mu )\) random variables. We want to test \(H_0:\lambda =\mu \) against the alternative \(H_A:\lambda \ne \mu \).

(i) Find \(T_{m,n}\) the likelihood ratio test. (7 points)

(ii) Prove that \(-2\log T_{m,n}\) converges in distribution under the null hypothesis when \(\min (m,n)\to \infty \). (7 points)

(iii) Show that the likelihood ratio test is asymptotically consistent and prove that the power goes to 1 as \(\min (m,n)\to \infty \). (7 points)

Let \(\{X_i\}_{i=1}^{\infty }\) be i.i.d. with the “standard Cauchy distribution.” That is, the density function of \(X_1\) is \[ f(x)=\frac {1}{\pi (1+x^2)} \qquad -\infty <x<\infty . \] Prove that \(\max _{1\le i\le n}X_i/n\) converges weakly. Identify the limit.

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with distribution function \[ F(t)=\frac {1}{1+e^{-t}},\qquad -\infty <t<\infty . \] Show that \[ Y_n=X_{n,n}-\log n \] converges in distribution, where \(X_{n,n}=\max \{X_1,X_2,\ldots ,X_n\}\).

Let \(\xi _n\) and \(\eta _m\) be two sequences of random variables. Prove or give counterexamples to the following statements:

(a) If \(\xi _n\overset {d}{\to }\xi \) and \(\eta _m\overset {d}{\to }\eta \), then \(\xi _n+\eta _m\overset {d}{\to }\xi +\eta \).

(b) \(\xi _n\overset {P}{\to }\xi \) and \(\eta _m\overset {P}{\to }\eta \), then \(\xi _n+\eta _m\overset {P}{\to }\xi +\eta \).

(c) \((\xi _n,\eta _n)\overset {P}{\to }(\xi ,\eta )\) then \(\xi _n+\eta _n\overset {P}{\to }\xi +\eta \).

(\(\overset {d}{\to }\) and \(\overset {P}{\to }\) denote convergence in distribution and convergence in probability.)

Let \(X_1,X_2,\ldots ,X_n\) be independent, identically distributed random variables with \(P\{X_i=0\}=\theta \) and \(P\{X_i=1\}=1-\theta \) \((0<\theta <1)\). Find the numerical sequences \(a_n\) and \(b_n\) such that \[ Y_n=a_n\hat {X}_n(1-\hat {X}_n)-b_n \] converges in distribution to a nondegenerate random variables, where \[ \hat {X}_n=\frac {1}{n}\sum _{1\leq i\leq n}X_i. \]

Let \(\{X_n\}_{n=1}^{\infty }\) be a sequence of i.i.d. random variables with \(\mathbb {E}[X_1]=0\) and \(\mathbb {E}[X_1^2]=1\). Define \(S_k=X_1+\cdots +X_k\) for all \(k\ge 1\). Let \(N_m\) be another independent random variable with a mean-\(m\) Poisson distribution. Prove that there exist non-random \(\alpha _m\) and \(\beta _m\) such that \((S_{N_m}-\alpha _m)/\beta _m\) converges weakly to a standard-normal distribution as \(m\to \infty \). Describe an explicit example of \(\alpha _m\) and \(\beta _m\).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed exponential (1) random variables with common density function \[ f(t)= \begin {cases} 0, & \text {if } -\infty <t<0,\\ e^{-t}, & \text {if } 0\leq t<\infty . \end {cases} \] (a) Show that \(\left (\sum _{1\leq i\leq n}X_i\right )^{1/2}-n^{1/2}\) converges in distribution to a normal random variable.

(b) Compute the mean and the variance of the limiting normal random variable.

Let \(X_1,X_2,X_3,\ldots \) be i.i.d. nonnegative random variables with mean 1 and variance 1, and put \(S_n=X_1+X_2+\cdots +X_n\). Show that \(2(\sqrt {S_n}-\sqrt {n})\) converges in distribution to a standard normal as \(n\to \infty \).

Let \(X_1,X_2,X_3,\ldots \) be a sequence of random variables, each of which has all moments finite, and suppose that \(\lim _{n\to \infty }\mathbb {E}[(X_n)^k]=k!\) for each \(k\ge 1\). Show that \(X_n\) converges in distribution.

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with a continuous distribution function \(F\). The ordered observations are \(X_{1,n}\leq X_{2,n}\leq \cdots \leq X_{n,n}\). Let \[ T_n=\sum _{i=1}^{n-1}\left (F(X_{i+1,n})-F(X_{i,n})\right )^2. \] Show that there are numerical sequences \(a_n\) and \(b_n\) such that \((T_n-a_n)/b_n\) converges in distribution to a standard normal random variable. (20 points)

Let \(\{B(t),t\geq 0\}\) be a standard Brownian motion. Prove directly, using \[ V_n=\sum _{i=1}^{2^n}\left |B(i/2^n)-B((i-1)/2^n)\right |; \] that \(B(\cdot ,\omega )\) is of unbounded variation on \([0,1]\) for a.e. \(\omega \).

Let \(\{B(t),\, t\ge 0\}\) be a standard Brownian motion. Prove directly, using \[ V_n=\sum _{i=1}^{2^n}\left |B(i/2^n)-B((i-1)/2^n)\right |, \] that \(B(\cdot ,\omega )\) is of unbounded variation on \([0,1]\) for a.e. \(\omega \).

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \((\Omega ,\mathcal {A},P)\) be a probability space and \(\{B(t),0\leq t\leq 1\}\) be a Brownian motion on it. Since almost all sample paths of \(B\) are continuous, \(\int _0^t B(t)\,dt\) can be defined as a usual Riemann integral. Compute the distribution of \(\int _0^1 B(t)\,dt\).

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.

Let \(X_1,X_2,\cdots \) be independent, identically distributed random variables, uniformly distributed on \([0,1]\). Show that \[ Y_n=\frac {4\sum _{1\le k\le n}kX_k-n^2}{n^{3/2}}, \] converges in distribution to a normal random variable.

Let \((\Omega ,\mathcal A,P)\) be a probability space and \(\{B(t),0\le t\le 1\}\) be a Brownian motion on it. Since almost all sample paths of \(B\) are continuous, \(\int _0^t B(t)dt\) can be defined as a usual Rieman integral. Compute the distribution of \(\int _0^1 B(t)dt\).

Let \(X_1,X_2,\cdots \) be independent, identically distributed normal random variables. Find two numerical sequences, \(a_n\) and \(b_n\), such that \[ \frac {\max _{1\le i\le n}X_i-a_n}{b_n} \] converges in distribution to a non-degenerate random variable.