Conditional Expectation and Conditional Independence

Let \(X_{n}, Y_{n}\) be positive, in \(L^{1}\), and measurable with respect to the filtration \(\mathcal {F}_{n}\). Suppose

with \(\sum Y_{n} < \infty \) a.s.

1. Show that \(\prod _{i=1}^{n}(1 + Y_{i})\) converges a.s. to a finite limit.

2. Show

   \[ M_{n} = \frac {X_{n}}{\prod _{i=0}^{n-1}(1 + Y_{i})} \]
   is a super-martingale.
3. Use (a) and (b) to prove that \(X_{n}\) converges a.s. to a finite limit.

Let \(X\), \(Y\), \(Z\) be random variables on some probability space. Recall that

is the conditional expectation of the indicator function of the event \(\{X \in A\}\), given the \(\sigma \)-algebra \(\sigma \{Y, Z\}\) generated by \(Y\) and \(Z\). We say that \(X\) and \(Y\) are independent, given \(Z\), if for all Borel sets \(A, B \subset \mathbb {R}\),

Show that this condition is equivalent to having \(P(X \in A \mid Y, Z) = P(X \in A \mid Z)\) \(P\)-almost surely, for all Borel sets \(A\).

Let \(\{\mathcal {F}_{n}\}_{n \ge 0}\) be a filtration on the probability space \((\Omega , \mathcal {F}, P)\), let \(X \in L^{1}(\Omega , \mathcal {F}, P)\), and define \(M_{n} = \mathbb {E}[X \mid \mathcal {F}_{n}]\). Let \(\tau \) be a stopping time and recall the definition \[ \mathcal {F}_{\tau } = \{A \in \mathcal {F} : A \cap \{\tau = n\} \in \mathcal {F}_{n}\ \forall n \ge 0\}. \]

1. Show that \(1_{\{\tau < \infty \}}M_{\tau }\) is integrable.
2. Show that \(\mathbb {E}[X \mid \mathcal {F}_{\tau }] = 1_{\{\tau < \infty \}}M_{\tau } + 1_{\{\tau = \infty \}}X\). Do not forget the necessary measurability condition.

Let \(X\) be a non-negative, integer-valued random variable.

a) Prove that \(\mathbb {E}[X]=\sum _{n=1}^{\infty }\mathbb {P}(X\geq n)\).

b) A dresser has \(k\) distinct pairs of socks (so \(2k\) socks total) and the socks are unmatched. We select, at random and without replacement, one sock at a time until a pair has been drawn. Compute the expectation of the total number of draws needed.

Let \(X\) and \(Y\) be any two random variables. Suppose \(\mathbb {E}[X^2]<\infty \). The conditional variance of \(X\) given \(Y\) is defined to be \[ \operatorname {Var}(X\mid Y)=\mathbb {E}\bigl [(X-\mathbb {E}[X\mid Y])^2\mid Y\bigr ]. \] Prove the conditional variance formula: \[ \operatorname {Var}(X)=\mathbb {E}[\operatorname {Var}(X\mid Y)]+\operatorname {Var}(\mathbb {E}[X\mid Y]). \]

Choose and fix two integers \(N\geq 1\) and \(x\in \{0,1,\ldots ,N\}\). Let \(X_0=x\) and suppose that, conditionally, \(X_{n+1}\) has a \(\operatorname {Binomial}(N,X_n/N)\) distribution given \(X_1,\ldots ,X_n\). More precisely, for all \(n\geq 0\) and \(k=0,\ldots ,N\), \[ \mathbb {P}(X_{n+1}=k\mid \mathcal {F}_n) = \binom {N}{k} \left (\frac {X_n}{N}\right )^k \left (1-\frac {X_n}{N}\right )^{N-k}, \] where \(\mathcal {F}_n=\sigma (\{X_1,\ldots ,X_n\})\) for all \(n\geq 1\). Prove that:

(a) \(X_{\infty }=\lim _{n\to \infty }X_n\) exists a.s. and in \(L^p(\mathbb {P})\) for all \(1\leq p<\infty \).

(b) Prove that \(\mathbb {E}(X_{\infty })=x\) and \(\mathbb {E}(X_{\infty }^2)=Nx\).

(c) Compute the distribution of \(X_{\infty }\).

Let \(n\geq 1\) be a fixed integer and recall that there are \(n!\) permutations of \([n]:=\{1,\ldots ,n\}\). Let \(\sigma :=\{\sigma (1),\ldots ,\sigma (n)\}\) denote an arbitrary permutation of \([n]\), and choose and fix some integer \(j\in [n]\). We say that \(\sigma \) fixes \(j\) if \(\sigma (j)=j\). Now suppose that \(\sigma \) is selected at random, all permutations of \([n]\) equally likely.

(a) What is the probability that \(\sigma \) fixed \(j\)?

(b) What is the expectation of the number of integers in \([n]\) that are fixed by \(\sigma \)?

A sample of 400 people was asked their degree of support of a balanced budget and their degree of support of public education, with the following results:

Test the hypothesis of independence at \(\alpha =0.05\). (A chi-square table is provided.)

Fix \(n\geq 2\) and let \(X,Y_1,\ldots ,Y_n\) be jointly distributed random variables. We say that \(Y_1,\ldots ,Y_n\) are conditionally i.i.d. given \(X\) if \[ \mathbb {P}(Y_1\leq y_1,\ldots ,Y_n\leq y_n\mid X) =\mathbb {P}(Y_1\leq y_1\mid X)\cdots \mathbb {P}(Y_1\leq y_n\mid X) \]

for all \(y_1,\ldots ,y_n\). Show that, if \(Y_1,\ldots ,Y_n\) are conditionally i.i.d. given \(X\), then \[ \operatorname {Var}(Y_1+\cdots +Y_n)=n^2\operatorname {Var}(Y_1)-n(n-1)\mathbb {E}[\operatorname {Var}(Y_1\mid X)]. \]

A random experiment has exactly three possible outcomes, referred to as outcomes 1, 2, and 3, with probabilities \(p_1>0\), \(p_2>0\), and \(p_3>0\), where \(p_1+p_2+p_3=1\). We consider a sequence of independent trials, at each of which the specified random experiment is performed. For \(i=1,2\), let \(N_i\) be the number of trials needed for outcome \(i\) to occur, and put \(N:=\min (N_1,N_2)\).

(a) Show that \(N\) is independent of \(1_{\{N_1<N_2\}}\).

(b) Evaluate \(\mathbb {E}[N_1\mid N_1<N_2]\).

(c) Roll a pair of dice until a total of 6 or 7 appears. Given that 6 appears before 7, what is the (conditional) expected number of rolls?

Let \(X_1,X_2,\ldots \) be an i.i.d. sequence of integer-valued random variables, and let \(\lambda _0\) be a root of the equation \(\mathbb {E}[\lambda ^{X_1}]=1\). (If \(\lambda _0\) is complex, it suffices to notice that expectations, conditional expectations, and martingales extend easily to complex-valued random variables.) Let \(S_n:=X_1+\cdots +X_n\) and \(M_n:=\lambda _0^{S_n}\) for each \(n\geq 0\), where \(S_0:=0\).

(a) Show that \(\{M_n\}_{n\geq 0}\) is a martingale with respect to \(\mathcal {F}_n:=\sigma (X_1,\ldots ,X_n)\).

(b) Consider the special case that \(\mathbb {P}\{X_1=1\}=p\) and \(\mathbb {P}\{X_1=-1\}=1-p\), where \(0<p<1\). Show that, in this case, \(\lim _{n\to \infty }M_n\) exists a.s. and is finite.

(c) Find another example to show that \(\lim _{n\to \infty }M_n\) need not exist a.s.

Let \(X\) and \(Y\) be independent, each having the standard normal distribution, and let \((R,\Theta )\) be the polar coordinates for \((X,Y)\).

(a) Show that \(X+Y\) and \(X-Y\) are independent, and that \(R^2=[(X+Y)^2+(X-Y)^2]/2\), and conclude that the conditional distribution of \(R^2\) given \(X-Y\) is the chi-squared with one degree of freedom translated by \((X-Y)^2/2\).

(b) Show that the conditional distribution of \(R^2\) given \(\Theta \) is chi-squared with two degrees of freedom.

(c) If \(X-Y=0\), the conditional distribution of \(R^2\) is chi-squared with one degree of freedom. If \(\Theta =\pi /4\) or \(\Theta =5\pi /4\), the conditional distribution of \(R^2\) is chi-squared with two degrees of freedom. But the events \(\{X-Y=0\}\) and \(\{\Theta =\pi /4\}\cup \{\Theta =5\pi /4\}\) are the same. Resolve the apparent contradiction.

Let \(X_1\) and \(X_2\) be independent standard normal random variables. Find the necessary and sufficient condition for the independence of \(Y_1=a_1X_1+a_2X_2\) and \(Y_2=b_1X_1+b_2X_2\). (7 points)

Suppose \(Y,X_1,X_2,\ldots \) have the following properties:

(a) \(Y\) has the exponential distribution. That is, \(\mathbb {P}\{Y>t\}=e^{-t}\) for \(t>0\).

(b) Conditionally on \(Y\), \(X_1,\ldots ,X_n\) are i.i.d. exponentials with parameter \(Y\), i.e., for all \(t_1,\ldots ,t_n>0\), \[ \mathbb {P}(X_1>t_1,\ldots ,X_n>t_n\mid Y) =\prod _{i=1}^{n}e^{-Yt_i} =e^{-Y\sum _{i=1}^{n}t_i} \qquad \text {a.s.} \]

Compute \(\mathbb {E}[Y\mid X_1,\ldots ,X_n]\). Use this to prove that \(\{M_n\}_{n=1}^{\infty }\) is a martingale, where \[ M_n=\frac {n+1}{1+X_1+\cdots +X_n} \qquad n=1,2,\ldots . \]

Let \(X_1,X_2,\ldots ,X_n\) denote a random sample from a Poisson distribution with parameter \(\theta >0\).

(a) Show that \((-1)^{X_1}\) is unbiased for \(e^{-2\theta }\), and use the Lehmann–Scheffé theorem to deduce a UMVUE of \(e^{-2\theta }\) based on the sample. (Hint: The conditional distribution of \(X_1\) given \(X_1+\cdots +X_n\) has a simple form.) (5 points)

(b) Argue that the UMVUE in part (a) is a consistent estimator of \(e^{-2\theta }\). (5 points)

(a) Let \(X_1,X_2,\ldots \) be i.i.d. Poisson random variables with mean 1, and let \(Z\) be \(N(0,1)\). Denote \(a^-=-\min \{a,0\}\), and prove that \[ \text {(*)}\qquad \mathbb {E}\left [\left (\frac {S_n-n}{\sqrt n}\right )^-\right ]\to \mathbb {E}[Z^-]. \]

(b) Evaluate both expectations in (*) explicitly, and, noting the telescoping sum, deduce Stirling’s formula, for \(n!\).