Symmetric Matrices

(20 points) Let \(Z = (Z_{1}, Z_{2}, Z_{3})^{\top } \sim \mathcal {N}(0, I_{3})\) and define the random vector \[ X = \begin {pmatrix} X_{1} \\ X_{2} \end {pmatrix} = \begin {pmatrix} Z_{1} + Z_{2} \\ Z_{2} + Z_{3} \end {pmatrix}. \]

1. Compute the mean vector \(\mathbb {E}[X]\) and the covariance matrix \(\Sigma = \operatorname {Cov}[X]\).
2. Does \(X\) have a probability density function on \(\mathbb {R}^{2}\)? Briefly justify your answer.
3. Does the joint moment generating function satisfy \[ M_{X}(t_{1}, t_{2}) = M_{X}(t_{1}, 0)M_{X}(0, t_{2})? \] Explain your reasoning.
4. Let \(Q = \|X\|^{2}\). Identify a symmetric matrix \(A\) such that \(Q = Z^{\top }AZ\) and compute \(\mathbb {E}[Q]\).

(20 points) Let \[ v = (1, -1, 0)^{\top }, \qquad \Omega = \operatorname {Span}\{v\} \subseteq \mathbb {R}^{3}. \]

1. Write the projection matrix \(P_{\Omega }\) onto \(\Omega \).
2. Verify that \(P_{\Omega }\) is a projection matrix by checking that it is symmetric and idempotent. (To avoid a long calculation, you can write out these calculations in terms of \(v\).)
3. Determine the eigenvalues of \(P_{\Omega }\) and their multiplicities. What is \(\operatorname {rank}(P_{\Omega })\)?
4. Find an orthogonal basis \(\{b_{1}, b_{2}\}\) for \(\Omega ^{\perp }\) and verify that \(P_{\Omega }b_{i} = 0\) for \(i = 1, 2\).

Let \(\mathbf {A}\) be a square symmetric matrix.

1. Show that if \(\mathbf {A}^{k}\) is the identity matrix for some \(k \ge 2\) then \(\mathbf {A}^{2}\) is the identity matrix.
2. Show that if \(\mathbf {A}^{k}\) is the zero matrix for some \(k \ge 2\) then \(\mathbf {A}\) is the zero matrix.

(a) Let \(X\) be a random variable with characteristic function \(\varphi \). Show that \(X\) is symmetric if and only if \(\varphi (t)\) is real for all \(t\).

(b) Give three distinct examples of real characteristic functions.

Let \(X\) be a random variable with characteristic function \(\varphi \). Show that \(X\) is symmetric if and only if \(\varphi (t)\) is real for all \(t\).

Let \(X_1,X_2,\ldots \) be i.i.d., and \(S_n:=X_1+\cdots +X_n\). Assume also that \(\mathbb {P}\{X_1=1\}=\mathbb {P}\{X_1=-1\}=1/2\), so that \(\{S_n\}_{n=1}^{\infty }\) is a simple symmetric random walk. Define \[ T:=\inf \{k\ge 1: |S_k|\ge 2\}. \] As usual, \(\inf \emptyset :=\infty \). Prove that \(\mathbb {E}[T^2]<\infty \).

Let \(X_1,X_2,\ldots ,X_n\) be independent identically distributed random variables with common density function \(f\). Let \[ \hat {f}_n(t)=\frac {1}{nh}\sum _{1\leq i\leq n}K\left (\frac {t-X_i}{h}\right ), \] where \(K\) is a symmetric density function and \(K(t)=0\) if \(|t|>1\). We assume that \(h=h(n)\to 0\) as \(n\to \infty \). We assume that the second derivative of \(f\) exists and continuous in a neighborhood of \(t\). We use \(\hat {f}\) to estimate \(f\). Compute the bias of \(\hat {f}_n(t)\).