Ph.D. preliminary Examination on Linear Models (Math 6010)

Let \(\mathbf {A}\) be a square symmetric matrix.

1. Show that if \(\mathbf {A}^{k}\) is the identity matrix for some \(k \ge 2\) then \(\mathbf {A}^{2}\) is the identity matrix.
2. Show that if \(\mathbf {A}^{k}\) is the zero matrix for some \(k \ge 2\) then \(\mathbf {A}\) is the zero matrix.

Let \(\epsilon _{1}, \ldots , \epsilon _{n}\) and \(\eta _{1}, \ldots , \eta _{m}\) be independent normal random variables with \(\mathbb {E}\epsilon _{i} = \mathbb {E}\eta _{j} = 0\) and \(\mathbb {E}\epsilon _{i}^{2} = 1, \mathbb {E}\eta _{j}^{2} = 4\). We have the models

where the \(x_{i}, y_{i}\) are given and non-random.

1. Find the maximum likelihood estimator for \((\beta _{1}, \beta _{2}, \beta _{3} )\).
2. Compute the distribution of the estimator.

Let \((X_{1}, X_{2})\) be a bivariate normal with \(\mathbb {E}X_{1} = \mathbb {E}X_{2} = 0\) and covariance matrix \[ \begin {bmatrix} 2 & \rho \\ \rho & 2 \end {bmatrix}. \] Determine the \(\rho \) such that \(2X_{1} + X_{2}\) and \(X_{1} + 3X_{2}\) are independent.

Consider the linear model

where the \(\epsilon _{i}\) are independent normals with \(\mathbb {E}\epsilon _{i} = 0\) and \(\mathbb {E}\epsilon _{i}^{2} = i\sigma ^{2}\). Here \(\beta \in \mathbb {R}^{d}\) and \(\sigma ^{2} > 0\) are unknown parameters, while the \(\mathbf {x}_{i} \in \mathbb {R}^{d}\) are all known and deterministic. Compute the maximum likelihood estimator of \((\beta , \sigma ^{2} )\).

Consider the linear model

where the \(\epsilon _{i}\) are independent normal random variables with \(\mathbb {E}\epsilon _{i} = 0\) and \(\mathbb {E}\epsilon _{i}^{2} = \sigma ^{2}\). Assume \(\beta \) and \(\sigma ^{2} > 0\) are deterministic but unknown, while the \(x_{i}\) are deterministic and known. Assume also that \(\sum _i x_i \ne 0\). Find the likelihood ratio test for \(\beta = 0\) versus

Let \((X_{1}, X_{2})\) be independent normal random variables with \(\mathbb {E}X_{1} = \mathbb {E}X_{2} = 0\) and \(\mathbb {E}X_{1}^{2} =\) \(1, \mathbb {E}X_{2}^{2} = 4\). Define \(Y_{1} = X_{1} + X_{2}\) and \(Y_{2} = X_{1} + 2X_{2}\) and let \(\mathbf {Y} = (Y_{1}, Y_{2})^{\top }\).

1. Compute the covariance matrix of \(\mathbf {Y}\).
2. Find a matrix \(\mathbf {D}\) such that \(\mathbf {Y}^{\top }\mathbf {D}\mathbf {Y}\) has a chi-squared distribution.

Consider the density function

where \(C\) is the constant \[ C = \int _{-\infty }^{\infty } e^{-t^{4}}\,dt. \]

Consider the linear model

where the \(\epsilon _{i}\) are iid with pdf \(f(t)\). Assume that \(\beta \in \mathbb {R}^{d}\) and \(\sigma ^{2} > 0\) are deterministic but unknown, while \(\mathbf {x}_{i} \in \mathbb {R}^{d}\) are deterministic and known.

1. Write down the likelihood function of the observations as a function of \(\beta \) (do not try to find the maximum likelihood estimator).
2. Assume temporarily that the \(\epsilon _{i}\) are iid normal with mean zero and variance one, and write down the likelihood function of the observations as a function of \(\beta \) in that case.
3. If you were to solve for the maximum likelihood estimators in the first two parts would they be the same? (Note you are not being asked to actually solve for the estimators).