Math 6610 • Analysis of Numerical Methods I

**Linear Least Squares:** The Linear Least Squares problem for an \(m\times n\) real matrix \(A\) and \(b\in\mathbb{R}^m\) is the problem: \[ \text{Find } x\in\mathbb{R}^n \text{ such that } \|Ax-b\|_2 \text{ is minimized.} \] **a)** Suppose that you have data \(\{(t_j,y_j)\}\), \(j=1,2,\ldots,m\) that you wish to approximate by an expansion \[ p(t)=\sum_{k=1}^{n} x_k\phi_k(t). \] Here, the functions \(\phi_k(t)\) are given functions. Which norm on the difference between the approximation function \(p\) and the data gives rise to a linear least squares problem for the unknown expansion coefficients \(x_k\)? What is the matrix \(A\) in this case, and what is the vector \(b\)? **b)** Suppose that \(A\) is a real \(m\times n\) matrix of full rank and let \(b\in\mathbb{R}^m\). What are the `normal equations' for the Least Squares problem? How can they be used to solve the Least Squares problem? What is the QR factorization of \(A\) and how can it be used to solve the Least Squares problem? Compare and contrast these approaches for numerically solving the Least Square problem.