LS Problems and SVD Let \(A\) be a real \(m\times n\) matrix \((m>n)\) of full rank and let \(b\in \mathbb {R}^m\), the least square problem is to find \(x\in \mathbb {R}^n\) such that \[ \lVert Ax-b\rVert _2 \] is minimized.
a) (13%) Describe the following three approaches to solve the problem: 1) normal equations, 2) QR decomposition, and 3) singular value decomposition (SVD). Discuss briefly the advantages and disadvantages of each approach. In particular, give a geometric interpretation of the normal equation approach, and explain why it is often to be avoided.
b) (7%) In addition to minimizing the above norm, we impose the constraint: \(\lVert x\rVert _2\le \beta \), for some constant \(\beta >0\). This is a constrained LS problem. Use SVD to formulate and solve this problem, notice that there are different cases where a Lagrange multiplier may be needed.