Mathematics Qualifying Exam Question Bank

Department of Mathematics

Question

Math 6610 • Analysis of Numerical Methods I

Fall 2000 • Question 2

Question

Math 6610 • Analysis of Numerical Methods I Fall 2000 • Question 2

Linear Least Squares Let \(A\) be a real \(m\times n\) matrix of full rank \((m>n)\) and let \(b\in \mathbb {R}^m\). The Least Square problem is to find \(x\in \mathbb {R}^n\) such that \[ \lVert Ax-b\rVert _2 \] is minimized.

Describe the following three approaches to solve the problem: 1) normal equations, 2) QR decomposition, and 3) Singular Value Decomposition (SVD). Discuss briefly the advantages and disadvantages of these approaches. In particular, give a geometric interpretation of the normal equation approach and explain why it is not often used?

{Linear Least Squares}
Let \(A\) be a real \(m\times n\) matrix of full rank \((m>n)\) and let \(b\in\R^m\). The
Least Square problem is to find \(x\in\R^n\) such that
\[
  \lVert Ax-b\rVert_2
\]
is minimized.

Describe the following three approaches to solve the problem: 1) normal equations,
2) QR decomposition, and 3) Singular Value Decomposition (SVD). Discuss briefly the
advantages and disadvantages of these approaches. In particular, give a geometric
interpretation of the normal equation approach and explain why it is not often used?

Topic Tags

Least Squares and Normal Equations Linear Programming and Optimization Matrix Rank and Fundamental Linear Algebra Singular Value Decomposition