TABLE OF CONTENTS In the last lecture we showed how the Arnoldi process can be used to find eigenvalues. Here we show that it can also be used to solve systems of equations Ax = b. The standard algorithm of this kind is known as GMRES, which stands for "generalized minimal residuals."
As in the last two lectures, let 
be a square matrix and 
a vector, and let 
denote the Krylov subspace ? b, Ab,..., A n ?1 b ? of (33.5). Now, however, we assume that A is nonsingular, for our goal is to solve a system of equations Ax = b. It will be convenient to have a notation for the exact solution of this problem: x * = A ?1 b.
The idea of GMRES is a one-liner. At step n we shall approximate x * by the vector 
that minimizes the norm of the residual r n = b ? Ax n. In other words, we shall determine x n by solving a least squares problem, illustrated in Figure 35.1.
The obvious way to solve this least squares problem would be as follows.
Let K n the m n Krylov matrix (33.6), so that we have
The column space of this...
