TABLE OF CONTENTS With this lecture the flavor of the book changes. We move from direct methods, a classical topic that is rather thoroughly understood, to the relatively untamed territory of iterative methods. These are the methods that seem likely to dominate the large-scale computations of the future.
The importance of iterative algorithms in linear algebra stems from a simple fact: noniterative or "direct" algorithms for general matrices require O ( m 3) work. This is too much! It is too much both in the absolute sense that m 3 is huge when m is large, and in the relative sense that since the input to most matrix problems involves only O ( m 2) numbers, it seems unreasonable that O ( m 3) work must be expended in solving them.
The following table gives a thumbnail history of matrix computations over the years:
| 1950: m = 20 | (Wilkinson) |
| 1965: m = 200 | (Forsythe and Moler) |
| 1980: m = 2000 | (LINPACK) |
| 1995: m = 20000 | (LAPACK) |
These numbers represent a rough approximation to what dimensions might have been considered "very large" for a dense, direct matrix computation at the indicated dates. In the "Forsythe and Moler era" of the mid-1960s (named here after an influential textbook published in 1967), for example, a matrix of dimension in the hundreds was large, stretching the limits of what could be calculated on available machines in a reasonable amount of time.
Evidently, in...
