TABLE OF CONTENTS This chapter gives an overview of the relevant concepts in linear algebra that are useful in later chapters. It begins with a review of basic matrix theory and introduces the elementary notation used throughout the book. The convergence analysis of iterative methods requires a good level of knowledge in mathematical analysis and in linear algebra. Traditionally, many of the concepts presented specifically for these analyses have been geared toward matrices arising from the discretization of partial differential equations (PDEs) and basic relaxation-type methods. These concepts are now becoming less important because of the trend toward projection-type methods, which have more robust convergence properties and require different analysis tools. The material covered in this chapter will be helpful in establishing some theory for the algorithms and defining the notation used throughout the book.
For the sake of generality, all vector spaces considered in this chapter are complex, unless otherwise stated. A complex n m matrix A is an n m array of complex numbers
The set of all n m matrices is a complex vector space denoted by 
. The main operations with matrices are the following:
Addition: C = A + B, where A, B, and C are matrices of size n m and
Multiplication by a scalar: C = ? A, where
Multiplication by another matrix:
where 
, 
, 
, and
Sometimes, a notation with column vectors...
