15.1 Useful Matrix Types

• A Toeplitz matrix A from a sequence a = [ a _{? N +1}, , a _{M ?1}] of length M + N ? 1 is an M N matrix where A _{i, j} = a _{i ? j}, i = 0, , M ? 1, j = 0, , N ? 1, in other words, a matrix where every diagonal from left to right is constant. For instance a 4 5 Toeplitz matrix is given by

  
  

• An M N circulant matrix A from a sequence a = [ a ₀, , a _{M ?1}] is a Toeplitz matrix where a _i = a _{i + N}, so that

  
  

• A band matrix (or banded matrix) A is a matrix for which A _{i, j} = 0 for j ? i < ? K ₁ or j ? i > K ₂ for some K ₁, K ₂ > 0.

• The concept of Toeplitz, circulant, and band matrices can be extended to block matrices, by replacing scalars in the above definitions by matrices.

15.2 Random Variables and Distributions

We will write random variables using capital letters, and realizations/instances of random variables by small letters. So Z is a random variable and z