5.1 Ordinary Least Squares

The general form of the model equation (5.3) and the regression equation (5.4) can be written using vector and matrix notation as

and

where

• z=[ z(1) z(2) z( N)] ^T= N 1 vector

• ?=[ ? ₀ ? ₁ ? _n] ^T= n _p 1 vector of unknown parameters, n _p= n+1

• X =[1 ? ₁ ? _n] =N n _p matrix of vectors of ones and regressors

• v =[ v(1) v(2) v(N)] ^T= N 1 vector of measurement errors

The regressor vectors ? _j, j=1,2,..., n, are known postulated functions of the vectors of independent variables. Usually, at least some of the regressors are equal to the independent variables themselves.

Regression equation (5.6) is equivalent to measurement equation (4.2), with H = X . For the least-squares model, there are no probability statements regarding ? or v , but v is assumed to be zero mean and uncorrelated, with constant variance,

As discussed in Chapter 4, the best estimator of ? in a least-squares sense comes from minimizing the sum of squared differences between the measurements and the model,

The parameter estimate that minimizes the cost function J( ?) must satisfy

or

or

The n _p= n+1 equations represented in Eqs. (5.9) are called the normal equations.