5.3 Nonlinear Least Squares

In some modeling problems, the relationship between the regressors and the response variable is nonlinear in the parameters. For this case, the least-squares model was formulated in Chapter 4 as

which is equivalent to a nonlinear regression model,

where x ^T (i) is a row vector of regressors computed from measured data at the ith data point, and f is a nonlinear function of x (i) and the parameters in the vector ? .

As before, the least-squares estimator can be obtained by minimizing the sum of squared errors,

The minimum of the preceding cost function is found by satisfying the normal equations,

where ?J/ ? ? is a row vector containing the partial derivatives of the nonlinear scalar function J( ? ) with respect to the elements of ? , and ?f[ x( i), ?]/ ? ? is a row vector of output sensitivities to changes in the model parameters.

Equation (5.116) is a set of nonlinear algebraic equations. This means that cannot be obtained by simple matrix algebra, as in the case of a model that is linear in the parameters. Instead, an iterative nonlinear optimization technique must be used. There are many different numerical methods to solve this nonlinear minimization problem (see Ref. 12). Some of these methods will be explained in Chapter 6.