Our proposed prior (8.68) makes use of the assumption of prior independence,

which states that, prior to the experiment, the belief systems about ? and ? are independent. The posterior density is then

Assuming the Gauss Markov conditions hold and that the errors are normally distributed, the likelihood function is

and thus the posterior is

If, as we have assumed in (8.68), the prior is uniform in the region of appreciable nonzero likelihood, p( ?) ~ c, then the most probable value of ?, for any value of ?, is that which minimizes S( ?), (8.58). Therefore, the least-squares method is justified statistically, as long as the Gauss Markov conditions hold and the errors are normally distributed.

It is shown in the supplemental material in the accompanying website that, in the frequentist view, least squares is an unbiased estimator of the true value (i.e., if we repeat the set of experiments many times, the average estimate is the true value) if merely the zero-mean Gauss Markov condition (8.50) is satisfied.

Numerical treatment of nonlinear least-squares problems

For a linear model y ^{[ k ]} = x ^{[ k ]} ? + ? ^{[ k ]}, we obtain the least-squares estimate by solving an algebraic system [ X ^TX] ? _LS = X ^T y. For a nonlinear model

we must find the least-squares estimate through numerical optimization. For notational convenience, we...