TABLE OF CONTENTS If we know that our individual errors and fluctuations follow the magic bell-shaped curve exactly, then the resulting estimates are known to have almost all the nice properties that people have been able to think of.
John W.Tukey, 1965
Least squares are popular for solving inverse problems because they lead to the easiest computations. Their only drawback is their lack of robustness, i.e., their strong sensitivity to a small number of large errors (outliers) in a data set.
In this book, the least-squares criterion is justified by the hypothesis that all initial uncertainties in the problem can be modeled using Gaussian distributions. Covariance operators play a central role in the method; the underlying mathematics is simple and beautiful.
When the equation solving the forward problem is linear, posterior uncertainties are also Gaussian, and an explicit expression is obtained for the posterior probability distribution. When the forward equation is nonlinear, the posterior probability isn t Gaussian, but, if nonlinearities are not too severe, finding the maximum likelihood point of the distribution and estimating the shape of the distribution around this point (i.e., estimating the covariance matrix of the distribution) may satisfactorily solve the problem.
We are about to see that least-squares methods are intimately linked to Gaussian probability distributions. These are only defined over linear spaces (which is why on the surface of the sphere one uses the Fisher distribution, as there is no Gaussian defined).
Later in this section, therefore, the...
