I. Introduction

Least squares estimation, originally stimulated by astronomical studies, is one of the oldest problems in estimation theory with numerous engineering applications, including flight vehicles. It was invented and applied during the period 1795 1806 independently by A.-M. Legendre and C.-F. Gauss to describe planetary motion.1 ^,2 Historians have, however, acknowledged Gauss as the inventor, who applied his method to a meager set of three well-chosen observations to obtain a preliminary estimate, and used further observations made over a period of 41 days to improve the prediction of the orbital position of the asteroid Ceres discovered by the astronomer Piazzi in January 1801, and thereby helped to precisely relocate Ceres after a period of several months in December 1801.3 Application of his approach to several other planetary bodies located by astronomers proved the generality of the technique and marked the beginning of estimation theory.

Least squares (LS) techniques, also called regression analysis, belong to a class of methods called the equation-error methods, because they minimize a cost function defined directly in terms of an input output equation. The cost function is not based on probability theory, as was the case with the maximum-likelihood method discussed in Chapter 4. The linear least squares technique is characterized by its mathematical simplicity, in the sense that the estimates are obtained by applying matrix algebra operations in a one-shot computational procedure. The regression techniques can be applied to nonlinear models as well, because the basic principle is to minimize the sum of squares of...