TABLE OF CONTENTS Least-squares fitting is often called regression analysis for historical reasons. For example, Mathematica provides the package called Statistics Linear Regression for doing a linear least-squares fit. Regression analysis applies when we have, for example, two quantities like height and weight, or income and education and we want to predict one from information about the other. This is possible if there is a correlation between the two quantities, even if we lack a model to account for the correlation. Typically in these problems, there is little experimental error in the measurements compared to the intrinsic spread in their values. Thus, we can talk about the regression line for income on education. The regression line is often called the least-squares line because it makes the sum of the squares of the residuals as small as possible.
In contrast, model fitting usually assumes that there is an underlying exact relationship between some measured quantities, and we are interested in the best choice of model parameters or in comparing competing models. In some areas, especially in the life sciences, the phenomena under study are sufficiently complex that good models are hard to come by and regression analysis is the name of the game.
The results from a regression analysis may make no physical sense but may point the way to a physical model. Consider the following simple example which assumes the experimenter is ignorant of an elementary geometrical relationship, that the area of a rectangle is equal...
