We first discuss the regression, or estimation, of parameters in linear models from single-response data. Let us say that we have performed a set of N experiments in which for each experiment k = 1, 2, , N, the set of predictor variables is known a priori, and a measurement is made of the single-response variable y ^[ k ^]. We assume that this single-response variable depends linearly upon the predictors,

? ^{[ k ]} is some random measurement error for the kth experiment. Due to this error, the measured response is not equal to the true response of the system, and the statistical properties of the error are very important, although generally unknown at the time of measurement. The true parameters { ? ₀, ? ₁, , ? _M} are those that describe the system behavior perfectly in the absence of any random, model, or predictor error. That is, we hypothesize that the model is indeed a valid one. Thus, the set of predictor variables chosen indeed completely specifies the response of the system (in the absence of random error) and the relationship between the response and each predictor is truly linear.

In some instances, we wish to fit a model with a zero y-intercept ? ₀ = 0, such that

We introduce a common notation for both cases by defining for each experiment the vectors of predictors and parameters,

Then, the...