TABLE OF CONTENTS Once the model architecture is chosen, its structure and its parameters must be determined. The present section will focus on the latter problem, and the former will be dealt within the next section.
As the models are nonlinear, parameter estimation is an iterative process. If the parameter update is performed after the presentation of a data set, which will be used in every iteration of the learning algorithm, the process is denominated offline learning, batch learning, or simply training. On the other hand, if the update is performed in a pattern-by-pattern basis, or after the presentation of a data set that changes from iteration to iteration, the learning process is called online learning or adaptation.
Training a neural model means to determine the model parameters in such a way that the sum of the squared errors ( e), between the process output ( y) and the model output ( ?), subject to the same inputs ( x), is minimised:
Training methods will be divided into two classes: methods that can be applied to all the architectures referred above, and those specific to each architecture.
There are two classes of algorithms which are applicable to all architectures described in the last section: gradient-based methods and evolutionary algorithms. The former will be detailed first.
All the models presented in the last section are differentiable in their parameters and, as a consequence, gradient-based methods are...
