TABLE OF CONTENTS Many of the qualitative features of the BCM theory depend on its incorporation of statistics beyond the second order. (Other possible learning rules, discussed in the previous chapters, such as kurtosis also incorporate third or higher order statistics). Numerous proposed learning rules that incorporate only second order statistics have been extensively investigated in many complex situations. The simulations used in these investigations employ various initial and final points as well as a variety of parameters so that it is often difficult to tell what features are producing the particular result. To help clarify the connections between assumptions and conclusions for these models, we introduce some new analytic tools. In this chapter, aided by these tools, we analyze the properties of these second order rules as well as their ability to account for the experimental data.
The cornerstone of all unsupervised learning rules is the Hebb rule [Hebb, 1949]. However this rule, in its original form, suffers from two major problems: (1) all modification are positive so that all weights grow (2) it is unstable, i.e., the synaptic weights will all be driven to their maximal value. Numerous learning rules have been proposed to overcome these difficulties. Stability can be attained by normalization of the weights[von der Malsburg, 1973], appropriate decay terms added to the function for the change in the weights [Oja, 1982; Sanger, 1989] or saturation limits on the weights themselves[Linsker, 1986b; Miller et al., 1989]. In addition inputs and outputs can be measured with respect to...
