TABLE OF CONTENTS Most theoretical learning rules, derived from Hebb's original postulate, depend primarily on the second order statistics of the input. In this chapter we investigated this family of learning rules, and show that there is a simple common methodology for studying them. This methodology is used to explain how orientation selectivity, ocular dominance, and many deprivation experiments are accounted for by models based on these learning rules.
As observed long ago for this family of plasticity rules to be stable, additional constraints must be imposed. Some of these constraints such as normalization [von der Malsburg, 1973] or the more biologically plausible decay term [Oja, 1982] produce receptive fields that are the principal component of the inputs. Other constraints such as saturation bounds or subtractive decay terms [Miller, 1992; MacKay and Miller, 1994], produce receptive fields that are combinations of several of the largest principal components. Additional constraints are important not only for stabilization of learning but also for producing receptive fields with interesting structure or with structure resembling that found experimentally. For example, a decay term that subtracts the DC projection from the weight growth, might be necessary in many cases in order to reduce the growth rate of principal components with a strong DC component.
We have also shown how orientation selectivity can be produced, by choosing an appropriate correlation function, or from natural images by choosing the appropriate retinal filter. Both of these approaches produce oriented receptive fields that are quite broadly tuned. More significantly, these results...
