TABLE OF CONTENTS The original Hebb rule, states how synapse efficacies (or weights) are strengthened [Hebb, 1949]:
When an axon in cell A is near enough to excite cell B and repeatedly and persistently takes part in firing it, some growth process or metabolic change takes place in one or both cells such that A's efficiency in firing B, is increased.
Later, it was suggested that synaptic strengths may decrease during learning as well. This may happen for two distinct reasons, first strengths change as a function of the correlations between the pre- and post- synaptic neurons, i.e. they increase when the activities of the neurons are correlated, and decrease when they are anti-correlated as was originally suggested by Stent [Stent, 1973] and used subsequently in much of the work on this issue [Oja, 1982; Linsker, 1986b]; Miller et al., 1989]. Furthermore, as discussed in the next section, synaptic efficacy decreases may occur in order to stabilize learning [Rochester et al., 1956; von der Malsburg, 1973; Oja, 1982; Linsker, 1986b].
A mathematically simple form, that will nevertheless be general enough for understanding the basic properties of such learning rules, is the linear rule [Linsker, 1986b]
where d i are the activities of presynaptic neurons, c is the activity of the postsynaptic neuron, ? m i is the change in the value of the synaptic efficacy between presynaptic neuron i and the postsynaptic neuron, and ?, d o and c o,...
