Spike-based learning rules and stabilization of 
persistent neural activity 
Xiaohui Xie and H. Sebastian Seung 
Dept. of Brain & Cog. Sci., MIT, Cambridge, MA 02139 
xhxie, seung @mit.edu 
Abstract 
We analyze the conditions under which synaptic learning rules based 
on action potential timing can be approximated by learning rules based 
on firing rates. In particular, we consider a form of plasticity in which 
synapses depress when a presynaptic spike is followed by a postsynaptic 
spike, and potentiate with the opposite temporal ordering. Such differen- 
tial anti-Hebbianplasticity can be approximated under certain conditions 
by a learning rule that depends on the time derivative of the postsynaptic 
firing rate. Such a learning rule acts to stabilize persistent neural activity 
patterns in recurrent neural networks. 
1 INTRODUCTION 
Recent experiments have 
demonstrated types of synaptic 
plasticity that depend on the 
temporal ordering of presynap- 
tic and postsynaptic spiking. At 
cortical[ 1] and hippocampal[2] 
synapses, long-term potenti- 
ation is induced by repeated 
pairing of a presynaptic spike 
and a succeeding postsynaptic 
spike, while long-term depres- 
sion results when the order 
is reversed. The dependence 
of the change in synaptic 
strength on the difference 
/X't = tpost -- tpre between 
postsynaptic and presynaptic 
spike times has been measured 
quantitatively. This pairing 
function, sketched in Figure 
1A, has positive and negative 
width of tens of milliseconds. 
pairing function as differential 
A 
B 
0 
o 
tpost - tpr e 
(re II Illl IllIll II II Illlllll![I I llll 
post IIIIlllllll Illllllllllll 
o.51 
o 1 ooo 2000 
time (ms) 
Figure 1: (A) Pairing function for differential Heb- 
bian learning. The change in synaptic strength is plot- 
ted versus the time difference between postsynaptic 
and presynaptic spikes. (B) Pairing function for dif- 
ferential anti-Hebbian learning. (C) Differential anti- 
Hebbian learning is driven by changes in firing rates. 
The synaptic learning rule of Eq. (1) is applied to two 
Poisson spike trains. The synaptic strength remains 
roughly constant in time, except when the postsynap- 
tic rate changes. 
lobes correspond to potentiation and depression, and a 
We will refer to synaptic plasticity associated with this 
Hebbian plasticity--Hebbian because the conditions for 
200 X. Xie and H. S. Seung 
potentiation are as predicted by Hebb[3], and differential because it is driven by the 
difference between the opposing processes of potentiation and depression. 
The pairing function of Figure 1A is not characteristic of all synapses. For example, an 
opposite temporal dependence has been observed at electrosensory lobe synapses of elec- 
tric fish[4]. As shown in Figure lB, these synapses depress when a presynaptic spike is 
followed by a postsynaptic one, and potentiate when the order is reversed. We will refer to 
this as differential anti-Hebbian plasticity. 
According to these experiments, the maximum ranges of the differential Hebbian and anti- 
Hebbian pairing functions are roughly 20 and 40 ms, respectively. This is fairly short, and 
seems more compatible with descriptions of neural activity based on spike timing rather 
than instantaneous firing rates[5, 6]. In fact, we will show that there are some conditions 
under which spike-based learning rules can be approximated by rate-based learning rules. 
Other people have also studied the relationship between spike-based and rate-based learn- 
ing rules[7, 8]. 
The pairing functions of Figures 1A and lB lead to rate-based learning rules like those 
traditionally used in neural networks, except that they depend on temporal derivatives of 
firing rates as well as firing rates themselves. We will argue that the differential anti- 
Hebbian learning rule of Figure lB could be a general mechanism for tuning the strength 
of positive feedback in networks that maintain a short-term memory of an analog variable 
in persistent neural activity. A number of recurrent network models have been proposed 
to explain memory-related neural activity in motor [9] and prefrontal[10] cortical areas, 
as well as the head direction system [11] and oculomotor integrator[12, 13, 14]. All of 
these models require precise tuning of synaptic strengths in order to maintain continuously 
variable levels of persistent activity. As a simple illustration of tuning by differential anti- 
Hebbian learning, a model of persistent activity maintained by an integrate-and-fire neuron 
with an excitatory autapse is studied. 
2 SPIKE-BASED LEARNING RULE 
Pairing functions like those of Figure 1 have been measured using repeated pairing of a 
single presynaptic spike with a single postsynaptic spike. Quantitative measurements of 
synaptic changes due to more complex patterns of spiking activity have not yet been done. 
We will assume a simple model in which the synaptic change due to arbitrary spike trains is 
the sum of contributions from all possible pairings of presynaptic with postsynaptic spikes. 
The model is unlikely to be an exact description of real synapses, but could turn out to be 
approximately valid. 
We will write the spike train of the ith neuron as a series of Dirac delta functions, si (t) = 
y', 6(t - T), where T is the nth spike time of the ith neuron. The synaptic weight from 
neuron j to i at time t is denoted by Wij (t). Then the change in synaptic weight induced 
by presynaptic spikes occurring in the time interval [0, T] is modeled as 
Wij(T + A) - Wij(A) = dtj dti f(ti - tj)si(ti)sj(tj) 
(1) 
Each presynaptic spike is paired with all postsynaptic spikes produced before and after. 
For each pairing, the synaptic weight is changed by an amount depending on the pairing 
function f. The pairing function is assumed to be nonzero inside the interval [-r, r], and 
zero outside. We will refer to r as the pairing range. 
According to our model, each presynaptic spike results in induction of plasticity only after 
a latency A. Accordingly, the arguments T + A and  of Wij on the left hand side of the 
equation are shifted relative to the limits T and 0 of the integral on the right hand side. We 
Spike-based Learning and Stabilization of Persistent Neural Activity 201 
will assume that the latency A is greater than the pairing range r, so that Wi3 at any time is 
only influenced by presynaptic and postsynaptic spikes that happened before that time, and 
therefore the learning rule is causal. 
3 RELATION TO RATE-BASED LEARNING RULES 
The learning rule of Eq. (1) is driven by correlations between presynaptic and postsynaptic 
activities. This dependence can be made explicit by making the change of variables u -- 
ti - tj in Eq. (1), which yields 
Wij(T + )- WO(A) 
where we have defined the cross-correlation 
duf(u)Ci(u) (2) 
T 
Ci(u) = fo dt si(t + u) s(t) . 
(3) 
and made use of the fact that f vanishes outside the interval I-r, r]. Our immediate goal 
is to relate Eq. (2) to learning rules that are based on the cross-correlation between firing 
rates, 
j0 T 
C[jate(u) - dtvi(t + u) vj(t) (4) 
There are a number of ways of defining instantaneous firing rates. Sometimes they are 
computed by averaging over repeated presentations of a stimulus. In other situations, they 
are defined by temporal filtering of spike trains. The following discussion is general, and 
should apply to these and other definitions of firing rates. 
The "rate correlation" is commonly subtracted from the total correlation to obtain the "spike 
correlation" Cid. pik = Cij - C ate. To derive a rate-based approximation to the learning 
rule (2), we rewrite it as 
: /: spike 
Wij(T + ) - Wij(A) = du f(u)C[?(u) + du f(u)Cij (u) 
T T 
(5) 
and simply neglect the second term. Shortly we will discuss the conditions under which 
this is a good approximation. But first we derive another form for the first term by applying 
the approximation t/i(t + u)  t/i(t) + ubi(t) to obtain 
 fo r 
/_ duf(u)C[jate(u)  dt[/3o"i(t) + lPi(t)]"j(t) 
T 
(6) 
where we define 
= au(u) 
T 
/3 = duuf(u) 
T 
(7) 
This approximation is good when firing rates vary slowly compared to the pairing range 
r. The learning rule depends on the postsynaptic rate through/30'i +/31hi. When the 
first term dominates the second, then the learning rule is the conventional one based on 
correlations between firing rates, and the sign of/30 determines whether the rule is Hebbian 
or anti-Hebbian. 
In the remainder of the paper, we will discuss the more novel case where/30 = 0. This 
holds for the pairing functions shown in Figures 1A and lB, which have positive and neg- 
ative lobes with areas that exactly cancel in the definition of/30. Then the dependence on 
202 X. Xie and H. S. Seung 
postsynaptic activity is purely on the time derivative of the firing rate. Differential Hebbian 
learning corresponds to/52 > 0 (Figure 1A), while differential anti-Hebbian learning leads 
to/52 < 0 (Figure lB). To summarize the/50 = 0 case, the synaptic changes due to rate 
correlations are approximated by 
(diff. Hebbian) 
Wij cr -biv (diff. anti-Hebbian) (8) 
for slowly varying rates. These formulas imply that a constant postsynaptic firing rate 
causes no net change in synaptic strength. Instead, changes in rate are required to induce 
synaptic plasticity. 
To illustrate this point, Figure 1C shows the result of applying differential anti-Hebbian 
learning to two spike trains. The presynaptic spike train was generated by a 50 Hz Poisson 
process, while the postsynaptic spike train was generated by an inhomogeneous Poisson 
process with rate that shifted from 50 Hz to 200 Hz at 1 sec. Before and after the shift, 
the synaptic strength fluctuates but remains roughly constant. But the upward shift in firing 
rate causes a downward shift in synaptic strength, in accord with the sign of the differential 
anti-Hebbian rule in Eq. (8). 
The rate-based approximation works well for this example, because the second term of Eq. 
(5) is not so important. Let us return to the issue of the general conditions under which 
this term can be neglected. With Poisson spike trains, the spike correlations C ike (u) are 
zero in the limit T ---> cx), but for finite T they fluctuate about zero. The integral over u in 
the second term of (5) dampens these fluctuations. The amount of dampening depends on 
the pairing range r, which sets the limits of integration. In Figure 1C we used a relatively 
long pairing range of 100 ms, which made the fluctuations small even for small T. On the 
other hand, if r were short, the fluctuations would be small only for large T. Averaging 
over large T is relevant when the amplitude of f is small, so that the rate of learning is 
slow. In this case, it takes a long time for significant synaptic changes to accumulate, so 
that plasticity is effectively driven by integrating over long time periods T in Eq. (1). 
In the brain, nonvanishing spike correlations are sometimes observed even in the T ---> oc 
limit, unlike with Poisson spike trains. These correlations are often roughly symmetric 
about zero, in which case they should produce little plasticity if the pairing functions are 
antisymmetric as in Figures 1A and lB. On the other hand, if the spike correlations are 
asymmetric, they could lead to substantial effects[6]. 
4 EFFECTS ON RECURRENT NETWORK DYNAMICS 
The learning rules of Eq. (8) depend on both presynaptic and postsynaptic rates, like learn- 
ing rules conventionally used in neural networks. They have the special feature that they 
depend on time derivatives, which has computational consequences for recurrent neural 
networks of the form 
+ = Wrr(x ) + bi (9) 
J 
Such classical neural network equations can be derived from more biophysically realistic 
models using the method of averaging[ 15] or a mean field approximation[ 16]. The firing 
rate of neuron j is conventionally identified with v = a(xj). 
1 
The cost function E({ xi }; { Wij }) =  Y'i v quantifies the amount of drift in firing rate at 
the point x,... , xv in the state space of the network. If we consider bi to be a function of 
x and Wij defined by (9), then the gradient of the cost function with respect to Wij is given 
by OE/OW 0 = a' (x)t)i vj. Assuming that rr is a monotonically increasing function so that 
a' (xi) > 0, it follows that the differential Hebbian update of (8) increases the cost function, 
Spike-based Learning and Stabilization of Persistent Neural Activity 203 
and hence increases the magnitude of the drift velocity. In contrast, the differential anti- 
Hebbian update decreases the drift velocity. This suggests that the differential anti-Hebbian 
update could be useful for creating fixed points of the network dynamics (9). 
5 PERSISTENT ACTIVITY IN A SPIKING AUTAPSE MODEL 
The preceding arguments about drift velocity were based on approximate rate-based de- 
scriptions of learning and network dynamics. It is important to implement spike-based 
learning in a spiking network dynamics, to check that our approximations are valid. 
Therefore we have numerically simu- 
lated the simple recurrent circuit of 
integrate-and-fire neurons shown in Fig- 
ure 2. The core of the circuit is the 
"memory neuron," which makes an exci- 
tatory autapse onto itself. It also receives 
synaptic input from three input neurons: 
a tonic neuron, an excitatory burst neu- 
ron, and an inhibitory burst neuron. It is 
known that this circuit can store a short- 
term memory of an analog variable in 
persistent activity, if the strengths of the 
autapse and tonic synapse are precisely 
tuned[17]. Here we show that this tun- 
ing can be accomplished by the spike- 
based learning rule of Eq. (1), with a d- 
ifferential anti-Hebbian pairing function 
like that of Figure 1B. 
The memory neuron is described by the equations 
IIIIIIIIIIIIIlllllllllttlllllllllll 
TONIC 
EXCITATORY BURST 
MEMORY 
II II I I I I lltlllllltllll!11111111t IIIIIIIIIII 
INHIBITORY BURST 
Figure 2: Circuit diagram for autapse model 
= -gL(v' - v'L) - gE(v - v'E) - g(v - ) 
dV 
C. dt 
dr 
+ r = 5(t- (1 l) 
(10) 
where V is the membrane potential. When V reaches h,-es, a spike is considered to have 
occurred, and V is reset to V,.,et. Each spike at time T, causes a jump in the synaptic 
activation r of size a,./ry,, after which r decays exponentially with time constant 
until the next spike. 
The synaptic conductances of the memory neuron are given by 
g = Wr + Woro + W+r+ gr = W_r_ (12) 
The term Wr is recurrent excitation from the autapse, where W is the strength of the au- 
tapse. The synaptic activations r0, r+, and r_ of the tonic, excitatory burst, and inhibitory 
burst neurons are governed by equations like (10) and (11), with a few differences. These 
neurons have no synaptic input; their firing patterns are instead determined by applied cur- 
rents Ipp,o, Ipp,+ and Ipp,_. The tonic neuron has a constant applied current, which 
makes it fire repetitively at roughly 20 Hz (Figure 3). For the excitatory and inhibitory 
burst neurons the applied current is normally zero, except for brief 100 ms current pulses 
that cause bursts of action potentials. 
As shown in Figure 3, if the synaptic strengths W and Wo are arbitrarily set before learning, 
the burst neurons cause only transient changes in the firing rate of the memory neuron. 
After applying the spike-based learning rule (1) to tune both W and Wo, the memory 
204 X. Xie and H. S. Seung 
llllllllllll I IIIIIIII1111 I IIIIIIIlllll I 
untuned 
I I 
tuned 
IIIIIIII I I I I I I I II III I 
! 
1 sec 
II ]iiiiiiiiiiiii:---ir'lm#", I I IIIIIIIIIIIIlllll 
Figure 3: Untuned and tuned autapse activity. The middle three traces are the membrane 
potentials of the three input neurons in Figure 2 (spikes are drawn at the reset times of 
the integrate-and-fire neurons). Before learning, the activity of the memory neuron is not 
persistent, as shown in the top trace. After the spike-based learning rule (1) is applied to 
the synaptic weights W and W0, then the burst inputs cause persistent changes in activity. 
Cm = 1 nF, g = 0.025 itS, V = -70 mV, VE = 0 mV, VI = -70 mV, Vtnre8 = -52 
mV, V.eset = -59 mV, c = 1, ry = 100 ms, Iapp,o = 0.5203 hA, Iapp,+= 0 or 0.95 
hA, rymo = 100 ms, ryn,+ = *-sy,- = 5 ms, W+ = 0.1, W_ = 0.05. 
neuron is able to maintain persistent activity. During the interburst intervals (from A after 
one burst until A before the next), we made synaptic changes using the differential anti- 
Hebbian pairing function f(t) = -A sin(rt/r) for spike time differences in the range 
[-r, r] with A - 1.5 x 10 -4 and r=A=120 ms. The resulting increase in persistence time 
can be seen in Figure 4A, along with the values of the synaptic weights versus time. 
To quantify the performance of the system at maintaining persistent activity, we determined 
the relationship between dv/dt and v using a long sequence of interburst intervals, where v 
was defined as the reciprocal of the interspike interval. If W and W0 are fixed at optimally 
tuned values, there is still a residual drift, as shown in Figure 4B. But if these parameters are 
allowed to adapt continuously, even after good tuning has been achieved, then the residual 
drift is even smaller in magnitude. This is because the learning rule tweaks the synapfic 
weights during each interburst interval, reducing the drift for that particular firing rate. 
Autapse learning is driven by the autocorrelation of the spike train, rather than a cross- 
correlation. The peak in the autocorrelogram at zero lag has no effect, since the pairing 
function is zero at the origin. Since the autocorrelation is zero for small time lags, we used 
a fairly large pairing range in our simulations. In a recurrent network of many neurons, a 
shorter pairing range would suffice, as the cross-correlation does not vanish near zero. 
6 DISCUSSION 
We have shown that differential anti-Hebbian learning can tune a recurrent circuit to main- 
tain persistent neural activity. This behavior can be understood by reducing the spike-based 
learning rule (1) to the rate-based learning rules of Eqs. (6) and (8). The rate-based approx- 
imations are good if two conditions are satisfied. First, the pairing range must be large, or 
the rate of learning must be slow. Second, spike synchrony must be weak, or have little 
effect on learning due to the shape of the pairing function. 
The differential anti-Hebbian pairing function results in a learning rule that uses -hi as a 
negative feedback signal to reduce the amount of drift in firing rate, as illustrated by our 
simulations of an integrate-and-fire neuron with an excitatory autapse. More generally, 
the learning rule could be relevant for tuning the strength of positive feedback in network- 
s that maintain a short-term memory of an analog variable in persistent neural activity. 
Spike-based Learning and Stabilization of Persistent Neural Activity 205 
250 
200 
5O 
wor w 1 w 
3850 10.12 
B 
% 5 10 15 20 25 20 40 60 80 100 
ttme (s) rate (Hz) 
Figure 4: Tuning the autapse. (A) The persistence time of activity increases as the weight- 
s W and Wo are tuned. Each transition is driven by pseudorandom bursts of input (B) 
Systematic relationship between drift dr/dr in firing rate and v, as measured from a long 
sequence of interburst intervals. If the weights are continuously fine-tuned ('*') the drift is 
less than with fixed well-tuned weights (' o'). 
For example, the learning rule could be used to improve the robustness of the oculomotor 
integrator[12, 13, 14] and head direction system[11] to mistuning of parameters. In deriv- 
ing the differential forms of the learning rules in (8), we assumed that the areas under the 
positive and negative lobes of the pairing function are equal, so that the integral defining 
/o vanishes. In reality, this cancellation might not be exact. Then the ratio of/ and/o 
would limit the persistence time that can be achieved by the learning rule. 
Both the oculomotor integrator and the head direction system are also able to integrate 
vestibular inputs to produce changes in activity patterns. The problem of finding general- 
izations of the present learning rules that train networks to integrate is still open. 
References 
[11 
[2] 
[31 
[41 
[51 
[61 
[71 
[8] 
[9] 
[10] 
[111 
[121 
[131 
[14] 
[15] 
[161 
[17] 
H. Markram, J. Lubke, M. Frotscher, and B. Sakmann. Science, 275(5297):213-5, 1997. 
G. Q. Bi and M. M. Poo. JNeurosci, 18(24):10464-72, 1998. 
D. O. Hebb. Organization of behavior. Wiley, New York, 1949. 
C. C. Bell, V. Z. Han, Y. Sugawara, and K. Grant. Nature, 387(6630):278-81, 1997. 
W. Gerstner, R. Kempter, J. L. van Hemmen, and H. Wagner. Nature, 383(6595):76-81, 1996. 
L. F. Abbott and S. Song. Adv. Neural Info. Proc. Syst., 11, 1999. 
P. D. Roberts. J. Cornput. Neurosci., 7:235-246, 1999. 
R. Kempter, W. Gerstner, and J. L. van Hemmen. Phys. Rev. E, 59(4):4498-4514, 1999. 
A. P. Georgopoulos, M. Taira, and A. Lukashin. Science, 260:47-52, 1993. 
M. Camperi and X. J. Wang. J Cornput Neurosci, 5(4):383-405, 1998. 
K. Zhang. J. Neurosci., 16:2112-2126, 1996. 
S.C. Cannon, D. A. Robinson, and S. Shamma. Biol. Cybern., 49:127-136, 1983. 
H. S. Seung. Proc. Natl. Acad. Sci. USA, 93:13339-13344, 1996. 
H. S. Seung, D. D. Lee, B. Y. Reis, and D. W. Tank. Neuron, 2000. 
B. Ermentrout. Neural Cornput., 6:679-695, 1994. 
O. Shriki, D. Hansel, and H. Sompolinsky. Soc. Neurosci. Abstr., 24:143, 1998. 
H. S. Seung, D. D. Lee, B. Y. Reis, and D. W. Tank. J. Cornput. Neurosci., 2000. 
PART III 
THEORY 
