Analog Computation at a Critical Point: A Novel 
Function for Neuronal Oscillations? 
Leonid Kruglyak and William Bialek 
Department of Physics 
University of Califorlfia at Berkeley 
Berkeley, C, alifornia 94720 
and NEC Research Institute* 
4 Independence Way 
Princeton, New Jersey 08540 
Abstract 
We show that a simple spin system biased at its critical point can en- 
code spatial characteristics of external signals, such as the dilnensions of 
"objects" in the visual field, in the temporal correlation functions of indi- 
vidual spins. Qualitative arguments suggest that regularly firing neurons 
should be described by a planar spin of unit length, and such XY models 
exhibit critical dynamics over a broad range of paralneters. We show how 
to extract these spins from spike trains and then measure the interaction 
Hamiltonia. n using simulations of small clusters of cells. Static correla- 
tions among spike trains obtained kom silnulations of large arrays of cells 
are in agreement with the predictions from these Hamiltonians, and dy- 
namic correlations display the predict. ed encoding of spatial inforlnation. 
We suggest that this novel representation of object, dilnensions in temporal 
correlations may be relevant to recent experiments on oscillatory neural 
firing in the visual cortex. 
1 INTRODUCTION 
Physical systems at, a critical point exhibit tong-range correlations even though 
the interactiols among the constituent particles are of short range. Through the 
flucroat, ion-dissipation theorem this implies that the dynalnics at one point iu the 
'Current address. 
137 
138 Kruglyak and Bialek 
system are sensitive to external perturbations which are applied very far away. If 
we build an analog computer poised precisely at. such a critical point. it. should be 
possible to evaluate highly non-local fimctionals of the input signals using a locally 
interconnected architecture. Such a scheme would be very useful for visual compu- 
tations, especially those which require comparisons of widely separated regions of 
the image. Froin a biological poiut of view long-range correlations at. a critical point 
might provide a robust scenario for "responses from beyond the classical receptive 
field" [1]. 
In this paper we present an explicit model for analog computation a.t. a critical 
point and show that this model has a remarkable consequence: Because of dynamic 
scaling, spatial properties of input signals are mapped into temporal correlations 
of the local dynamics. One can, for example, measure the size and topology of 
"objects" in a scene using only the temporal correlations in the output of a single 
COlnputational unit (neuron) located within the object. We then show that our 
abstract model can be realized ill networks of senfi-realistic spiking neurons. The 
key to this construction is that neurons biased in a regime of regular or oscillatory 
firing can be mapped to XY or planar spins [2,3], and two-dimensional arrays of 
these spins exhibit a broad range of parameters in which the system is generically 
at a critical point.. Non-oscillatory lieurolm cannot, in general, be forced to operate 
at a critical point without delicate fine tuning of tile dynamics, fine tuning which 
is iraplausible both for biolog,v and for man-made analog circuits. We suggest that 
these arguments ilia}' be relevant to the recent observations of oscillatory firing in 
the visual cortex [4,5,6]. 
2 A STATISTICAL MECHANICS MODEL 
We consider a silnple two-dilnensional array of spins whose states are defined by unit 
two-vectors Sr. These spins interact with their neighbors so that the total energy of 
the systeln is H = -J ]] S,.S,,, with the sum restricted to nearest neighbor pairs. 
This is the XY model, which is interesting in part because it possesses not a critical 
point. but rather a critical line [7]. At a given temperature, for all J > J one finds 
that correlations alnong spins decay algebraically, {Sty'S,,}  1/[r, -- r,,[" so that. 
there is no characteristic scale or correlation length; more precisely the correlation 
length is infinite. In COltrast, for J < J we },ave 
which defiues a finite correlation length 
In the algebraic phase the dynamics of the spins on long length scales are rigorously 
described by the spin wave approximation, in which one assumes that fluctuations 
in the angle between neighboring spins are small. In this regime it makes sense to 
use a continuum approximation rather than a lattice, and the energy of the system 
becomes H = J f d:xlV(x)[ -", where (x) is the orientation of the spin at position 
x. The dynamics of the system are det. ermined by the Langevin equation 
_ jv2(x,t ) + (1) 
Ot 
where q is a Gaussian thermal noise source with 
= 2x:aT$(x - x'(t - (2) 
Analog Computation at a Critical Point 139 
,Ve Call then show that. the time correlation function of the spin at, a single site x 
is given by 
(S(x, t).S(x, 0)) = exp 
-2kT f dw 
d"2 k 1 - e -iwt ] 
(2rr)2 0; 2 + j.k 4 . (3) 
In fact. Eq. 3 is valid only for an infinite array of spins. Imagine that external signals 
to this array of spins Call "activate" and "deactivate" the spins so that one must 
really solve Eq. 1 on finite regions or clusters of active spills. Then we can write 
the analog of Eq. 3 as 
where /', and ,, are the eigenfimctious and associat, ed eigenvalues of (-V'-") on 
the region of active spills. The key point here is that the spill aut, o-correlation 
function ill time deterlllilles the spectrunl of the Laplacian on the region of act, ivity. 
But, from the classic work of Kac [8] we know that this spectrum gives a great 
deal of infornlat,ion about the size and shape of the act,ive region -- we call ill 
general determine the area, the length of the perilneter, and t,he topology (nunlber 
of holes) from the set, of eigenvalues {,,}, and this is true regardless of the absolute 
dimensions of the region. Thus by operating at a critical point we can achieve 
a scale-independent encoding of object. dimension and topology in the temporal 
correlations of a locally connected system. 
3 
MAPPING REAL NEURONS ONTO THE 
STATISTICAL MODEL 
All current nlodels of neural networks are based on the hope that. most. microscopic 
("biological") details are unilnportant. for the macroscopic, collective conlputational 
behavior of t. he system as a whole. Here we provide a rigorous comlection between a 
more realistic neural nlodel and a simplified lnodel with spin variables and effective 
int. eractions, essentially the XY model discussed above. A more detailed account is 
given in [2,3]. 
We use the Fitzhugh-Nagumo (FN) model [9,10] to describe the electrical dynanlics 
of an individual neuron. This model denlonstrates a threshold for firing action 
potentials, a refractory period, a.nd single-shot as well as repetitive firing --- in 
short., all the qualitative properties of neural firing. It is also known to provide a 
reasonable quaut.itative description of several cell types. To be realistic it. is essential 
to add a noise current 5l,(t) which we t.ake to be Gaussian, spectrally white. and 
independent in each cell ,. 
We COlmect each neuron to its neighbors ill regular one- and two-dilnensional arrays. 
More general local connections are easily added and do not significantly change the 
restilts presented below. We lnodel a synapse between two neurons by exponenti- 
at. ing the volla.ge from one and injecting it. as current into the other. Our choice is 
motivated by the fact t. hat the nnnlber of translnitter vesicles released at a synapse 
is exponential ill the presynaptic voltage Ill]; other synaptic transfer character- 
istics, including sinall delays, give results qualitatively similar to those described 
140 Kruglyak and Bialek 
liere. The resulting equations of motion are 
dt 
dW. 
dt 
(1/T1) 
Io + 6I,,(1) - t](I " - 1) - W. + E 
-- (1/r2)[V,, - 
(5) 
where V'. is the t, ransmelnbrane voltage in cell n, I0 is tile DC bias current. and the 
B' are auxiliary variables; V0 sets the scale of voltage sensitivity in the synapse. 
Voltages and currents are dinensionless, and the pal'ameters of the systeln are 
expressed in terlns of the time constants rl and D and a dimensionless ratio 
Froin the voltage traces we extract the spike arrival times in tile n th neuron, 
With the appropriate clioice of parameters the FN lnodel can be made to fire 
regularly--the interspike intervals are tightly chlstered around a incan value. The 
power spectrum of the spike train s(t) 5-4 6(1 - ti) has well resolved peaks at +w0. 
+2w0 .... We then low-pass filter s(t) to keep only the +w0 peaks. obtailfing a 
phase-nodulated cosine, 
(6) 
where IFs](t) denotes the filtered spike train. By looking at [Fs](t) and its tilne 
derivative, we can extract the phase (t) which describes the oscillation that un- 
derlies regular firing. Since the orientation of a planar spin is also described by a 
single phase variable, we can reduce the spike train to a time-dependent planar spin 
S(t). We now want. to see how these spins interact when we connect two cells via 
synapses. 
We characterize the two-neuron interaction by accumulating a histogram of the 
phase differences between two connected neurons. This probability distribntion 
deftnes an effective Hamiltonian, P(,_.,) x exp[-H(- 2)]. With excitatory 
synapses (J > 0) the interaction is ferromagnetic, as expected (see Fig. 1). The 
Hamiltonian takes other interesting forms for inhibitory, delayed, and nonreciprocal 
synapses. By simulating sinall clusters of cells we find that interactions other than 
nearest neighbor are negligible. This leads us to predict that the entire network is 
described by the effective Hamiltonian H = Y]ij Hij(q 3i -3j), where Hij(q3 i --qSj) 
is the effective Halniltonian measured for the pair of connected cells i, j. 
One crucial consequence of Eq. 6 is that correlatiols of the filtered spike trains 
are exactly proportional to the spin-spin correlations which are natural objects in 
statistical lnechanics. Specifically, if we have two cells n and m, 
(S.,. Sin) = (COS(C,- ,,)) = w '2 ([Fs,(t)[rs,](t)). 
(7) 
This relation shows us how the statistical description of the network can be tested 
in experiments which monitor actual neural spike trains. 
4 DOES THE MAPPING WORK? 
When planar spins are connected in a one-dimensional chain with nearest-neighbor 
interactions, correlations between spins drop off exponentially with distalice. To test 
Analog Computation at a Critical Point 141 
this predictiou we have run simulations on chains of 32 Fitzhugh-Nagumo neurons 
connected to their nearest neighbors. Correlations computed directly from the 
filtered spike trains a.s indicated above indeed decay exponentially. as seen in the 
insert to Fig. 1. Fig. 1 shows that the predictions for the correlation length froIn 
the simple model are in excellent agreelnent with the correlation lengths observed 
in the simulations of spiking neurons; there are no free parmneters. 
50 r-- 
40 
30 
2.0 
O0    t 
0.0 1.0 2.0 3.0 4.0 5.0 
Predated Cowrelatiogt Length 
Figure 1: Correlation length obtained froin fits to the simulation data vs. correlation 
length predicted from the Hamiltonians. Inset, upper left: Correlation function rs. 
distance from simulations, with exponential fit. Inset, lower right.: Corresponding 
Hamiltonian as a function of phase difference. 
In the two-dimensional case we connect each neuron to its four nearest neighbors 
on a square lattice. The corresponding spin model is essentially the XY mode. 
Hence we expect. a low-temperature (high synaptic strength) phase with correla- 
tions that decay slowly (as a small power of distance) and a high-temperature (low 
synaptic strength) disordered phase with exponential decay. These predictions were 
confirmed by large-scale simulations of two-dimensional arrays [2]. 
OBJECT DIMENSIONS FROM TEMPORAL 
CORRELATIONS 
We believe that we have presented convincing evidence for the description of regu- 
larly firing neurons in terms of XY spins, at least as regards their static or equilib- 
rium correlations. In our theoretical discussion we showed that the temporal corre- 
lation functions of XY spins in the algebraic phase contained information about the 
142 Kruglyak and Bialek 
lxl 
0.0 00.0 I000.0 I00.0 
Figure 2: Auto-correlation functions for the spike trains of single cells at the center 
of square arrays of different sizes. 
dimensions of "objects." Here we test this idea in a very sinple numerical exper- 
iment. Imagine that we have an array of N xN connected cells which are excited 
by incoming signals so that. they are in the oscillatory regime. Obviously we can 
measure the size of this "object" by looking at the entire network, but. our theo- 
retical results suggest that one can sense these dimensions (N) using the temporal 
correlations in just one cell, most simply the cell in the center of the array. 
In Fig. 2 we show tile auto-correlation functions for the spike trains of the center 
cell in arrays of different dimensions. It is clear that changing tile dimensions 
of the array of active cells has profound effects on these spatiaily local temporal 
correlations. Because of the fact that the model is on a critical line these correlations 
continue to change as tile dimensions of the array increase, rather than saturating 
after some finite correlation length is reached. Qualitatively similar results are 
expected throughout the algebraic phase of the associated spin model. 
Recently it has been shown that when cells in the cat visual cortex are excited by 
appropriate stimuli they enter a regime of regular firing. These firing statistics are 
somewhat more complex than simulated here because there are a variable number 
of spikes per cycle, but we have reproduced all of our major results in models which 
capture this feature of the real data. We have seen that networks of regularly 
firing cells are capable of qualitatively different types of computation because these 
networks can be placed at a critical point without fine tuning of parameters. Most 
dramatically dynamic scaling allows us to trade spatial and temporal features and 
thereby encode object dimension in temporal correlations of single cells, as in Fig. 
2. To see if such novel computations are indeed mediated by cortical oscillations 
Analog Computation at a Critical Point 143 
we suggest. the direct analog of our numerical experiment, in which the correlation 
finctions of single cells would be monitored in response to structured stimuli (e.g., 
textures) with different total spatial extent. in the two dimensions of the visual 
field. We predict that these correlation fimctions will show a clear dependence on 
the area of the visual field being excited, with some sensit. iviW to the shape and 
topology as well. Most importantly this dependence on "object" dimension will 
extend to very large objects because the network is at a critical point. In this sense 
the temporal correlations of single cells will encode any object dimension, rather 
than being detectors for objects of some critical size. 
Acknowledgements 
Ve thank O. Alvarez, D. Arovas, A. B. Bonds, K. Brueckner. M. Crair, E. 
Knobloch, H. Lecar, and D. Rohksar for helpful discussions. Work at Berkeley 
wa.s supported in part by the National Science Foundation through a Presidential 
Young Investigator Award (to W.B.), supplelnented by funds from Cray Research, 
Sun Microsystelns, and the NEC Research Institute, by the Fannie and John Hertz 
Foundation through a Graduate Fellowship (t.o L.K.), and by the USPHS through 
a Biomedical Research Support Grant. 
References 
[1] J. Alhnan, F. Meizin, and E. McGuiness. Ann. Rev. Neurosci., 8:407, 1985. 
[2] L. Kruglyak. From biological reality to simple physical models: Networks of 
oscillating neurons and the XY model. PhD thesis, University of California at 
Berkeley, Berkeley, California, 1990. 
[3] W. Bialek. In E. Jen, editor, 1989 Lectures i Complex Systems, SFI Stud- 
ies in the Sciences of Complexity, vohlme 2, pages 513-595. Addison-Vesley, 
Reading, glass., 1990. 
[4] R. Eckhorn, R. Bauer, W. Jordan, M. Brosch, W. Kruse, M. Munk, and H. J. 
Reitboeck. Biol. Cybern., 60:121, 1988. 
[5] C. M. Gray and W. Singer. Proc. Nat. Acad. Sci. USA, 86:1698, 1989. 
[6] C. M. Gray, P. KSnig, A. K. Engel, and W. Singer. Nature, 338:334, 1989. 
[7] D. R. Nelson. In C. Domb and J. L. Lebowitz, editors, Phase Transitions and 
Critical Phenomena, volume 7, chapter 1. Aca.demic Press, London, 1983. 
[8] M. Kac. The American Mathematical Mo,thly, 73:1-23, 1966. 
[9] Richard Fitzhugh. Biophysical Journal, 1:44.5-466, 1961. 
[10] J. S. Nagumo, S. Arimoto, and S. Yoshizawa. Proc. I. R. E., 50:2061, 1962. 
[11] D. J. Aidley. The Physiology of Excitable Cells. Cambridge University Press, 
Cambridge, 1971. 
Part IV 
Temporal Reasoning 
