584 
PHASOR NEURAL NETWORKS 
Andr J. Noest, N.I.B.R., NL-1105 AZ Amsterdam, The Netherlands. 
ABSTRACT 
A novel network type is introduced which uses unit-length 2-vectors 
for local variables. As an example of its applications, associative 
memory nets are defined and their performance analyzed. Real systems 
corresponding to such 'phasor' models can be e.g. (neuro)biological 
networks of limit-cycle oscillators or optical resonators that have 
a hologram in their feedback path. 
INTRODUCTION 
Most neural network models use either binary local variables or 
scalars combined with sigmoidal nonlinearities. Rather awkward coding 
schemes have to be invoked if one wants to maintain linear relations 
between the local signals being processed in e.g. associative memory 
networks, since the nonlinearities necessary for any nontrivial 
computation act directly on the range of values assumed by the local 
variables. In addition, there is the problem of representing signals 
that take values from a space with a different topology, e.g. that 
of the circle, sphere, torus, etc. Practical examples of such a 
signal are the orientations of edges or the directions of local optic 
flow in images, or the phase of a set of (sound or EM) waves as they 
arrive on an array of detectors. Apart from the fact that 'circular' 
signals occur in technical as well as biological systems, there are 
indications that some parts of the brain (e.g. olfactory bulb, cf. 
Dr.B.Baird's contribution to these proceedings) can use limit-cycle 
oscillators formed by local feedback circuits as functional building 
blocks, even for signals without circular symmetry. With respect to 
technical implementations, I had speculated before the conference 
whether it could be useful to code information in the phase of the 
beams of optical neurocomputers, avoiding slow optical switching 
elements and using only (saturating) optical amplification and a 
 American Institute of Physics 1988 
585 
hologram encoding the (complex) 'synaptic' weight factors. At the 
conference, I learnt that Prof. Dana Anderson had independently 
developed an optical device (cf. these proceedings) that basically 
works this way, at least in the slow-evolution limit of the dynamic 
hologram. Hopefully, some of the theory that I present here can be 
applied to his experiment. In turn, such implementations call for 
interesting extensions of the present models. 
BASIC ELEMENTS OF GENERAL PHASOR NETWORKS 
Here I study the perhaps simplest non-scalar network by using unit- 
length 2-vectors (phasors) as continuous local variables. The signals 
processed by the network are represented in the relative phaseangles. 
Thus, the nonlinearities (unit-length 'clipping') act orthogonally to 
the range of the variables coding the information. The behavior of 
the network is invariant under any rigid rotation of the complete set 
of phasors, representing an arbitrary choice of a global reference 
phase. Statistical physicists will recognize the phasor model as a 
generalization of 02-spin models to include vector-valued couplings. 
All 2-vectors are treated algebraically as complex numbers, writing 
Ixl for the length, /x/ for the phase-angle, and  for the complex 
conjugate of a 2-vector x. 
A phasor network then consists of N>>I phasors s. , with Isil=!, 
interacting via couplings cij , with cii = 0. The clj are allowed 
to be complex-valued quantities. For optical implementations this 
is clearly a natural choice, but it may seem less so for biological 
systems. However, if the coupling between two limitcycle oscillators 
with frequency f is mediated via a path having propagationdelay d, 
then that coupling in fact acquires a phaseshift of f.d.2radians. 
Thus, complex couplings can represent such systems more faithfully 
than the usual models which neglect propagationdelays altogether. 
Only 2-point couplings are treated here, but multi-point couplings 
Cijk, etc., can be treated similarly. 
The dynamics of each phasor depends only on its local field 
1 .. sj + n. where z is the number of inputs 
hi=  cij  ' 
586 
c..0 per cell and n. is a local noise term (complex and Gaussian). 
Various dynamics are possible, and yield largely similar results: 
Continuous-time, parallel evolution: ("type A") 
d (/si/) = Ihil.sin(/h./ - /s./) 
Discrete-time updating: si(t+t)= h./ Ihil either serially in 
1 ' 
random i-sequence ("type B"), or in parallel for all i ("type C"). 
The natural time scale for type-B dynamics is obtained by scaling 
the discrete time-interval t as,,,1/N ; type-C dynamics has t=l. 
LYAPUNOV FUNCTION (alias "ENERGY", or "HAMILTONIAN" ) 
If one limits the attention temporarily to purely deterministic 
(ni=0) models, then the question suggests itself whether a class of 
couplings exists for which one can easily find a Lyapunov function 
i.e. a function of the network variables that is monotonic under the 
dynamics. A well-known example 1 is the 'energy' of the binary and 
scalar Hopfield models with symmetric interactions. It turns out that 
a very similar function exists for phasor networks with type-A or B 
dynamics and a Hermitian matrix of couplings. 
= hi = cij sj 
x i,j 
Hermiticity (clj=jl) makes H real-valued and non-increasing in time. 
This can be shown as follows, e.g. for the serial dynamics (type B). 
Suppose, without loss of generality, that phasor i=l is updated. 
Then -z H = z ! h 1 + Zi cil S 1 + EE i cij sj 
x>l i,j>l 
= z 1 hl + Sl'ZCil i + constant. 
i>1 
With Hermitian couplings, H becomes real-valued, and one also has 
..Cil ' = i>lli ' = z  1 . 
x>l x x 
Thus, - H - constant = 1 hl + Sl 1 = 2 Re(s 1 1) . 
Clearly, H is minimized with respect to s 1 by Sl(t+l) = hl/Ihll - 
Type-A dynamics has the same Lyapunovian, but type C is more complex. 
The existence of Hermitian interactions and the corresponding energy 
function simplifies greatly the understanding and design of phasor 
networks, although non-Hermitian networks can still have a Lyapunov- 
587 
function, and even networks for which such a function is not readily 
found can be useful, as will be illustrated later. 
AN APPLICATION : ASSOCIATIVE MEMORY. 
A large class of collective computations, such as optimisations 
and content-addressable memory, can be realised with networks having 
an energy function. The basic idea is to define the relevant penalty 
function over the solution-space in the form of the generic 'energy' 
of the net, and simply let the network relax to minima of this energy. 
As a simple example, consider an associative memory built within the 
framework of Hermitian phasor networks. 
In order to store a set of patterns in the network, i.e. to make 
a set of special states (at least approximatively) into attractive 
fixed points of the dynamics, one needs to choose an appropriate 
set of couplings. One particularly simple way of doing this is via 
the phasor-analog of "Hebb's rule" (note the Hermiticity) 
c.. =  s(k) (k! where s(k)is phasor i in learned pattern k. 
3 k  3 
The rule is understood to apply only to the input-sets i of each i. 
Such couplings should be realisable as holograms in optical networks, 
but they may seem unrealistic in the context of biological networks 
of oscillators since the phase-shift (e.g. corresponding to a delay) 
of a connection may not be changeable at will. However, the required 
coupling can still be implemented naturally if e.g. a few paths with 
different fixed delays exist between pairs of cells. The synaps in 
each path then simply becomes the projection of the complex coupling 
on the direction given by the phase of its path, i.e. it is just a 
classical Hebb-synapse that computes the correlation of its pre- and 
post-synaptic (imposed) signals, which now are phase-shifted versions 
of the phasors s!.k)The required complex c.. are then realised as the 
1 13 
vector sum over at least two signals arriving via distinct paths with 
corresponding phase-shift and real-valued synaps. Two paths suffice 
if they have orthogonal phase-shifts, but random phases will do as 
well if there are a reasonable number of paths. 
We need to have a concise way of expressing how 'near' any state 
of the net is to one or more of the stored patterns. A natural way 
588 
of doing this is via a set of p order parameters called "overlaps" 
N 
1 (k){ ; 0 < M k < 1 ; 1 < k < p . 
Mk= Isi. i .... 
i , 
P 2 
Note the constraint on the p overlaps M k 5 1 if all the patterns 
k 
are orthogonal, or merely random in the limit N-OO. This will be 
assumed from now on. Also, one sees at once that the whole behaviour 
of the network does not depend on aSy rigid rotation of all phasors 
over some angle since H, Mk, cij and the dynamics are invariant under 
multiplication of all s. by a fixed phasor : s = S.s. with ISI=i. 
1 1 1 
Let us find the performance at low loading: N,p,z-oo, with p/z-0 
and zero local noise. Also assume an initial overlap m>0 with only 
one pattern, say with k=l. Then the local field is 
1 j  s(k)(k) h!l)+ h* 
= s.. i ' j = 1 i , where 
hi  i 3 k 
h(1) 1 s(l! - !l!s j = ml s(l!s + 0(1//) with sf(i);Isl=l 
i =  i j-i 3 ' i ' ' 
* Z : o< 
and h = s . . 
i  ji 3 3 
Thus, perfect recall (Mi=I) occurs in one 'pass' at loadings p/zO. 
EXACTLY SOLVABLE CASE: SPARSE and ASYMMETRIC couplings 
Although it would be interesting to develop the full thermodynamics 
of Hermitian phasor networks with p and z of order N (analogous to the 
analysis of the finite-T Hopfield model by the teams of Amir 2 and van 
Hemmen3), I will analyse here instead a model with sparse, asymmetric 
connectivity, which has the great advantages of being exactly solvable 
with relative ease, and of being arguably more realistic biologically 
and more easily scalable technologically. In neurobiological networks 
a cell has up to zg104 asymmetric connections, whereas Ng101 This 
probably has the same reason as applies to most VLSI chips, namely to 
alleviate wiring problems. For my present purposes, the theoretical 
advantage of getting some exact results is of primary interest 4. 
Suppose each cell has z incoming connections from randomly selected 
t 
other cells. The state of each cell at time t depends on at most z 
cells at time t=0. Thus, if zt<< N1/2and N large, then the respective 
589 
4 
trees of 'ancestors' of any pair cells have no cells in common . In 
particular, if z~ (logN) x, for any finite x, then there are no common 
ancestors for any finite time t in the limit N-O. For fundamental 
information-theoretic reasons, one can hope to be able to store p 
patterns with p at most of order z for any sort of 2-point couplings. 
Important questions to be settled are: What are the accuracy and 
speed of the recall process, and how large are the basins of the 
attractors representing recalled patterns? 
Take again initial conditions (t=0) with, say, m(t)= M 1 > M>i = 0. 
Allowing again local random Gaussian (complex) noise n., the local 
fields become, in now familiar notation, h.= h!l)+ h  n.. 
1 1 1 1 
As in the previous section, the h(1)term consists of the 'signal' 
1 
m(t).s i (modulo the rigid rotation S) and a random term of variance 
, 
at most 1/z. For p ~ z, the h. term becomes important. Being sums of 
1 , 
z(p-1) phasors oriented randomly relative to the signal, the h. are 
independent Gaussian zero-mean 2-vectors with variance (p-1)/z , as 
2 
p,z and N- . Finally, let the local noises n. have variance r . 
Then the distribution of the si(t+l ) phasors can be found in terms of 
the signal m(t) and the total variance a=(p/z)+r2of the random h+n.. 
1 1 
After somewhat tedious algebraic manipulations (to be reorted in 
detail elsewhere) one obtains the dynamic behaviour of m(t) : 
m(t+l) = F(m(t),a) for discrete parallel (type-C) dynamics, 
and 
d m(t) = F(m(t),a) - m(t) for type-A or type-B dynamics , 
where the function F(m,a) = 
m , fdx. (l+cos2x).exp[-(m.sinx)2/a]. (l+erf[ (m.cosx)/4) 
* * 
The attractive fixed points M (a)= F(M ,a) represent the retrieval 
accuracy when the loading-plus-noise factor equals a. See figure 1. 
* a 3 ) 
For a<<l one obtains the expansion 1-M (a) = a/4 + 3a2/32 + O(  
* 1/2at 
The recall solutions vanish continuously as M ~(ac-a) a =/4. 
c 
One also obtains (at any t) the distribution of the phase scatter of 
the phasors around the ideal values occurring in the stored pattern. 
59O 
where 
P(/ui/) = (1/2R).exp(-m2/a).(l+/.L.exp(L2).(l+erf(L)) 
cos(/ui/ s. !k)(modulo S). 
L = (m//). ) , and ui= 
Useful approximations for the high, respectively low M regimes are: 
 /)2/a] 
 >>,J': P(/ui/) = (M/,/ exp[-(M./ui 
; I/ui/I 
M <<t': p(/ui/) = (1/2).(l+L.v') 
Figure 1. 
RETRIEVAL-ERROR and BASIN OF ATTRACTION versus LOADING + NOISE. 
o'. oo 
0.10 
0.20 
0.30 
0. qo o. ;o o. 60 
a = p/z + 
0 . 0 0 . 80 0 . 90 1 . 00 
591 
DISCUSSION 
It has been shown that the usual binary or scalar neural networks 
can be generalized to phasor networks, and that the general structure 
of the theoretical analysis for their use as associative memories can 
be extended accordingly. This suggests that many of the other useful 
applications of neural nets (back-prop, etcO can also be generalized 
to a phasor setting. This may be of interest both from the point of 
view of solving problems naturally posed in such a setting, as well 
as from that of enabling a wider range of physical implementations, 
such as networks of limit-cycle oscillators, phase-encoded optics, 
or maybe even Josephson-junctions. 
The performance of phasor networks turns out to be roughly similar 
to that of the scalar systems; the maximum capacity p/z=/4 for 
phasor nets is slightly larger than its value 2/ for binary nets, 
but there is a seemingly faster growth of the recall error 1-M at 
small a (linear for phasors, against exp(-1/(2a)) for binary nets). 
However, the latter measures cannot be compared directly since they 
stem from quite different order parameters. If one reduces recalled 
phasor patterns to binary information, performance is again similar. 
Finally, the present methods and results suggest several roads to 
further generalizations, some of which may be relevant with respect 
to natural or technical implementations. The first class of these 
involves local variables ranging over the k-sphere with k>l. The 
other generalizations involve breaking the O(n) (here n=2) symmetry 
of the system, either by forcing the variables to discrete positions 
on the circle (k-sphere), and/or by taking the interactions between 
two variables to be a more general function of the angular distance 
between them. Such models are now under development. 
REFERENCES 
1. J.J.Hopfield, Proc. Nat.Acad. Sci.USA 79, 2554 (1982) and 
idem, Proc.Nat.Acad. Sci.USA 81, 3088 (1984). 
2. D.J.Amit, H.Gutfreund and H.Sompolinski, Ann. Phys. 173, 30 (1987). 
3. D.Grensing, R.Kuhn and J.L. van Hemmen, J.Phys.A 20, 2935 (1987). 
4. B.Derrida, E.Gardner and A.Zippelius, Europhys.Lett. 4, 167 (1987) 
