Parameterising Feature Sensitive Cell 
Formation in Linsker Networks in the 
Auditory System 
Lance C. Walton 
University of Kent at Canterbury 
Canterbury 
Kent 
England 
David L. Bisset 
University of Kent at Canterbury 
Canterbury 
Kent 
England 
Abstract 
This paper examines and extends the work of Linsker (1986) on 
self organising feature detectors. Linsker concentrates on the vi- 
sual processing system, but infers that the weak assumptions made 
will allow the model to be used in the processing of other sensory 
information. This claim is examined here, with special attention 
paid to the auditory system, where there is much lower connec- 
tivity and therefore more statistical variability. On-line training is 
utilised, to obtain an idea of training times. These are then com- 
pared to the time available to pre-natal mammals for the formation 
of feature sensitive cells. 
I INTRODUCTION 
Within the last thirty years, a great deal of research has been carried out in an 
attempt to understand the development of cells in the pathways between the sensory 
apparatus and the cortex in mammals. For example, theories for the development of 
feature detectors were forwarded by Nass and Cooper (1975), by Grossberg (1976) 
and more recently Obermayer et al (1990). 
Hubel and Wiesel (1961) established the existence of several different types of fea- 
ture sensitive cell in the visual cortex of cats. Various subsequent experiments have 
1007 
1008 Walton and Bisset 
shown that a considerable amount of development takes place before birth (i.e. 
without environmental input). This must either be dependent on a genetic pre- 
dispostion for individual cells to develop in an appropriate way without external 
influence, or some low level rules sufficient to create the required cell morphologies 
in the presence of random action potentials. 
Although there is a great deal of a priori information concerning axon growth 
and synapse arborisation (governed by chemical means in the brain), it is difficult 
to conceive of a biological system that could use genetic information to directly 
manipulate the spatial information about the pre-synaptic target with respect to 
the axon with which the synapse is made. However, there is considerable random 
activity in the sensory apparatus that could be used to effect synaptic development. 
Various authors have constructed models that deal with different aspects of self- 
organisation of this kind and some have pointed out the value of these types of 
cells in pattern classification problems (Grossberg 1976), but either the biological 
plausibility of these models is questionable, or the subject of pre-natal development 
is not addressed (i.e. without environmental input). 
In this paper, the networks of Linsker (1986) will be examined. Although these 
networks have been analysed quite extensively by Linsker, and also by Mackay and 
Miller (1990), the biological aspects of parameter ranges and choices have only 
been touched upon. It is our aim in this paper, to add further detail in this area 
by examining the one-dimensional case which represents the auditory pathways. 
2 LINSKER NETWORKS 
The network is based on a Multi Layer Perceptron, with feed forward connections in 
all layers, and lateral connections (inhibition and excitation) in higher layers. The 
neural outputs are sums of the weighted inputs, and the weights develop according 
to a constrained Hebbian Rule. Each layer is lettered for reference starting from A 
and subsequent layers are lettered B,C,D etc. The superscript M will be used to 
refer to an arbitrary layer, and L is used to refer to the previous layer. Each layer has 
a set of parameters which are the same for all neurons in that layer. Connectivity 
is random but is based on a Gaussian density distribution 2 2 
(exp(-r /rm)), where 
rm is the arbor radius for layer M. 
Each layer is a rectangular array of neurons (or vector of neurons for the one di- 
mensional case). The layers are assumed to be large enough so that edge effects 
are not important or do not occur. Layers develop one at a time starting from the 
B layer. The A layer is an input layer, which is divided into boxes, within each of 
which activity is uniform. This is biologically realistic, since sensory neurons fan 
out to a number of cells (an average of 10 in the cochlea) each of which only take 
input from one sensory cell. Hence the input layer for the network acts like a layer 
of tonotopically organised neurons. 
Parameterising Feature Sensitive Cell Formation in Linsker Networks in the Auditory System 1009 
3 NETWORK DEVELOPMENT 
The output of a neuron in layer M is given by 
Lr 
Where, 
r indexes a pattern presentation, 
The subscript n is used to index the M layer neurons, 
Ra, Rb are layer parameters, 
F ' is the output of the L layer neuron which is pre-synaptic to the j'th input 
of the n'th M layer neuron. 
The synaptic weights develop according to a constrained Hebbian learning rule, 
(/Xc,.) + - 
= --  0 I'kpre(ni) 
Where, 
(Aci)  is the change in the i'th weight of neuron n, 
k, k, F , F are layer parameters. 
Synaptic weights are constrained to lie within the ra,nge (n - 1,n,). (In this 
work, n, = 0.5) 
Linsker (1986a) derives an Ensemble Averaged Development equation which shows 
how development depends on the parameters, and how correlations develop between 
spatially proximate neurons in layers beyond the first. In so doing, the number of 
parameters is reduced kom five per layer o two per layer, and therefore the equation 
is a very useful aid in understanding the self-organising nature of this model. The 
development equation is 
 Qpre(ni).pre(nj).cnJ 
ci = K + K'z.d + j  NM (3) 
f (4) 
Where, 
NM is the number of synaptic connections to an M layer hem'on, 
ff is the average output activity in the L layer, 
+ - - 
Z = Nkf (5) 
J is a unit of activity used to normalise the two point correlation function Q.. In 
this work f is chosen to set Qiq: = 1 
Angie brackets denote an average taken over the ensemble of input patterns. 
1010 Walton and Bisset 
4 MORPHOLOGICAL REGIMES 
From equation 3, an expression can be found for the average weight value 6 in a 
layer, and therefore certain properties of the system can be described. Although 
Mackay and Miller (1990) have described the regimes with the aid of eigenvalues 
and eigenfunctions, there is a much simpler method which will provide the same 
information. 
For an all-excitatory (AE) layer, the average weight value is equal to nero. Since 
all weights are equal to nero, the summation in equation 3 can be re-written 
hera j QpLre(ni).pre(n j -- nera.NM., where q- 2.No. 
 ) ' 
A similar expression can be found for all-inhibitory (AI) layers, and therefore the 
Ki - K2 plane can be sub-divided into three regions which will yield AE cells, AI 
cells, and mixed-mode cells (see figure 1). 
The plane can be divided further for the mixed-mode cell type in the C layer. On- 
center and off-center cells develop close to the AE and AI boundaries respectively. 
Mackay and Miller have shown why these cells develop and have placed a theoretical 
lower bound on b which agrees with experiinental data. However, in so doing the 
effect of the intercept on the K2 axis was deemed small, due to a large number 
of synaptic connections. This approximation depends upon the large number of 
connections between the B and C layers. In the auditory case, the number of 
connections is smaller, and it is possible that this assumption no longer holds. 
From equation 3, it can be seen that movement into the On-Centre region from 
the AE region, causes the va. lue of 
(pre(ni).pre(nj).Cnj to decrease. This has the 
effect of moving the intercept of the constant 6 line from K = q towards K2 = 0. 
Ka finally reaches 0 when 6 = 0, and then begins to move ba.ck towards (] as the AI 
regime is approached. 
This has two potentially important effects. Firstly, it means that the tolerance of 
K2 varies with/(1; for a particular value of/(1, there are upper and lower limits on 
the value of/f2 which will allow maturation of on-center cells. This range of values 
(i.e. the difference between the limits) varies in a linear way with K, but the ratio 
of the range to a value of K2 which is within the range (i.e. the center value) is not 
linear with K. Here, tolerance is defined as that ratio. Secondly, there is a region 
of negative I(.,where the nature of the cell morphology which will be produced is 
unknown. 
It is therefore important that K should be larger than this value in order to produce 
On-Center or Off-Center cells reliably. Mackay and Miller use ]K2] --* oo in their 
analysis. Unfortunately, this would require the fundamental network parameter 
F0  -. oo from equation 6, and therefore is an unsuitable choice. It is reasonable 
to assume that F0  is of the same order as fir, and hence an order for K2 can be 
established. For a concrete example, assume inputs are binary (giving Q -- 0.25) 
and F0  = ff x 1.2, this will ensure K ( 0 (equation 6) while adhering to the 
assumption made above. Equation 6 now gives the order for K = 0.2. 
To find the value of r], which will place a lower bound on IK2], a particular system 
should be chosen. The auditory system is chosen here. 
Parameterising Feature Sensitive Cell Formation in Linsker Networks in the Auditory System 1011 
-K2 
Figure 1: Graph of Morphological Regions for C Layer 
There are approximately 3000 inner hair cells in the cochlea, each of which fans out 
to an average of 10 neurons (which sets our box size p = 10). These neurons take 
input from only one hair cell. The anteroventral cochlea nucleus takes input from 
this layer of cells, with a fan in NB  50 (c.f. the value of NB = 1000 in Linsker 
(1986a)). The assumption is made that the three sections of the cochlea nucleus 
each contain approximately the same number of cells. With this smaller number of 
connections, the correlation function for this layer is somewhat coarser, and does 
not follow the theoretical curve for the continuum limit so well. 
In addition, the on-center cells found in the posteroventral cochlea nucleus and 
the dorsal nucleus have centres with a tuning curve response Q of about 2.5 which 
corresponds to about 2000 B layer cells. If it is assumed that the surround of the cell 
is half the width of the core, then there is a total rc , 3000 neurons. Simulations 
here use Nc = 100 which is a realistic number of connections in the context of a 
one-dimensional network. 
In general, the arbor radius increases as layers become closer to the cortex. From 
Linsker, rc/r = 3. r is therefore equal to 1000. This yields the average number 
of connections to a given B cell from a particular A box being approximately unity, 
which agrees well with the condition expressed by Linsker. 
Using the expression above, rj can be calculated as approximately 1.5 x 10 -a. This 
value is certainly insignificant with respect to the value of K2 = 0.2 quoted earlier, 
and therefore any effects due to the summation term in equation 3 can be ignored 
in the calculation of  for this system. This means that the original approximation 
still holds even in this low connectivity case. 
1012 Walton and Bisset 
5 SIMULATION RESULTS 
A network was trained using the connectivity stated above to give various values of 
5 with K2 = 0.2. To obtain an idea. of the total number of presentations that were 
required to train the network, without any artifacts that might be produced as a 
result of batch training, the original network equations were used. In all of these 
simulations, Ra, F0 M = 0 so that the value of K could be easily controlled. 
The findings were that the maximum value of kb was about 10 -3 which required 
2.5 million pattern presentations to mature the network. With this value, on-center 
cells with an average weight value less than about 0.3 would not mature. However 
as the value of kb was decreased (keeping K constant), the value of 5 could be 
made lower, at the expense of more pattern presentations. The figures obtained 
for the maturation of feature sensitive cells are extremely biologically realistic in 
the light of the number of pattern presentations available to an average mammal. 
For example, the foetal cat has sufficient time for about 25 million presentations 
(assuming 10 presentations per second). 
6 CONCLUSION 
We have shown that the class of network developed by Linsker is extendable to 
the auditory system where the number and density of synapses is considerably 
smaller than in the visual case. It has also been shown that the time for layer 
maturation by this method is sufficiently short even for mammals with a relatively 
short gestation period, and therefore should also be sufficient in mammals with 
longer foetal development times. We conclude that the model is therefore a good 
representation of feature detector development in the pre-natal mammal. 
References 
Grossberg S. (1976) - On the Development of Feature Detectors in the Visual 
Cortex with Applications to Learning and Reaction Diffusion Systems, Biological 
Cybernetics 21, 145- 159 
Grossberg S. (1976)- Adaptive Pattern Classification and Universal Recoding: 1 
Parallel Development and Coding of Neural Feature Detectors, Biological Cybernet- 
ics 23, 121 - 134 
Hubel D. H. and Wiesel T. N. (1961) - Receptive Fields, Binocular Interaction and 
Functional Architechture in the Cat's Visual Cortex, Journal of Phlsiology, 150, 
106- 154 
Kalil R. E. (1989) - Synapse Formation In The Developing Brain, Scientific Amer- 
ican, December 989, 38- 45 
Klinke R. (1986) - Physiology of Hearing, In Schmidt R. W. (ed.), Fundamentals 
of Sensory Physiology, 199 - 223 
MacKay D. J. C. and Miller K. D. (1990) - Analysis of Linsker's Simulations of 
Hebbian Rules, Neural Computation, 2, 173- 187 
von der Malsburg C. (1979) - Development of Ocularity Domains and Growth 
Parameterising Feature Sensitive Cell Formation in Linsker Networks in the Auditory System 1013 
Behaviour of Axon Terminals, Biological Cybernetics, 32, 49 - 62 
Linsker R. (1986a) - From Basic Network Principles To Neural Architecture: Emer- 
gence Of Spatial-Opponent Cells, Proceedings of the National Academy of Sciences 
(USA), 83, 7508- 7512 
Linsker R. (1986b) - From Basic Network Principles To Neural Architecture : 
Emergence of Orientation-Selective Cells, Proceedings of the National Academy of 
Sciences (USA), 83, 8390- 8394 
Linsker R. (1986c) - From Basic Network Principles To Neural Architecture: Emer- 
gence of Orientation-Columns, Proceedings of the National Academy of Sciences 
(USA), 83, 8779- 8783 
Nass M. M. and Cooper L. N. (1975) - A Theory for the Development of Feature 
Detecting Cells in the Visual Cortex, Biological Cybernetics, i9, 1 - 18 
Obermayer K. Ritter H. and Schulten K. (1990) - Development and Spatial 
Structure of Cortical Feature Maps: A Model Study NIPS, $, 11 - 17 
Sloman A. (1989) - On Designing a Visual System (Towards a Gibsonian Compu- 
tational Model of Vision) Journal of Experimental and Theoretical Artificial Intel- 
ligence, 1, 289- 337 
Tanaka S. (1990) - Interaction among Ocularity, Retinotopy and On-Center/Off 
Center Pathways During Development NIPS, 3, 18 - 25 
