Neural Network Routing for Random Multistage 
Interconnection Networks 
Mark W. Goudreau 
Princeton University 
and 
NEC Research Institute, Inc. 
4 Independence Way 
Princeton, NJ 08540 
C. Lee Giles 
NEC Research Institute, Inc. 
4 Independence Way 
Princeton, NJ 08540 
Abstract 
A routing scheme that uses a neural network has been developed that can 
aid in establishing point-to-point communication routes through multi- 
stage interconnection networks (MINs). The neural network is a network 
of the type that was examined by Hopfield (Hopfield, 1984 and 1985). 
In this work, the problem of establishing routes through random MINs 
(RMINs) in a shared-memory, distributed computing system is addressed. 
The performance of the neural network routing scheme is compared to two 
more traditional approaches - exhaustive search routing and greedy rout- 
ing. The results suggest that a neural network router may be competitive 
for certain RMINs. 
I INTRODUCTION 
A neural network has been developed that can aid in establishing point-to- 
point communication routes through multistage interconnection networks (MINs) 
(Goudreau and Giles, 1991). Such interconnection networks have been widely stud- 
ied (Huang, 1984; Siegel, 1990). The routing problem is of great interest due to 
its broad applicability. Although the neural network routing scheme can accommo- 
date many types of communication systems, this work concentrates on its use in a 
shared-memory, distributed computing system. 
Neural networks have sometimes been used to solve certain interconnection network 
722 
Neural Network Routing for Random Multistage Interconnection Networks 723 
Input Output 
Ports Ports 
t i Network i.._ 
I Neura' I I Interconnection 
Logicl  5,',,,o,-' -- Logic2 l: Network 
[_ _[7 ':' __= = == J Controller 
Externa{ Control 
Figure 1: The communication system with a neural network router. The input 
ports (processors) are on the left, while the output ports (memory modules) are on 
the right. 
problems, such as finding legal routes (Brown, 1989; Hakim and Meadows, 1990) 
and increasing the throughput of an interconnection network (Brown and Liu, 1990; 
Marrakchi and Troudet, 1989). The neural network router that is the subject of 
this work, however, differs significantly from these other routers and is specially 
designed to handle parallel processing systems that have MINs with random inter- 
stage connections. Such random MINs are called RMINs. RMINs tend to have 
greater fault-tolerance than regular MINs. 
The problem is to allow a set of processors to access a set of memory modules 
through the RMIN. A picture of the communication system with the neural network 
router is shown in Figure 1. The are m processors and n memory modules. The 
system is assumed to be synchronous. At the beginning of a message cycle, some 
set of processors may desire to access some set of memory modules. It is the 
job of the router to establish as many of these desired connections as possible in 
a non-conflicting manner. Obtaining the optimal solution is not critical. Stymied 
processors may attempt communication again during the subsequent message cycle. 
It is the combination of speed and the quality of the solution that is important. 
The object of this work was to discover if the neural network router could be compet- 
itive with other types of routers in terms of quality of solution, speed, and resource 
724 Goudreau and Giles 
RMIN1 
1 
? 
1 
RMIN3 
1 2 
1 
RMIN2 
1 
7 
$ 
3 4 
Figure 2: Three random multistage interconnection networks. The blocks that are 
shown are crossbar switches, for which each input may be connected to each output. 
utilization. To this end, the neural network routing scheme was compared to two 
other schemes for routing in RMINs - namely, exhaustive search routing and greedy 
routing. So far, the results of this investigation suggest that the neural network 
router may indeed be a practicable alternative for routing in RMINs that are not 
too large. 
2 EXHAUSTIVE SEARCH ROUTING 
The exhaustive search routing method is optimal in terms of the ability of the router 
to find the best solution. There are many ways to implement such a router. One 
approach is described here. 
For a given interconnection network, every route from each input to each output 
was stored in a database. (The RMINs that were used as test cases in this paper 
always had at least one route from each processor to each memory module.) When 
a new message cycle began and a new message set was presented to the router, 
the router would search through the database for a combination of routes for the 
message set that had no conflicts. A conflict was said to occur if more than one 
route in the set of routes used a single bus in the interconnection network. In the 
case where every combination of routes for the message set had a conflict, the router 
would find a combination of routes that could establish the largest possible number 
of desired connections. 
If there are k possible routes for each message, this algorithm needs a memory of 
size (rank) and, in the worst case, takes exponential time with respect to the size 
Neural Network Routing for Random Multistage Interconnection Networks 725 
of the message set. Consequently, it is an impractical approach for most RMINs, 
but it provides a convenient upper bound for the performance of other routers. 
3 GREEDY ROUTING 
When greedy routing is applied, message connections are established one at a time. 
Once a route is established in a given message cycle, it may not be removed. Greedy 
routing does not always provide the optimal routing solution. 
The greedy routing algorithm that was used required the same route database as 
the exhaustive search router did. However, it selects a combination of routes in 
the following manner. When a new message set is present, the router chooses one 
desired message and looks at the first route on that message's list of routes. The 
router then establishes that route. Next, the router examines a second message 
(assuming a second desired message was requested) and sees if one of the routes 
in the second message's route list can be established without conflicting with the 
already established first message. If such a route does exist, the router establishes 
that route and moves on to the next desired message. 
In the worst case, the speed of the greedy router is quadratic with respect to the 
size of the message set. 
4 NEURAL NETWORK ROUTING 
The focal point of the neural network router is a neural network of the type that 
was examined by Hopfield (Hopfield, 1984 and 1985). The problem of establishing 
a set of non-conflicting routes can be reduced to a constraint satisfaction problem. 
The structure of the neural network router is completely determined by the RMIN. 
When a new set of routes is desired, only certain bias currents in the network change. 
The neural network routing scheme also has certain fault-tolerant properties that 
will not be described here. 
The neural network calculates the routes by converging to a legal rouing array. A 
legal routing array is 3-dimensional. Therefore, each element of the routing array 
will have three indices. If element ai,5, is equal to 1 then message i is routed 
through output port k of stage j. We say ai,j,k and al,m,n are in the same row if 
i = l and k = n. They are in the same column if i = I and j = m. Finally, they are 
in the same rod if j = m and k = n. 
A legal routing array will satisfy the following three constraints: 
1. one and only one element in each column is equal to 1. 
2. the elements in successive columns that are equal to I represent output ports 
that can be connected in the interconnection network. 
3. no more than one element in each rod is equal to 1. 
The first restriction ensures that each message will be routed through one and 
only one output port at each stage of the interconnection network. The second 
restriction ensures that each message will be routed through a legal path in the 
726 Goudreau and Giles 
interconnection network. The third restriction ensures that any resource contention 
in the interconnection network is resolved. In other words, only one message can 
use a certain output port at a certain stage in the interconnection network. When 
all three of these constraints are met, the routing array will provide a legal route 
for each message in the message set. 
Like the routing array, the neural network router will naturally have a 3-dimensional 
structure. Each al,j,k of a routing array is represented by the output voltage of a 
neuron, ,j,k. At the beginning of a message cycle, the neurons have a random 
output voltage. If the neural network settles in one of the global minima, the 
problem will have been solved. 
A continuous time mode network was chosen. It was simulated digitally. The neural 
network has N neurons. The input to neuron i is ui, its input bias current is Ii, and 
its output is ld. The input ul is converted to the output  by a sigmoid function, 
g(x). Neuron i influences neuron j by a connection represented by i. Similarly, 
neuron j affects neuron i through connection qj. In order for the Liapunov function 
(Equation 5) to be constructed, T must equal Ti. We further assume that 5qi = 0. 
For the synchronous updating model, there is also a time constant, denoted by r. 
N 
= + + z, O) 
j=l 
,-= 
 = g(us) (3) 
1 
g(x) - 1 + e - (4) 
The equations which describe the output of a neuron i are: 
dui 
dt 
The equations above force the neural net into stable states that are the local minima 
of this approximate energy equation 
i N N N 
i=1 j=l i=1 
(5) 
For the neural network, the weights (5q's) are set, as are the bias currents (Ii's). 
It is the output voltages ('s) that vary to to minimize E. 
Let M be the number of messages in a message set, let $ be the number of stages 
in the RMIN, and let P be the number of ports per stage (P may be a function 
of the stage number). Below are the energy functions that implement the three 
constraints discussed above: 
A MS-1P P 
= 5 5] E 5] + Z; 
m=l =1 p=l i=1 
(6) 
Bs-! P 
2 
:1 p=l 
M M 
m=l /=1 
(7) 
Neural Network Routing for Random Multistage Interconnection Networks 727 
MS-1P P 
=  + + 
m=l s=l p=l d=l 
m=l I.s=2 p=l 
+ am,j)Vma,S +d(,5, j,fm)Vm,s-,S) 
./=1 
A, B, C, and D are arbitrary positive constants. 1 E and Ea handle the first 
constraint in the routing array. E4 deals with the second constraint. E2 ensures the 
third. From the equation for E4, the function d(sl, pl, p2) represents the "distance" 
between output port pl from stage sl - I and output port p2 from stage sl. If pl 
can connect to p2 through stage sl, then this distance may be set to zero. If pl 
and p2 are not connected through stage sl, then the distance may be set to one. 
Also, am is the source address of message m, while m is the destination address 
of message m. 
The entire energy function is: 
E = E + E=+ Es+ E4 
(lO) 
Solving for the connection and bias current values as shown in Equation 5 results 
in the following equations: 
T(ml,sl,pl),(m,,p) = -(A + C)Sml,m2Ss,s(1 -- 5pl,p) (11) 
-DSm,m2 [5,+,sd(s2, pl, p2) + 5, x,2+d(sl, p2, pl)] 
Im,,,r = C- O[5,,d(1,am,p) + a,,$_d(S,p, ftm)] (12) 
5i,S is a Kronecker delta (Sij = 1 when i = j, and 0 otherwise). 
Essentially, this approach is promising because the neural network is acting as a 
parallel computer. The hope is that the neural network will generate solutions much 
faster than conventional approaches for routing in RMINs. 
The neural network that is used here has the standard problem- namely, a global 
minimum is not always reached. But this is not a serious difficulty. Typically, 
when the globally minimal energy is not reached by the neural network, some of 
the desired routes will have been calculated while others will not have. Even a 
locally minimal solution may partially solve the routing problem. Consequently, 
this would seem to be a particularly encouraging type of application for this type 
of neural network. For this application, the traditional problem of not reaching 
the global minimum may not hurt the system's performance very much, while the 
expected speed of the neural network in calculating the solution will be a great 
asset. 
For the simulations, r = 1.0, A = (7 = D = 3.0, and B = 6.0. These values for A, B, 
(7, and D were chosen empirically. 
728 Goudreau and Giles 
Table 1: Routing results for the RMINs shown in Figure 2. The * entries were not 
calculated due to their computational complexity. 
RMIN1 RMIN2 RMIN3 
M E Eqr E,, E Er E.. Ee Egr E 
1 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 1.00 
2 1.86 1.83 1.87 1.97 1.97 1.98 1.99 1.88 1.94 
3 2.54 2.48 2.51 2.91 2.91 2.93 2.99 2.71 2.87 
4 3.08 2.98 2.98 3.80 3.79 3.80 3.94 3.49 3.72 
5 3.53 3.38 3.24 4.65 4.62 4.61 * 4.22 4.54 
6 3.89 3.67 3.45 5.44 5.39 5.36 * 4.90 5.23 
7 4.16 3.91 3.66 6.17 6.13 6.13 * 5.52 5.80 
8 4.33 4.10 3.78 6.86 6.82 6.80 * 6.10 6.06 
The neural network router uses a large number of neurons. If there are rn input 
ports, and rn output ports for each stage of the RMIN, an upper bound on the 
number of neurons needed is rn2S. Often, however, the number of neurons actually 
required is much smaller than this upper bound. 
It has been shown empirically that neural networks of the type used here can con- 
verge to a solution in essentially constant time. For example, this claim is made for 
the neural network described in (Takefuji and Lee, 1991), which is a slight variation 
of the model used here. 
5 SIMULATION RESULTS 
Figure 2 shows three RMINs that were examined. The routing results for the three 
routing schemes are shown in Table 1. Ee represents the expected number of 
messages to be routed using exhaustive search routing. Egr is for greedy routing 
while En,, is for neural network routing. These values are functions of the size 
of the message set, M. Only message sets that did not have obvious conflicts 
were examined. For example, no message set could have two processors trying to 
communicate to the same memory module. The table shows that, for at least these 
three RMINs, the three routing schemes produce solutions that are of similar virtue. 
In some cases, the neural network router appears to outperform the supposedly 
optimal exhaustive search router. That is because the Ees and Egr values were 
calculated by testing every message set of size M, while E,was calculated by 
testing 1,000 randomly generated message sets of size M. For the neural network 
router to appear to perform best, it must have gotten message sets that were easier 
to route than average. 
In general, the performance of the neural network router degenerates as the size of 
the RMIN increases. It is felt that the neural network router in its present form will 
not scale well for large RMINs. This is because other work has shown that large 
neural networks of the type used here have difficulty converging to a valid solution 
(Hopfield, 1985). 
Neural Network Routing for Random Multistage Interconnection Networks 729 
6 CONCLUSIONS 
The results show that there is not much difference, in terms of quality of solution, for 
the three routing methodologies working on these relatively small sample RMINs. 
The exhaustive search approach is clearly not a practical approach since it is too 
time consuming. But when considering the asymptotic analyses for these three 
methodologies one should keep in mind the performance degradation of the greedy 
router and the neural network router as the size of the RMIN increases. 
Greedy routing and neural network routing would appear to be valid approaches 
for RMINs of moderate size. But since asymptotic analysis has a very limited 
significance here, the best way to compare the speeds of these two routing schemes 
would be to build actual implementations. 
Since the neural network router essentially calculates the routes in parallel, it can 
reasonably be hoped that a fast, analog implementation for the neural network 
router may find solutions faster than the exhaustive search router and even the 
greedy router. Thus, the neural network router may be a viable alternative for 
RMINs that are not too large. 
References 
Brown, T. X., (1989), "Neural networks for switching," IEEE Commun. Mag., Vol. 
27, pp. 72-81, Nov. 1989. 
Brown, T. X. and Liu, K. H., (1990), "Neural network design of a banyan network 
controller," IEEE J. on Selected Areas of Comm., pp. 1428-1438, Oct. 1990. 
Goudreau, M. W. and Giles, C. L., (1991), "Neural network routing for multiple 
stage interconnection networks," Proc. IJCNN 91, Vol. II, p. A-885, July 1991. 
Hakim, N. Z. and Meadows, H. E., (1990), "A neural network approach to the setup 
of the Benes switch," in Infocorn 90, pp. 397-402. 
Hop field, J. J., (1984), "Neurons with graded response have collective computational 
properties like those of two-state neurons," Proc. Natl. Acad. Sci. USA, Vol. 81, 
pp. 3088-3092, May 1984. 
Hopfield, J. J., (1985), "Neural computation on decisions in optimization problems," 
Biol. Cybern., Vol. 52, pp. 141-152, 1985. 
Huang, K. and Briggs, F. A., (1984), Computer Architecture and Parallel Processing, 
McGraw-Hill, New York, 1984. 
Marrakchi, A.M. and Troudet, T., (1989), "A neural net arbitrator for large cross- 
bar packet-switches," IEEE Trans. on Circ. and Sys., Vol. 36, pp. 1039-1041, July 
1989. 
Siegel, H. J., (1990), Interconnection Networks for Large Scale Parallel Processing, 
McGraw-Hill, New York, 1990. 
Takefuji, Y. and Lee, K. C., (1991), "An artificial hysteresis binary neuron: a model 
suppressing the oscillatory behaviors of neural dynamics", Biological Cybernetics, 
Vol. 64, pp. 353-356, 1991. 
