Neural Control for Nonlinear Dynamic Systems 
Ssu-Hsin Yu 
Department of Mechanical Engineering 
Massachusetts Institute of Technology 
Cambridge, MA 02139 
Email: hsin@mit.edu 
Anuradha M. Annaswamy 
Department of Mechanical Engineering 
Massachusetts Institute of Technology 
Cambridge, MA 02139 
Email: aanna@ mit.edu 
Abstract 
A neural network based approach is presented for controlling two distinct 
types of nonlinear systems. The first corresponds to nonlinear systems 
with parametric uncertainties where the parameters occur nonlinearly. 
The second corresponds to systems for which stabilizing control struc- 
tures cannot be determined. The proposed neural controllers are shown 
to result in closed-loop system stability under certain conditions. 
1 INTRODUCTION 
The problem that we address here is the control of general nonlinear dynamic systems 
in the presence of uncertainties. Suppose the nonlinear dynamic system is described as 
5:= f(x, u, 0) , y = h(x, u, 0) where u denotes an external input, y is the output, x is the 
state, and 0 is the parameter which represents constant quantities in the system. The control 
objectives are to stabilize the system in the presence of disturbances and to ensure that 
reference trajectories can be tracked accurately, with minimum delay. While uncertainties 
can be classified in many different ways, we focus here on two scenarios. One occurs 
because the changes in the environment and operating conditions introduce uncertainties 
in the system parameter 0. As a result, control objectives such as regulation and tracking, 
which may be realizable using a continuous function u = ?(x, 0) cannot be achieved since 
0 is unknown. Another class of problems arises due to the complexity of the nonlinear 
function f. Even if 0, f and h can be precisely determined, the selection of an appropriate 
? that leads to stabilization and tracking cannot be made in general. In this paper, we 
present two methods based on neural networks which are shown to be applicable to both 
the above classes of problems. In both cases, we clearly outline the assumptions made, 
the requirements for adequate training of the neural network, and the class of engineering 
problems where the proposed methods are applicable. The proposed approach significantly 
expands the scope of neural controllers in relation to those proposed in (Narendra and 
Parthasarathy, 1990; Levin and Narendra, 1993; Sanner and S!otine, 1992; Jordan and 
Rumelhart, 1992). 
Neural Control for Nonlinear Dynamic Systems 1 O11 
The first class of problems we shall consider includes nonlinear systems with parametric 
uncertainties. The field of adaptive control has addressed such a problem, and over the 
past thirty years, many results have been derived pertaining to the control of both linear 
and nonlinear dynamic systems (Narendra and Annaswamy, 1989). A common assumption 
in almost all of the published work in this field is that the uncertain parameters occur 
linearly. In this paper, we consider the control of nonlinear dynamic systems with nonlinear 
parametrizations. We design a neural network based controller that adapts to the parameter 
8 and show that closed-loop system stability can be achieved under certain conditions. Such 
a controller will be referred to as a 8-adaptive neural controller. Pertinent results to this 
class are discussed in section 2. 
The second class of problems includes nonlinear systems, which despite being completely 
known, cannot be stabilized by conventional analytical techniques. The obvious method for 
stabilizing nonlinear systems is to resort to linearization and use linear control design meth- 
ods. This limits the scope of operation of the stabilizing controller. Feedback linearization 
is another method by which nonlinear systems can be stably controlled (Isidori, 1989). This 
however requires fairly stringent set of conditions to be satisfied by the functions f and h. 
Even after these conditions are satisfied, one cannot always find a closed-form solution to 
stabilize the system since it is equivalent to solving a set of partial differential equations. 
We consider in this paper, nonlinear systems, where system models as well as parameters 
are known, but controller structures are unknown. A neural network based controller is 
shown to exist and trained so that a stable closed-loop system is achieved. We denote this 
class of controllers as a stable neural controller. Pertinent results to this class are discussed 
in section 3. 
2 O-ADAPTIVE NEURAL CONTROLLER 
The focus of the nonlinear adaptive controller to be developed in this paper is on dynamic 
systems that can be written in the d-step ahead predictor form as follows: 
y+d = 0) (1) 
where 0.,t r --- [yt,'",yt-+,u,-,'.',ut-m-,+], n _> 1, m > O, d > 1, m + d = n, 
y, b/ C  containing the origin and  C k are open, f  Y x L/ +d x 1  , yt 
and ut are the output and the input of the system at time t respectively, and 8 is an unknown 
parameter and occurs nonlinearly in f. The goal is to choose a control input 'u such that 
the system in (1) is stabilized and the plant output is regulated around zero. 
LetxtT A [Ytq-d-l, Yt+l,Oj[]T, Am-'-[e2, ''' erq-d-l,0, erq-dq-1, ''' eq-mq-2d--2, 
0], B, -- [, +], where , is an unit vector with the -th term equal to 1. The following 
assumptions are made regarding the system in Eq. (1). 
(A1) 
(A2) 
(A3) 
(A4) 
1 Here, as 
For every 8 E O, f(0, 0, 8) = 0. 
There exist open and convex neighborhoods of the origin 3;2 C Y and b/2 C b/l, an 
open and convex set 02 C O, and a function K: 2 x Y2 x 02  L/ such that for 
every 0-,t  2, yt+a E Y2 and 0 E 02, Eq. (1) can be written as nt = Ii'(wt, jt+, 0), 
where,2  y x L/ +d- 
K is twice differentiable and has bounded first and second derivatives on E1  2 x 
Y2 x 02, while f is differentiable and has a bounded derivative on 2 x I(E1 ) x 02. 
There exists 6 > 0 such that for every y E f(2, K(2,0, 02), 2), w E 2 and 
8, t9 e O2, l1 - ( o - o )l--l ' Ou  
well as in the following sections, A  denotes the n-th product space of the set A. 
1012 S. YU, A.M. ANNASWAMY 
(A5) There exist positive definite matrices P and Q of dimensions (n + m + 2d - 2) such 
that xtT(ATpA,- P)xt+ ['TBTpB,+ 2xtTATPB, ' <_ -xtTQxt, where 
fc = [0, a: 0, o)] r. 
Since the objective is to control the system in (1) where 0 is unknown, in order to stabilize 
the output y at the origin with an estimate Or, we choose the control input as 
ut = g(wt, 0, 
(2) 
2.1 PARAMETER ESTIMATION SCHEME 
Suppose the estimation algorithm for updating 0" is defined recursively as A0 zx 0" - 
't- = R(yt,wt-d, ut-d,t-) the problem is to determine the function R such that 0 
converges to 0 asymptotically. In general,  is chosen to depend on t, wt-a, 2tt-d and 
t- since they are measurable and contain information regarding 0. For example, in the 
case of linear systems which can be cast in the input predictor form, ut = 00, a well- 
known linear parameter estimation method is to adjust  as (Goodwin and Sin, 1984) 
A0 - '- ' 
- +_,,,_,, [ut-a - Lat-]. In other words, the mechanism for carrying out 
parameter estimation is realized by R. In the case of general nonlinear systems, the task 
of determining such a function R is quite difficult, especially when the parameters occur 
nonlinearly. Hence, we propose the use of a neural network parameter estimation algorithm 
denoted 0-adaptive neural network (TANN) (Annaswamy and Yu, 1996). That is, we adjust 
as 
{ t- +N(Yt,t-d,ut-d,O-,) ifAVd, <-e 
Ot : t--I otherwise (3) 
where the inputs of the neural network are t, wt-a, nt-a and ,_ 1, the output is A0, and 
 defines a dead-zone where parameter adaptation stops. 
The neural network is to be trained so that the resulting network can improve the pa- 
rameter estimation over time for any possible 0 in a compact set. In addition, the 
trained network must ensure that the overall system in Eqs. (1), (2) and (3) is stable. 
Toward this end, N in TANN algorithm is required to satisfy the following two proper- 
lc(&-')l  , and (P2) A - AVe, < l 
ties: (P1)[N(yt,t_d,ut_d,t_i)] 2 5 a(i+lC(&_,,)12 ? _ , 
2+10(,-,)12 
, > 0, where A = IO, I - lOt_l[ 2, Ot = t - o, Vd, = a(l+lC(,_,,)12)2_ d, 
w 0:0,, , &: 
a 6 (0, 1) and 00 is the point where K is linearized and often chosen to be the mean value 
of parameter variation. 
2.2 TRAINING OF TANN FOR CONTROL 
In the previous section, we proposed an algorithm using a neural network for adjusting 
the control parameters. We introduced two properties (P1) and (P2) of the identification 
algorithm that the neural network needs to possess in order to maintain stability of the 
closed-loop system. In this section, we discuss the training procedure by which the weights 
of the neural network are to be adjusted so that the network retains these properties. 
The training set is constructed off-line and should compose of data needed in the train- 
ing phase. If we want the algorithm in Eq. (3) to be valid on the specified sets Y. and 
L/., for various 0 and 0 in ., the training set should cover those variables appearing in 
n 
Eq. (3) in their respective ranges. Hence, we first sample 0 in the set y. x , , 
Neural Control for Nonlinear Dynamic Systems 1013 
and 0,  in the set 133. Their values are, say, wl, 01 and  respectively. For the 
particular 0 and 0 we sample 0 again in the set {0 
and its value is, say, '. Once w,, 0, ' and  are sampled, other data can then 
be calculated, such as ul = K(w,0,') and y = f(w, ul,0). We can also ob- 
tain the corresponding C(r}) = o: )2 
C 0 ((l,yl,00), AVdl 
(1+1c(,)12)2 (ul -  and 
Li = a Ic(')l ^ 2 
(,+1c($,)127 (u, - u,) , where r}l = [wf,y,] T and u, -- Ii'(w, y,,O{). A data 
element can then be formed as (yl, w ,u, ', 01, AVa, L ). Proceeding in the same man- 
ner, by choosing various w,, 0,, 0 and 03 in their respective ranges, we form a typical 
training set Tta, - {(y,,w,,u,,,O,,AVa ,L,) I <s< M}, where M denotes the 
total number of patterns in the training set. If the quadratic penalty function method (Bert- 
sekas, 1995) is used, properties (P1) and (P2) can be satisfied by training the network on 
the training set to minimize the following cost function: 
minJ /x min 1 - { (max{0, AVe, }) 2 
14/ : 14/ 5i= 1 
(max {0, IV(W)l = - })= } 
(4) 
To find a W which minimizes the above unconstrained cost function J, we can apply 
algorithms such as the gradient method and the Gauss-Newton method. 
2.3 STABILITY RESULT 
With the plant given by Eq. (1), the controller by Eq. (2), and the TANN parameter 
estimation algorithm by Eq. (3), it can be shown that the stability of the closed-loop system 
is guaranteed. 
Based on the assumptions of the system in (1) and properties (P1) and (P2) that TANN 
satisfies, the stability result of the closed-loop system can be concluded in the following 
theorem. We refer the reader to (Yu and Annaswamy, 1996) for further detail. 
Theorem 1 Given the compact sets y_+l x b/ +a x 03 where the neural network in Eq. (3) 
is trained. There exist ,  > 0 such'that for any interior point 0 0f133, there exist open 
sets 3;4 C 3;3, b/4 C b/3 and a neighborhood 134 of 0 such that if Yo," ', Yn+d-2 E 3;4, 
0,'  , u,_2 E b/4, and 0,_l,   , 0,+_2 E 134, then all the signals in the closed-loop 
system remain bounded and Yt converges to a neighborhood of the origin. 
2.4 SIMULATION RESULTS 
In this section, we present a simulation example of the TANN controller proposed in this 
oy, ( 1 -, ) 
section. The system is of the form Yt+ = +e-0..o,, + nt, where 0 is the parameter to be 
determined on-line. Prior information regarding the system is that  E [4, 10]. Based on 
g, v,.(1-v,) where 't denotes the parameter 
Eq. (2), the controller was chosen to be ut - +,-o..,,,, 
estimate at time t. According to Eq. (3), 0 was estimated using the TANN algorithm with 
inputs Yt+, Yt, nt and 0t, and  = 0.01. N is a Gaussian network with 700 centers. The 
training set and the testing set were composed of 6,040 and 720 data elements respectively. 
After the training was completed, we tested the TANN controller on the system with six 
different values of 0, 4.5, 5.5, 6.5, 7.5, 8.5 and 9.5, while the initial parameter estimate 
and the initial output were chosen as '1 -- 7 and o = -0.9 respectively. The results are 
plotted in Figure 1. It can be seen that Yt can be stabilized at the origin for all these values 
of 0. For comparison, we also simulated the system under the same conditions but with  
1014 S. YU, A.M. ANNASWAMY 
lOO 4 
t 
Figure 1' yt (TANN Controller) 
100 4 
t 0 
Figure 2: yt (Extended Kalman Filter) 
estimated using the extended Kalman filter (Goodwin and Sin, 1984). Figure 2 shows the 
output responses. It is not surprising that for some values of 0, especially when the initial 
estimation error is large, the responses either diverge or exhibit steady state error. 
3 STABLE NEURAL CONTROLLER 
3.1 STATEMENT OF THE PROBLEM 
Consider the following nonlinear dynamical system 
(5) 
where x  R  and u  R TM. Our goal is to construct a stabilizing neural controller as 
u - N(y; W) where N is a neural network with weights W, and establish the conditions 
under which the closed-loop system is stable. 
The nonlinear system in (5) is expressed as a combination of a higher-order linear part and 
a nonlinear part as 5:= Ax + Bu + Ri (x, u) and y = Ca: + R2(x), where f(0, 0) = 0 
and h(0) -- 0. We make the following assumptions: (A1) f, h are twice continuously 
differentiable and are completely known. (A2) There exists a Ii' such that (A - BE'C) is 
asymptotically stable. 
3.2 TRAINING OF THE STABLE NEURAL CONTROLLER 
In order for the neural controller in Section 3.1 to result in an asymptotically stable closed- 
loop system, it is sufficient to establish that a continuous positive definite function of the state 
variables decreases monotonically through output feedback. In other words, if we can find a 
scalar definite function with a negative definite derivative of all points in the state space, we 
can guarantee stability of the overall system. Here, we limit our choices of the Lyapunov 
function candidates to the quadratic form, i.e. V - xTpx, where ? is positive definite, 
and the goal is to choose the controller so that 1< 0 where 1= 2xTPf(x, N(h(x), W)). 
Based on the above idea, we define a "desired" time-derivative Vd as ld= -zTQx where 
Q _ QT > 0. We choose P and Q matrices as follows. First, according to (A1), we can 
find a matrix K to make (A - BKC) asymptotically stable. We can then find a (P, Q) 
pair by choosing an arbitrary positive definite matrix Q and solving the Lyapunov equation, 
(A - BKC)T p + P(A - BKC) = -Q to obtain a positive definite P. 
Neural Control for Nonlinear Dynamic Systems 1015 
With the controller of the form in Section 3.1, the goal is to find W in the neural network 
which yields l)_<l)d along the trajectories in a neighborhood 2( C ' of the origin in the 
state space. Let :ci denote the value of a sample point where i is an index to the sample 
variable :c C 2( in the state space To establish l)<l)d, it is necessary that for every x in 
a neighborhood 2( C  of the origin, l)i<l)&, where l)i= 2xiTPf(x,N(h(.'), W)) 
and V& = -xiTQxi. That is, the goal is to find a W such that the inequality constraints 
AVe, < 0, where i = 1,..., M, is satisfied, where AVe, =1 - l)d, and M denotes the 
total number of sample points in 2(. As in the training of TANN controller, this can also 
be posed as an optimization problem. If the same quadratic penalty function method is 
used, the problem is to find W to minimize the following cost function over the training 
set, which is described as Tti: {(x,,yi, V&)]I < i < M}' 
M 
minJ /x min 1 E (max{0, AVe })2 
w : w ' (6) 
i=1 
3.3 STABILITY OF THE CLOSED-LOOP SYSTEM 
Assumptions (A1) and (A2) imply that a stabilizing controller u = -K:y exists so that 
V -- :cP:c is a candidate Lyapunov function. More generally, suppose a continuous but 
unknown function ?(:y) exists such that for V = scTpsc, a control input u = 7(/) leads to 
1)<_ -scTQsc, then we can find a neural network N(:y) which approximates 7(:Y) arbitrarily 
closely in a compact set leading to closed-loop stability. This is summarized in Theorem 2 
(Yu and Annaswamy, 1995). 
Theorem 2 Let there be a continuous function ?( h( x ) ) such that 2x TPf(x, 7(h(x))) + 
xT (x <_ O for every x  2( where 2( is a compact set containing the origin as an interior 
point. Then, given a neighborhood O C 2( of the origin, there exists a neural controller u -- 
N(h(x); W) and a compact set 3)  2( such that the solutions of : f(x,N(h(x); W)) 
converge to (9, for every initial condition x(to)  Y. 
3.4 SIMULATION RESULTS 
In this section, we show simulation results for a discrete-time nonlinear systems using the 
proposed neural network controller in Section 3, and compare it with a linear controller to 
illustrate the difference. The system we considered is a second-order nonlinear system xt = 
9 
f(sct_, ut-, ), where f = [f, f2] T, f: :c,_, x (1 + x2,_, )+ :c2,_l x (1 -ut-1 + u_ ) and 
f2 -- x 2,_  + 2x2,_ + ut- ( 1 + x2,_, ). It was assumed that x is measurable, and we wished 
to stabilize the system around the origin. The controller is of the form ut = N(x,, 0c2, ). 
The neural network N used is a Gaussian network with 120 centers. The training set and 
the testing set were composed of 441 and 121 data elements respectively. 
After the training was done, we plotted the actual change of the Lyapunov function, AV, 
using the linear controller u: -Ii':c and the neural network controller in Figures 3 and 4 
respectively. It can be observed from the two figures that if the neural network controller is 
used, AV is negative definite except in a small neighborhood of the origin, which assures 
that the closed-loop system would converge to vicinity of the origin; whereas, if the linear 
controller is used, AV becomes positive in some region away from the origin, which implies 
that the system can be unstable for some initial conditions. Simulation results confirmed 
our observation. 
1 O16 S. YU, A.M. ANNASWAMY 
o 
X 2 -0 2 X I 
Figure 3: AV(n = -/ix) 
-0 02 
-0.06 
0 
Figure 4: AV(n- N(x)) 
Acknowledgments 
This work is supported in part by Electrical Power Research Institute under contract No. 
8060-13 and in part by National Science Foundation under grant No. ECS-9296070. 
References 
[10] 
[11] 
[1] A.M. Annaswamy and S. Yu. O-adaptive neural networks: A new approach to 
parameter estimation. IEEE Transactions on Neural Networks, (to appear) 1996. 
[2] D. P. Bertsekas. Nonlinear Programming. Athena Scientific, Belmont, MA, 1995. 
[3] G. C. Goodwin and K. S. Sin. Adaptive Filtering Prediction and Control. Prentice- 
Hall, Inc., 1984. 
[4] A. Isidori. Nonlinear Control Systems. Springer-Verlag, New York, NY, 1989. 
[5] M. I. Jordan and D. E. Rumelhart. Forward models: Supervised learning with a distal 
teacher. Cognitive Science, 16:307-354, 1992. 
[6] A. U. Levin and K. S. Narendra. Control of nonlinear dynamical systems using neural 
networks: Controllability and stabilization. IEEE Transactions on Neural Networks, 
4(2): 192-206, March 1993. 
[7] K. S. Narendra and A.M. Annaswamy. Stable Adaptive Systems. Prentice-Hall, Inc., 
1989. 
[8] K. S. Narendra and K. Parthasarathy. Identification and control of dynamical systems 
using neural networks. IEEE Transactions on Neural Networks, 1 (1):4-26, March 
1990. 
[9] R.M. Sanner and J.-J. E. Slotine. Gaussian networks for direct adaptive control. IEEE 
Transactions on Neural Networks, 3(6): 837-863, November 1992. 
S. Yu and A.M. Annaswamy. Adaptive control of nonlinear dynamic systems using 
O-adaptive neural networks. Technical Report 9601, Adaptive Control Laboratory, 
Department of Mechanical Engineering, M.I.T, 1996. 
S.-H. Yu and A.M. Annaswamy. Control of nonlinear dynamic systems using a 
stability based neural network approach. In Technical report 9501, Adaptive Control 
Laboratory, MIT, Submitted to Proceedings of the 34th IEEE Conference on Decision 
and Control, New Orleans, LA, 1995. 
