Table Of Content1. Introduction
This book is concerned with discrete-time nonlinear control systems described by
difference equations of the form
x(t + f(x(t), u(t), w(t)),
1) =
(1.1)
y(t)=h(x(t),u(t)),
where x denotes the state of the system, u and w are the inputs, y is the output.
By u is denoted the control input, i.e. the input which we can manipulate and by
w is denoted the disturbance input which we cannot manipulate. In some cases
the disturbance is unknown, the other cases it can be measured or it is missing
Mtogether. In this book by outputs we mean the variables-to-be-controlled, i.e. the
variables in whose behaviour we are especially interested, and not the measurements
that we can perform on the system. We assume throughout the book that the state
vector x of the system can be measured.
Before we will discuss the main ass.umptions on the system (1.1) which will be
supposed to hold throughout the book and the types of feedback that will be applied
to the system (1.1), we focus on several examples of control systems which fit into
(1.1). The examples serve as a motivation for considering discrete-time nonlinear
control systems. As one will see, the examples are taken from different scientific
disciplines such as economics, chemical engineering, biology and medicine.
1.1 Examples of Discrete-time Nonlinear Control Systems
1.1.1 An Example of Discrete-time Nonlinear Control Model in
Economics
Discrete-time nonlinear dynamic models arise naturally in economics. Here we pre-
sent a simple macroeconomic model that describes an economy composed of two
sectors:
Xl(~: + 1)--- 1--~ [xl(t) + w(t)f(x~(t),x~(t))l- ~S(xl(t),x~(t))Ul(t),
. i
x2(t + l)= )t(l 2Xy.+ + " ~" f(xl(t), x2(t))ul(t) w(t) f(xl(t)' x2(t))u2(t) .
+ 1 1 +-----~-
In the above model by xl and x2 are denoted the capital stocks in the private
and the government sectors, respectively; ul is the saving tax rate and u2 is the tax
2 .1 Introduction
w(t)
rate of consumption. The rate of saving by the private sector is denoted by and
V is a constant.
1.1.2 Stagewise Processes in Chemical Engineering
Chemical engineers make extensive use of stagewise processes whose typical examp-
les axe plate absorption and distillation columns and a series of batch reaction,
extraction or leaching tanks. In each case the operation is characterized by a finite
between-stage change in the value of the dependent variable. The finite transition
between stages can be best modelled by difference equations. We shall present here
a simple example of countercurrent liquid-liquid extraction.
The process of countercurrent liquid-liquid extraction consists of M separate
stages. Each stage consists a separate tank. Under steady--state operation, L moles
of extract solvent and R m les of raffinate solvent are charged to tank k, mixed,
allowed to settle, and the raffinate sent to tank k - 1 and the extract to tank k b- .1
This process takes place in each tank. A sketch of the system is shown in Fig. 1.1.
The initial raffinate charge to tank 1 on each cycle consists of R moles of solvent
with a composition y(0) (moles of transferable material A per mole of solvent). The
initial extract charge to tank M on each cycle consists of L moles of extract solvent
b--M(z
with a composition )1 (moles of transferable material A per mole of solvent).
Raffinate and solvent are immiscible. Equilibrium is obtained in each tank. The
equilibrium relation is
z(k)=fly(k).
(1.2)
)O(yR J ~ ] ~ j /Ry(k-1)J ]Ry(k) i ~ Ry(M) .
Lz(l) ~ ~ L z ( k ) k . - ' [-Lz(k+l) . M Lz(M+I)
Fig. 1.1. Countercurrent liquid-liquid extraction stages.
It is desired to determine the expression relating the composition in a given
stage to the feed composition for steady-state operation. Apply a material balance
to tank k:
ny(k - Lz(k -q Lz(k) ÷ Ry(k) =
1) b- )1 (1.3)
When (1.3) is combined with (1.2) it results
+ y(k ay(k -
1) = (a -q 1)y(k) - 1), k = 0, 1,..., M, (1.4)
where
R/(Lfl) = a
1.1 Examples of Discrete-time Nonlinear Control Systems 3
cazl be considered as the control variable u. Denoting
y(k) = x2(k)
the difference equation (1.4) may be written in the state space form
xl(k-[-1)--x2(k)
x~(k+l)=(u+l)z2(k) -uxl(k), k=0,1,...,M (i.5)
y(k)=~(k).
1.1.3 An Example of Discrete-time Nonlinear Control Model in
Medicine
Nonlinear control systems often appear in pharmacokinetics and chemotherapy. The-
re is a certain temptation to employ discrete-time models since in clinical practice
and in laboratory, systems are generally observed at discrete times.
The next model describes the behaviour of hyperthyroid patients undergoing
antithyroid drug therapy:
xl(t -t- 1) = al exp[-a2xa( ] + ) t a3x ) t ( l
z2(t + 1) = a4 exp[-ahu( t ]) + a6x2( ) t
x3(t + )i = a~(t + )i + asx4(t + )1 + a~x~(t)
x4(t -~- 1) = alox2(t + )i + anx4(t + 1)
xs(t + 1) = a12f(t) d- ai4xh(t)
where
f ,O if x3(t) < a~3
f(t)
1 - exp[a13 - xs(t)], if xs(t) > al~.
In this model the following notations have been used:
xl(t) - plasma TSH at time t, mU/liter
x2(t) - stored iodoprotein at time t, nM
xa(t) - plasma Ts concentration at time t
x4(t) - plasma 4T concentration at time t
xs(t) - clinical index at time t
u(t) - drug dosage per day at time t, mg/day.
The second, third and fourth equations describe the dynamics of thyroid hor-
mones and in the second equation, drug action on hormone synthesis is included.
1.1.4 An Example of Discrete-time Nonlinear Control Model in
Renewable Resource Management
Bioeconomic problems have been recognized as a rich source of mathematical control
problems. Ecosystem managing includes maximizing sustainable yield, controlling
4 .1 Introduction
pests and diseases. There are many situations where a difference equation model is
more appropriate than differential equation. This happens when population growth
occurs at discrete times and when generations are completely nonoverlapping. Bio-
logical examples are provided by some insect populations and certain fishes which
live in isolated generations. Discrete-time formulation is also widely used for descri-
bing age-structured population models since one ordinarily keeps track of age groups
instead of exact age. Very often there is a natural unit of time associated with a
particular resource system. In most cases, the structure of the model is a crude app-
roximation to reality and it is difficult to find instances where control may be applied
precisely to actual situation. It is common to see the control policies as indicative.
It has been recognized that ecosystem models must include a degree of uncer-
tainty in their description. In real ecosystems, this uncertainty arises due to fluc-
tuations in the environment, or neglecting less important interactions between po-
pulations in the ecosystem. Usually, these uncertainties are described via external
perturbation vector in the model over which we have no control and the model is
exactly (1.1) with w(t) as perturbation vector. The elements ~x of x, i = 1,..., n
usually denote the density (biomass, number) of the ith species, although in ecosys-
tem models, some of the variables ~x may represent the concentration of nutrients
and chemical pollutants. By u(t) is usually denoted the harvesting rate and by w(t)
the perturbations which describe uncertainties of the model. One-dimensional state
equations describe the growth oofn e species. The multidimensional models describe
the complex interactions between animal and plant populations in the development
of the ecosystem. Animal and plant populations compete and cooperate with one
another to obtain sufficient natural resources to sustain their growth and survival.
Next model describes the effects of two species on one another. The two species,
krill and baleen whale, are related by the fact that one is a source of food for other:
xl(t + )1 = xl(t)exp{r,[1 -- (xl(t) + ax2(t))/(1 + a)]} -4- blul(t)xl(t)) ,
x2(t + 1) = x2(t)exp{r2[1 - (x2(t)/xl(t)]} + b2u2(t)x2(t)).
Here xl(t) and x2(t) represent the krill (prey) and baleen whale (predator) po-
pulation, respectively, at time t; u(t) is the harvesting rate at time t, rl, r2, bl, 2b
and a are some constants.
1.1.5 Sampled-data Systems
Most commonly discrete-time nonlinear control systems appear as the discretiza-
tion of continuous-time nonlinear systems. In the controlo f continuous-times ystems
present-day technology often asks for digitally implemented controllers, and some-
times it is preferable to design a controller in the discrete-time domain rather than
discretizing the continuous-time control law.
Here an exact step-invariant sampled-data model of a multi-input-multi-output
continuous-time system S, linear is control, is given. The considered system is desc-
ribed by differential equations of the form
m
c~ = fo(x) + ~, fi(x)ul, x(O) = xo, (1.6)
i=1
1.1 Examples of Discrete-time Nonlinear Control Systems 5
where x E X C R ~, u = (ul ... u,,,) E R ~, fi : X -+ X, i = 0, 1,..., m are analytic
functions.
The exact step-invariant sampled-data model of a continuous-time system (1.6)
is defined as the one whose response to a step input (i.e. the type of control usually
available under digital control)
u(kT + t) = u(kT) , < t O < T
is identical to that of the continuous-time system at discrete instants of time kT,
k > 0, provided the initial states of both systems are the same.
Derivation of sampled-data model is based on the representation of the solution
of the differential equation (1.6) in terms of a formal Lie exponential series
• (kT + t) = ~ ~t ~ 0 <_ t < T (1.7)
LI0 +E l=y x f,~, )Tk(~I '
where by Lio+~,~ ~ ~u~l is denoted the Lie differential operator associated with the
function fo(x) + 1=~'~ fi(x)ui
I=?'E+oSL ,,~,s = Yo,j(x) + fl,j(x)ul Oxj'
j=l
by ~ L ~.~ its r-multiple composition
01 =i~.~"1" 1 ~iUi
= L ~, r>2,
fiui /O+ff'~i= 1/iui fo+~i=l fiui ' --
and by L~0+~ f.~ an identity operator I. Of course, (1.7) may only be defined for
T sufficiently small.
As we are interested in the states x only at sampling instants kT + T, we obtain
from (1.7) for t = T
x(kT + T) = ,_~ .v~ L~o+E~,/,,,,xl~(kT) F(x(kT), u(kT)),
r>0
where F(x, u) because of
L ~, = L/o + fi ~/L~u
i=1
takes the following form
F(x,u)=F0(x)+ 1,u)x(l,F + E
+...
i1=1 1=2i~1i
(1.8)
... -]- fi Fil...i,(x)ttil . . . ui, +... ,
il,...,is=l
&'"i'(x)= fi (r Tr+V +p)! Z: L/oLlilLfoL$i2 1, 2o - r°,+l~
... lvfiza~fo ~.
r=O el ...+Cp+l + =r
lC > -I-a-e,.-.,O 1 >0
6 .1 Introduction
The exact step-invariant sampled-data model (1.8) is defined in terms of infinite
series both with respect to the sampling period T and the control u. So, in the
general case, the model (1.8) is usually not computable. In reality, to compute the
model, one must confine oneself with finite number of terms in this series. In that way
we reach the notion of approximate sampled-date-models. Computing approximate
sampled-date model corresponds to the truncation of the infinite series with respect
to the sampling period T as well as to the control u, at the fixed orders r and ,A
respectively, which define the orders of approximation of the sampled system
F¢':~(x,u)=F;(x)+ ~ F~(xlu,~ +...+ fi F~...,~(x)u,,...u,~,
11 =1 Q ,...,i~=1
-p Tr+p
+ p)! L °' " L L
fOIJhl"~/o .'" hp fo X.
r=O cl +...+clo+ 1 =r
lC >O,...,Cp+l >0
The response of the (% A)th-order-approximate sampled-date model to a step
input will agree with that of the continuous-time system at given instants of time
t = kT, k _> ,O up to an error of order O(T'+l,u~+l).
The rth-order approximate sampled-date model can be computed recursively on
the bases of the (r - 1)th-order model, starting with the first-order model which is
equivalent to the classical Eulel discretization scheme
Fil...ip(x ) = ifoFi,...,~(x 1--~ ) + if, Fi2...,p(x 1-r" ) . (1.9)
1.2 Discrete-time Nonlinear Control Systems
Next we will briefly discuss discrete-time nonlinear control systems given by (1.1),
i.e. by the equations
x(t + 1) = f(x(t), u(t), w(t)),
y(t)=h(x(t),u(t)),
where as before x, u, w and y denote reprectively the state, the control, the distur-
bance and the output. We shall work on a finite time interval, that means we assume
(1.1) to hold for t = 0,1,2,... ,tF, where tF is some finite time instant, tF < 00.
We assume that the states x(t) belong to an open subset X of R ~, the controls u(t)
belong to an open subset U of R m, the disturbances wit ) belong to an open subset
W of/i7 and the outputs y(t) belong to an open subset Y of R p, all for 0 < t < tF.
Then (1.1) is a shorthand writing for
xl(t + 1) = fl(xl(t),... ,x~(t),ul(t),... ,um(t),wl(t),... ,w~(t)),
..o
x~(t + 1) f~(xl(t),... ,x,(t),ul(t),... ,um(t),wl(t),... ,w~(t)),
(I.i0)
{ )t(lv = ui(t),...,
yp(t) hp(xl(t),..,x,(t),ul(t), ..,u,,(t)).
1.2 Discrete-time Nonlinear Control Systems 7
We assume the state transition map f : X x U x W *-- X to be smooth
mapping. In this context smooth means C O° , i.e. that all of the partial derivati-
ves Os+~+k/Oxh... ~ixO Oujl... Ouj, C3wh...Owlk exist and are continuous, though
many results which will be given in the next chapters hold under weaker conditions.
In many circumstances f only needs to be sufficiently many times continuously diffe-
rentiable with respect to x, u and w. We work with C O° functions in order not to
have to keep track of the exact number of derivatives required in different situations.
Note that if f : X x U *-- X and h : X ~ Y are smooth and X,U,Y are open
subsets, then the composition h o f : X x U +-- Y is also smooth. Sometimes it
will be useful to strengthen the smoothness condition and to require that f is (real)
analytic. Similarly we assume the output map h : X x U ~-- Y to be smooth or
analytic. For convenience we consider the system (1.10), initialized at t = 0; since
the system is time-invaxiant (i.e. not explicitly depending on time) this may be done
without loss of generality.
tF}
The system (1.1) has as inputs the'sequences u = {u(t); 0 < t < of control
vectors from U C R m and sequences w = {w(t); 0 < t < tF} of disturbance vectors
from W C R .~ We denote the sets of such time sequences by L/and Y~ respectively.
Together with (1.1) we have to specify the classes of admissible controls L/ and
admissible disturbances }/Y. For discrete-time system (1.1), U may be any set of
time functions u : Z tr ~-- U where Z~ F stands for the set {0,1,2,..., tF}. Similarly,
a class of admissible disturbances 14/may be any set of time functions w : Z tr +-- W.
For difference equation
x(t + 1) = f(x(t), u(t), w(t)), x(O) = Xo,
as long as f is a well-defined function on X x U x W, there is no problem regar-
ding existence and uniqueness of its solution x(t), 0 < t < tr, for any admissible
control sequence u E/~, any admissible disturbance sequence w E W and any arbit-
rary initial state x0 E R ~. Such a solution will usually be denoted as x(t, ,oX u, w)
which is a shorthand writing for x(t, ,oX u(O),..., u(t), w(O),..., w(t)). Similarly,
we let y(t, x0, u, w) = y(t, ,oX u(O),..., u(t), w(O),..., w(t)) denote the trajectory
of the output for the system (1.1) corresponding to a choice of the control sequence
u(0),..., u(t), the disturbance sequence w(0),..., w(t) and the initial state x0.
In chapters 2, 3, 5, 6 we shall consider systems without disturbances
x(t + 1) = f(x(t), u(t)), x(O) = Xo,
(1.11)
y(t)=h(x(t),u(t))
where z, u, y and h are defined as before and f : X × U ~ X.
The system S = {U, Y,X, f, h} given by (1.11) generates for each initial state
x0 the input-output (IO) map Sff0 on the set U. The IO map
assigns to each input sequence u E/g the output sequence y = {y(t) E Y; 0 < t <
< tF} E Y according to the recursion (1.11) and assuming x(0) = x0.
8 1. Introduction
If C = {U c, yc, X c, it, h e} is another control system of the form (1.11) with
Y° _C U, we may define the series connection of S with C, denoted by S o C, to be
the system
S o C = {Uv,y,x x XV,fs°c, hS°°},
where
fa°C(x, xC, u°)= f(x'h(x°'u°)) o ' u ° )
and
hS°°(x,
x°,.) =
Thus S o C is the system which results when the output of C is made the input
to S. The initial state of S o C is (x0, oX c) where by x0 c is denoted the initial state
of C.
Clearly, the IO map of the composite system S o C is the composition of the IO
maps for S and C:
zSoC = ~S o . (1.12)
Throughout the book we shall adopt a local viewpoint. In the discrete-time case
the local study is useless around an arbitrary initial state since even in one step, the
state can move far away from the initial state, regardless of the small control and
disturbance values. In order not to loose localness one possibility is to work around
an equilibrium point of the system, that' is around (x °, u °, w °) 6 X x U x W such that
f(x °, u °, w °) = x °. Since in general the disturbances w(t), 0 < t < tF need not be
necessarily close to w °, for systems with disturbances working around an equilibrium
point is more restrictive if compared to the case without disturbances. Another, a
more general possibility, is to work around certain set of time sequences of state,
control, disturbance and output {~(t), ~(t),,~(t), ,3(t); 0 < t < tF} that satisfy the
system equations (1.1). This set of time sequences is called the (reference) trajectory
of the system. Note that an equilibrium point can also be considered a trajectory,
namely it is a trajectory consisting only in one point. Working around a trajectory
is useful in situations where there is no natural equilibrium point or an equilibrium
point has little relevance for control purposes. In this book mainly the first approach
will be followed but for a few problems local solutions will be presented around a
given trajectory.
If we assume to work in a neighbourhood of an equilibrium point and on a finite
time interval 0 < t < tf, then, using the control sequence {u(t); 0 < t < tf}
with each u(t) sufficiently close to u ° and provided that in the disturbance sequence
{w(t); 0 < t < tf} each w(t) is sufficiently close to w ° and that the initial state oX
is sufficiently close to x ° we can assure that the states x(t) are sufficiently close to
x °, and the outputs y(t) are sufficiently close to Oy = h(xO), both for 0 < t < tf.
Let us denote by X ° the open subset of X such that II x - x ° II< 7 for some
7 > 0. Let us denote by U ° the set of controls u which are sufficiently close to u °, that
is II u-u° II< ~ for some 6 > 0. Let us denote by Y°(Yi° ) the set of outputs y (scalar
output components )iY which are sufficiently close to yO(yO), that is II Y - y0 H<
< s(ll y~_ Oy ii < e) for some s > 0. Denote by b/° and 3;°(3~i) the sets of control
1.3 Static and Dynamic State Feedback 9
sequences {u(t) E U°; 0 < t < tF} and output sequences {y(t) E y0; 0 < t < tF}
(output component sequences {y~(t) E ~0; 0 < t < tF}), respectively.
1.3 Static and Dynamic State Feedback
In the next chapters we will discuss changing the structure of nonlinear control
systems via feedback. Since we assume throughout the book that the whole state
vector of the system can be measured, we shall only consider state feedback. We
will be interested in two types of state feedback for a system (1.1) or (1.11)--static
state feedback and dynamic state feedback.
A static state feedback for the nonlinear system (1.11) or (1.1) is definedas a
relation
u(t) = ~(x(t), v(t)) (1.13)
where v E R '~ represents a new control vector. Note that since we are working locally
around an equilibrium point (x °, u °) of the system (1.11), the feedback (1.13) is also
defined only locally around a point (x °, v °, u°), such that ° v satisfies the relation
~o = ~(xo, v0).
A feedback (1.13) is called a regular static state feedback if O~(x, v)/Ov is a
nonsingular matrix for all x and v in the domain of ~.
The regularity of feedback implies that the closed loop system, i.e. the system
composed from the original system (1.1) and the feedback (1.13)
x(t + 1) = f(x(t), ~(x(t), v(t)), w(t))
y(t) = h(x(t), ~o(x(t), v(t) ) )
admits as many independent controls as the original system.
The adjective "static" indicates the fact that the feedback (1.13) decides what
the value of u at a time-instant t should be only on the basis of what the value of
x and v at that specific time-instant is. In this sense (1.13) could also be called a
memoryless feedback. If we also incorporate some kind of memory in the feedback
we arrive at what is called a dynamic state feedback.
A dynamic state feedback for the system (1.11) or (1.1) is defined as
xc(t + 1) = F(x(t), xc(t), v(t))
u(t) = ~(x(t), xC(t), v(t)) (1.14)
where ° x E X c C R ~c, fc : X x X. ° x V ~-- X o, o~ : X × X c × V ~ U are
smooth mappings and v E V C m R represents a new control vector. Here c x can
be interpreted as the memory of feedback. Sometimes the system (1.14) is called a
compensator.
A feedback (1.14) is called a regular dynamic state feedback if the system (1.1),
(1.14)
x(t + 1) = f(x(t), ~o(x(t), xC(t), v(t) ), w(t) )
xC(t + 1) = fC(x(t), xe(t), v(t)) (1.15)
~(t) = ~(x(t), ~°(t), v(t))
01 .1 Introduction
v(t) u(t)
with inputs and outputs defines a one-to-one (x, x ¢, w)-dependent corres-
pondence between the input variables v and the output variables u.
In the case when measurements of the disturbances are available, we allow the
feedback to depend on disturbance w. So, in this case we shall use a static state
feedback of the form
u(t) = v(t), w(t))
and a dynamic state feedback of the form
xc(t + 1) = v(t), w(t))
u(t) = v(t), w(t)).
Notes and References
The economic example has been taken from Aok75a. The other examples of
discrete-time dynamic models in economics, simple as well more advanced can be
found in Aok75a, Aok75b, Aok76, INij89, NS90. A detailed exposition of the
process of countercurrent liquid-liquid extraction was given in MSR57 where more
examples of discrete-time nonlinear stagewise processes are discussed. For a further
discussion of the drug therapy model we refer to Swa84. The ecosystem model is
discussed in Fis87. For more ecosystem and renewable resourse management mo-
dels, simple and advanced see Cla76, Fis87, Get87, KL88, Lev84, May74,
May78, Ost78, PalS3, Pie77, Sch85.
The problem of finding the exact sampled-data model for continuous-time linear-
analytic system was first addressed and solved by Monaco and Normand-Cyrot
MNC85 for single-input systems. The sampled-data model (1.8) ha been taken
from Kot89 and the formulas (1.9) for recursive computation of the approximate
sampled-data model from Kot94. The different formulas for computing sampled-
data models can be found in MNC86a, MNC90. While all formulas for computing
sampled-data models are similar in the sense that their derivation is based on the
representation of the solution of the continuous-time system in terms of a formal Lie
exponential series FLL83, and that all formulas give the same smpled-data models
when applied to any concrete system, they have several differences. The formulas
given in MNC86a, MNC90 have a form which is better suited for studing seve-
ral properties of the sampled-data models, such as controllability and observability.
However, they are more complicated to "apply n~aht (1.8) since they contain combina-
torics, shuffle products and ad-operators. If one wants just to find the sampled-data
model of some concrete process, the easier formulas (1.8) will do the job. An exact
sampled-data model for multi-input continuous-time system, nonlinear in control is
presented in Kot94. In Bar89 the program written with the aid of computer al-
gebra system REDUCE for computing sampled-data model of linear-analytic system
has been presented.
We should like to end up with some remarks concerning the form of the state
transition map f in (1.11) (or in (1.1)). No significant simplification occurs when f
is linear in control, i.e. when f has the structure commonly used in the continuous
time. This is due to the fact that in discrete-time, one extensively uses the operation
