A new update rule to design Iterative Learning

Controller using current cycle parameters and

system dynamics
Shanta Sondur Suvarna Bharambe
V. E. S. Institute of Technology S.I.G. College of Engineering
Department of Instrumentation, Mumbai. Department of Instrumentation, Navi Mumbai.
shantasondur@gmail.com bharambe.suvarna@gmail.com

Abstract- We propose a new Iterative Learning Controller (ILC) uncertainties that are exhibited during actual system operation
algorithm based on M. Pandit and D. Andress [3] and Niels or to the lack of suitable design techniques. Thus, it is not easy
Johannes Maria Bosch [10] considering the current cycle
to achieve perfect tracking using traditional control theories.
parameters and some part of the system dynamics. We have
selected a Single Inverted Pendulum - which is multivariable and ILC is a design tool that can be used to overcome the
highly unstable system, as a control object to demonstrate the shortcomings of traditional controller design, especially for
effectiveness of the proposed ILC algorithm. The test results of obtaining a desired transient response, for the special case when
proposed ILC algorithm are compared with the other the system of interest operates repetitively. For such systems,
implementations such as LQR (non-learning), Lifted ILC and ILC
ILC can often be used to achieve perfect tracking, even when
algorithm suggested by M. Pandit and D. Andress [3]. The results
show that the proposed algorithm gives much better control as the model is uncertain or unknown and we have no information
compared to LQR, Lifted ILC algorithm and there is a significant about the system structure in details and nonlinearity.
improvement in the settling time as compared to ILC algorithm There are many application of ILC such as, industrial robots,
given by M. Pandit and D. Andress [3]. computer numerical control (CNC) machine tools, wafer stage
motion systems, injection-molding machines, aluminum
I. INTRODUCTION extruders, cold rolling mills, induction motors, chain conveyor
systems, camless engine valves, rapid thermal processing, semi
ILC is based on the notion that the performance of a system
batch chemical reactors etc [2]. There are many variations of
that executes the same task multiple times can be improved by
the basic ILC scheme and are discussed in [2]. ILC algorithms
learning from previous executions (trials, iterations, passes) [1].
can be divided in two basic forms Continuous-time ILC and
It differs from most existing control methods in the sense that it
Discrete time ILC algorithm [9]. In our work, we focus
exploits every possibility to incorporate past control
especially on discrete-time ILC.
information: the past tracking error signal and in particular the
For discrete time system, it uses lifted system representation.
past control input signal into the construction of present control
All signals are lifted to discrete time domain and update rule is
action.
formed considering controller output and error at previous
ILC is a relatively recent but well-established area of
iteration. One of the author M. Pandit and D. Andress suggested
study in control theory, and can be categorized as branch of an
ILC update rule in [3], which is based on reformulation of ILC
intelligent control methodology. From 1984, ILC started to
problem as multivariable control problem. This has enabled the
become an active research area. In 1984 [6]-[8] are
possibility of designing an iterative learning controller using
independently published describing a method that iteratively
well known methods from multi variable control theory. Here,
could compensate for model errors and disturbances. The
the controller K is designed such that it will generate correction
development of ILC stems originally from the robotics area
to the actual control input vector so as to minimize performance
where repetitive motion shows up naturally in many
measure.
applications [5]. It is an approach for improving the transient
In case of general discrete time ILC algorithm, convergence
performance of systems that operate repetitively over a fixed
condition includes only direct transmission part of the system to
time interval.
be controlled. In the method given by M. Pandit and D. Andress
Although control theory provides numerous design tools for
[3], the controller K is designed using Riccati equation
improving the response of a dynamic system, it is not always
considering system dynamics. This results in better control with
possible to achieve desired performance requirements, due to
faster convergence (less number of learning cycles) as
the presence of un-modeled dynamics or parametric

978-1-4244-5697-0/10/$25.00 ©2010 IEEE 5

compared to general ILC algorithm. In [3], authors have
employed pilot plant that is built using standard components
used in process industry to control flow and pressure of water.
Another author in [10] has discussed ILC for print head
considering the printer dynamics. In his thesis, he identified the
printer dynamics, modeled them and then designed the ILC in
Lifted setting to deal with problems concerning crosstalk. In his
approach if system has unobservable modes, then author has
suggested to neglect those unobservable modes and build only
closed loop system in trial domain with information of
observable modes of system, i.e. in other words, work with
reduced dimension system. Further author implement two linear
control methods such as pole assignment and LQ Optimal
control in iteration domain.
This motivates us to obtain better performance by optimizing
the learning process for SIP i.e. the system under consideration,
considering the current cycle parameters as given in [23] and
some part of system dynamics discussed in [10].
Since, SIP is a typical fast speed, multivariable nonlinear and
absolutely unstable system, we decide to select this system for
implementation.
In our paper we aim to implement ILC algorithm suggested
by M. Pandit and D. Andress [3]. Then, we propose extension
to this method in terms of correction to control input
considering the system information in transformed form as
discussed in [10] and implement it for the model of SIP. We
compare the results of these methods with Lifted ILC and LQR
based state regulator.
The rest of the paper is organized as follows: section 2 deals
with details of SIP along with its mathematical models. Section
3 gives details of proposed iterative learning control algorithm
along with results of implementation and observations. Finally,
we end with conclusions in section 4.
II. MODELING DETAILS OF SINGLE INVERTED PEMDULUM (SIP)
AND VARIOUS CONTROLLER DESIGN SCHEMES
In this section we present a brief introduction of SIP along
with its modeling details. We also present the various control
techniques applied to SIP so far.
A. Introduction
Inverted pendulum is a typical example of unstable system
and has been used for the verification of designed control
systems and/or control education in laboratories. In particular,
the control of the systems has been known as a good example to
show the power of modern control theory in an analogy with
the control of launching a rocket. The inverted pendulum
system is a standard problem in the area of control systems.
They are often useful to demonstrate concepts in linear control
such as the stabilization of unstable systems. Since the system
is inherently nonlinear, it has also been useful in illustrating
some of the ideas in nonlinear control.
6
This section gives details of the SIP models which are
described below. We have considered SIP model given in [4]
by Xianhe Zhang, Jie Wu, and Xisheng Zhan for
implementation of existing and proposed algorithms.
There also exists various control methodologies for SIP like,
the state transfer control of a double inverted pendulum
(TITech Pendulum) discussed in [13,14, 15], a systematical
method to construct stabilization fuzzy controllers for SIP based
on the SIRMs dynamically connected fuzzy inference model
given in [12] a iterative learning controller (ILC) design based
on resolving LMI method described in [4], controller design
using microcontroller based state feedback as in [11], linear
quadratic regulator (LQR) state-feedback controller discussed
in [17], state transfer control [16] etc.
B. SIP Model by Xianhe Zhang, Jie Wu, Xisheng Zhan [4]
The model details are given in [4] and reproduced here just to
give a clear idea of modeling details to the reader. The Single
Inverted pendulum is shown in figure 1. The pivot of the
pendulum is mounted on a carriage which can move in a
horizontal direction. The carriage is driven by small motor that
at time t exerts a force u(t) on the carriage. This force is input
variable for the system. This somewhat artificial system
example represents a dynamic model of a space booster on take
off. The pendulum is obviously unstable as it cannot remain in
the desired position or state without assistance of the control
force. We are assuming the movement of the cart will be
smooth and surface friction between the cart and rail is assumed
to be minimal enough to be neglected.
The cart with an inverted pendulum, shown below, is
"bumped" with an impulse force, F. Determine the dynamic
equations of motion for the system, and linearize about the
pendulum's angle, = 0 (in other words, assume that pendulum
does not move more than a few degrees away from the vertical).
Figure 1 Picture of Single Inverted Pendulum
Where, M - mass of the cart - 2.0 kg; m - mass of the pendulum - 0.1 kg; l -
length to pendulum center of mass - 0.2499 m; I - inertia of the pendulum -
0.006 kgm2; F - force applied to the cart; x - cart position coordinate; -
pendulum angle from vertical; N,P - Mutual force of horizontal and vertical
direction vector
According to Newton dynamics method, we have:
(For more details please refer in [4]).
 New Update rules for the proposed algorithm
Mx + N = F A new update rule for the proposed algorithm is give as
N = mx + mlθ cosθ - mlθ 2 sin θ
If one substitutes this equation into the first equation, we get the u k +1 = u k + ∆ u k + Ke k
first equation of motion for this system:
Proposed Iterative Learning rule: Problem statement is to add a
(M + m) x + mlθ cos θ − mlθ 2 sin θ = F correction to update rules so as to improve the convergence
To get the second equation of motion, speed of the system and to get a good tracking effect.
P sin θ + N cos θ − mg sin θ = mlθ + mx cos θ ∆ u k = L ek
− Pl sin θ − Nl cos θ = Iθ = L(Yk - Yd )
Combining these last two equations, we get the second dynamic Here Yd = 0, and Yk = Huk then
equation: ek = Yk
( I + ml 2 )θ − mgl sinθ = −mlx cosθ u k +1 = u k + LHu k + Ke k
= (I + LH)u k + Kek
Since Matlab can only work with linear functions, this set of
equations should be linearized about = 0. Considering tiny The SVD of H as discussed in [10] is given as:
vibration of the pendulum in the given point when =0, local
linearize the above equation, i.e. cos( ) 1, sin( ) 0 are used. T
After linearization the two equations of motion become (where Σ1 0 V1
u represents the input):
H = UΣV T
= [U1 U 2 ]
0 Σ 2 V2 T

( I + ml 2 )θ − mglθ + mlx = 0 Here, V is a matrix with singular vectors which represent

(M + m) x + mlθ = F input trajectories and U is a matrix with singular vectors of the
output trajectories. Both V and U are orthogonal matrices, UTU
Considering detailed data of the inverted pendulum from
= V T V = I. is a diagonal matrix containing the singular
reference [4], we get the state equations for this problem as:
values i, ordered from large to small. Thus, trajectories in the
direction vi are amplified with gain i in the direction ui. 2
θ 0 1 0 0 θ 0
contains part of the system nearly equal to zero.
θ 30.489 0 0 0 θ − 1 . 482
= + u (2. 1)
x 0 0 0 0 x 0
A new input w can be defined by
x - 0.363 0 0 0 x 0 . 494 Hw = Hw1 + Hw2;
1 0 0 0 θ
0 1 0 0 θ where w1 im V1 and w2 im V2. Thus
y = (2.2)
0 0 1 0 x
T
0 0 0 1 x Hw 1 U1 1V 1 w1;

as V T1 w2 = V T2 w1 = 0. Now, w1 is chosen as
The Eigen values of discrete system version of this system w1 = V1u
matrix are at 1.0000, 1.0000, 1.0568 and 0.9463. We can be see
that two poles are at 1.000 i.e. on the unit circle and one outside For the closed-loop system the expression
the unit circle at 1.0568. This confirms the intuition that the
u k +1 = u k + L * HV1u k + Ke k
system is unstable in open loop.
III. PROPOSED ILC ALGORITHM = (I + L * HV1 )u k + Kek (1)
is obtained.
In this section, a correction is proposed to ILC update rule
suggested by M. Pandit and D. Andress [3] based on the
By omitting 2, model reduction is applied (indicated by *).
approach discussed by Niels Johannes Maria Bosch in [10], for
Let
improving the response time of the system. The proposed new
algorithm is then implemented to SIP model discussed in L * HV1 = L * U 1 Σ 1V1T V1 = α I
section B and results along with observations are presented. −1
L* = α Σ 1 U 1T (2)

7

The parameter is chosen which decides the percentage of
correction applied to extension rule.
Steps of implementation
1. Build a system as in form of equation given in [3].
2. Design the controller K as given in [3].
3. Neglecting singular values which are nearly equal to
zero or in other words we can say that by considering
only l columns of , build matrices Ul, Vl and l.
4. Then build learning gain matrix from equation (2).
5. Generate control input uk.
6. Apply the control input to system.
7. Store control input and error of current iteration.
8. From this current iteration data, generate control input
from equation (1) to be applied to the system when
operated in the next iteration.
To demonstrate the effectiveness of the proposed algorithm,
we also design and implement the controller on SIP system
based on LQR, Lifted ILC and algorithm suggested by M.
The proposed algorithm gives better transient response as
compared to other three techniques. The percentage reduction
in peak overshoot, undershoot and settling time as compared to
LQR is 60%, 28% and 58%, respectively.
In comparison with the lifted ILC, the reduction in peak
overshoot, undershoot and in settling time is 35%, 9% and 68%,
respectively and as compared to the algorithm suggested by M.
Pandit and D. Andress [3], we obtained the percentage
reduction in peak overshoot by -89%, undershoot by 4% and
settling time by 73%.
Finally, it is found that as compared to ILC algorithm by
Pandit and Andress [3], the proposed ILC algorithm with new
update rules gives much faster response by reducing settling
time 73%.
It is seen that, there is a considerable improvement in the
settling time at the expense of increase in peak overshoot.
Since, peak overshoot is within +/- 0.05 radiance, this does not
affect the overall performance of the system, at the same time
this is much less than the LQR and Lifted ILC.
0.1

Pandit and D. Andress [3].

0.05
IV. RESULTS
0
The results of the proposed algorithm along with other non-
learning and learning algorithms like, LQR, Lifted ILC and −0.05
algorithm by M. Pandit and D. Andress [3] are presented in this
Amplitude

section. The system is tested for 10 number of iterations having −0.1

each duration of 20 seconds. Results are obtained and are as

given below. Comparison of response by learning and non −0.15

learning (LQR ) technique is presented in figure 3.

−0.2
From the results shown in table 1, we can say that, as number
of iterations increases, the pendulum stabilizes earlier with less −0.25
WO ILC
L−ILC
number of oscillations around the desired trajectory. P.and A.ILC
Proposed ILC
It can also be seen from figure 3 that overshoot and settling −0.3
0 2 4 6 8 10 12 14 16 18 20
time improves as it learns from previous iteration. From Time sec

comparison of response of the Proposed ILC algorithm with

that of LQR method, it is seen that variations in theta in Figure 3 Comparison of responses of proposed ILC with
transient period is limited as compared to non-learning method existing methods
and pendulum stabilizes earlier i.e. within 1.4 seconds as shown
in figure 3. Error in tracking the desired trajectory is very less Table 1 Comparison of transient response parameters with
as compared to non-learning controller and transient response proposed and existing methods
parameters are significantly improved as compared to LQR
response parameters as indicated in following tables 1 and 2. Sr. Method Peak Undershoot Settling
No. Overshoot Time(sec)
V. CONCLUSIONS
1 LQR 0.089 -0.2882 3.4
We proposed a new update rules for ILC algorithm
2 Lifted ILC 0.0552 -0.2285 4.5
considering the current cycle parameters and system dynamics
based on M. Pandit and D. Andress’s method [3] and Niels 3 Algorithm 0.0188 -0.2161 5.2
Johannes Maria Bosch method [10] respectively. We have by Pandit
and
successfully implemented the proposed algorithm on SIP and
Andress[3]
compared the performance of the proposed ILC algorithm with
4 Proposed 0.0356 -0.2068 1.4
LQR, Lifted ILC and ILC algorithm by M. Pandit and D. algorithm
Andress [3].

8
Table 2 Percentage reduction in transient response
parameters obtained by proposed algorithm over the
existing methods

Sr. No. Method Peak Undersh Settling

Overshoot oot time
1. LQR 60 28 58
2. Lifted ILC 35 9 68
3. By Pandit and -89 4 73
Andress[3]

REFERENCES
[1] Kevin L. Moore and Jian-Xin Xu (Eds). Special Issue Iterative Learning
Control. Int. J. of Control, 73(10):819–999, 2000.
[2] Hyo Sung Ahn,YangQuan Chen and Kevin Moore,”Iterative Learning
Control:Brief Survey and Categorization”, IEEE Trans. Systems, Man and
Cybernetics vol.37 .no.6, Nov2007, pp1099-1118.
[3] D.Andress,M.Pandit,”Iterative Learning Control for Discrete Time
System as MultiLoop Control”, CCSP2000 International Conference on
Communication Controls and Signal Processing, July2000.
[4] Xianhe Zhang, Jie Wu, XishengZhan, ”Iterative Learning control on
Single Inverted Pendulum”, International symposium on Intelligent
Tech.Application Workshop, 2008.
[5] YangQuan Chen,Chagyun Wen, Iterative Learning Control Convergenc,,
Robustness and Application, Springer-Verlag, London, 1999.
[6] S. Arimoto, S. Kawamura, and F. Miyazaki, “Bettering operation of
robots by learning,” J. Robot. Syst., vol. 1, no. 2, pp. 123–140, 1984.
[7] G. Casalino and G. Bartolini, “A learning procedure for the control of
movements of robotic manipulators,” in Proc. IASTED Symp. Robotics
and Automation, San Francisco, CA, May 23–24, 1984, pp. 108–111.
[8] J. J. Craig, “Adaptive control of manipulators through repeated trials,” in
Proc. ACC, San Diego, CA, June 1984, pp. 1566–1572.
[9] C. Leondes, Discrete Time Control System analysis and design,
Academic Press,1995
[10] Niels Johannes Maria Bosch, “Iterative Learning Control on an Inkjet
Printhead”, Research Report, Eindhoven University of Technology , June
2005.
[11] John Stang, “The Inverted Pendulum” , Design Project, Cornell
University, May, 2005
[12] Jianqiang Yi, Naoyoshi yu,bazaki and Kaoru Hirota,” Systematically
Stabilization Fuzzy Controller for Single and Double Pendulum
Systems”, IEEE, 2000.
[13] Masaki Yamakita, Tasuku hoshino, Katsuhisa furuta,” Control Practice
Using Pendulum”, Proceedings of the American Control Conference San
Diego, California June 1999
[14] M. Yamakita, K.Nonaka, K.Furuta,”Swing Up Coiitrol of Double
Pendulum”, Proc. of ACC’93, pp. 2229/2234, (1993)
[15] M.Yamakita and K.Furuta,”Stabilization of a Limit Cycle in TITech
Double Pendulum ”, Proc. IFAC Congrcss (1999)
[16] Kent H. Lundberg James K. Roberge,“Classical Dual-Inverted-Pendulum
Control”, presented at IEEE conference on DECISION AND CONTROL
4399, 2003.
[17] Carina Hansen, Cecilia Svensson, “Construction and Control of an
Inverted Pendulum”, Master Thesis, Lund Institute of Technology, July
2000.
[18] Fatai Olalekan, Ayman Abdallah, SE 514 OPTIMAL CONTROL FOR
SINGLE AND DOUBLE INVERTED PENDULUM, project report.
[19] Daoying Pi, D. Seborg, Jianxia Shou, Youxian Sun, and Qing Lin.
Analysis of current cycle error assisted iterative learning control for
discrete nonlinear time-varying systems. In IEEE International
Conference on Systems, Man, and Cybernetics, pages 3508 – 3513,
Nashville, TN USA, Oct. 2000.