| ഀ
ഀ ഀ | ഀ
ഀ
||
ഀ
| ഀ
||
ഀ
ഀ
ഀ
ഀ
ഀ
Adaptive Critic DesignsA
ഀ
ഀ
ഀഀ
ഀ
ഀ
ഀ
CpEng 458 /EE 458/System Eng 458
ഀ
ADAPTIVE CRITIC DESIGNS (ACDs)
ഀ
ഀഀ
All students, graduate and undergraduate are encouraged to take ഀ
this course.
ഀ
We will explore and exploit the field of Adaptive Dynamic Programming and Reinforcement Learning.
ഀ
ഀ
Prescribed Textbook ഀ
(PT)
ഀ
Handbook of Learning and ഀ
Approximate Dynamic Programming, Edited by J Si, A G Barto, W B Powell, D Wunsch, ഀ
Wiley-IEEE press, July 2004, ISBN: 0-471-66054-X.
ഀ
ഀ
Course Contents: Estimated 12 weeks
ഀ
ഀ
1. ഀ
Introduction –ഀ
(estimated 3 weeks)
ഀ
ഀ
a. ഀ
Review of Neural Networks – CpE 358 / EE ഀ
367 (Computational Intelligence)
ഀ
ഀ
b. ഀ
Review of neurocontrol techniques – ഀ
Chapter 3 of RR-1
ഀ
ഀ
c. ഀ
Review of optimization techniques - Notes
ഀ
ഀ
2. ഀ
Reinforcement learning (RL) – Chapters 2, ഀ
7 and 8 of PT (estimated 2 weeks)
ഀ
ഀ
3. ഀ
Dynamic Programming (DP) and Approximate ഀ
Dynamic Programming (ADP) – Chapters 1, 4, 5 and 19 of PT, Chapter 1 of RR – 2. ഀ
(estimated 1 ഀ
week)
ഀ
ഀ
4. ഀ
Backpropagation Through Time (BPTT) [RR ഀ
-5] and Chapter 15 of PT (estimated 1 week).
ഀ
ഀ
5. ഀ
Adaptive critics: Class of Adaptive ഀ
Critics – Chapters 3, 4, 5 and 19 of PT (estimated 3 weeks)
ഀ
ഀ
a. ഀ
Heuristic Dynamic Programming (HDP) and ഀ
Action Dependent HDP
ഀ
ഀ
b. ഀ
Dual Heuristic Programming (DHP)
ഀ
ഀ
c. ഀ
Global Dual Heuristic Programming (GDHP)
ഀ
ഀ
6. ഀ
Case studies on RL and ACDs –ഀ
Control Problems, Communication Systems, ഀ
Power Systems – Chapters 3, 4, 5, 10, 18, 19 and 20 of PT, RR-6. In addition, ഀ
any other publications relevant to case study will be used. (estimated 2 weeks)
ഀ
ഀ
ഀ
Recommendedഀ
ഀ
ഀ
1. ഀ
Handbook of Intelligent Control – Neural, ഀ
Fuzzy and Adaptive Approaches, Eds. White and Sofge, Van Nostrand Reinhold, New ഀ
York, 1992, ISBN 0-442-30857-4.
ഀ
ഀ
2. ഀ
D Bertsekas, Dynamic Programming: Deterministic and ഀ
Stochastic Models, Prentice Hall, ISBN 0132215810.
ഀ
ഀ
3. ഀ
D Bertsekas, ഀ
ഀ
Dynamic Programming and Optimal Control, Vols. ഀ
I and II, Athena Scientific, 1995, ഀ
(2nd Edition Vol. I, 2000, 2nd Edition Vol. II, 2001).
ഀ
ഀ
4. ഀ
D Bertsekas and J Tsitsiklis, ഀ
ഀ
Neuro-Dynamic Programming, Athena Scientific, 1996, ഀ
ISBN: ഀ
1-886529-10-8.
ഀ
ഀ
5. ഀ
PJ Werbos, “Roots of ഀ
Backpropagation”, Wiley, ISBN 0-471-59897-6, 1994.
ഀ
ഀ
6. ഀ
“Adaptive ഀ
Critic Based Optimal Neurocontrol for Synchronous Generator in Power System ഀ
Using MLP/RBF Neural Networks”, IEEE ഀ
Transactions on Industry Applications, vol. 39, no. 5, September/October ഀ
2003, pp. 1529-1540.
ഀ
ഀ
ഀ
Prerequisites
ഀ
Computational Intelligence (CpEng/EE/ME/Sys Eng 301 – known referred to as CpEng ഀ
358/EE 367) or EE 368 Neural Networks or Permission of the Instructor.
ഀ
ഀ
ഀ
Project
ഀ
Projects will be carried out individually. Topics will be decided in ഀ
consultation with the instructor or assigned by the instructor. Projects will ഀ
involve the application of adaptive critic designs.
ഀ
ഀഀ
ഀ
Please email any questions to Dr. G K Venayagamoorthy at ഀ
ഀ
ഀ
ganeshv@mst.eduഀ
ഀ
ഀ
ഀ
ഀ
ഀ
ഀ
ഀഀ
ഀ
ഀ
ഀ
ഀ ഀ ഀ
ഀ
ഀ