CpEng 458 /EE 458/System Eng 458

ADAPTIVE CRITIC DESIGNS (ACDs)

 

All students, graduate and undergraduate are encouraged to take this course.
We will explore and exploit the field of Adaptive Dynamic Programming and Reinforcement Learning.

 

Prescribed Textbook (PT)

Handbook of Learning and Approximate Dynamic Programming, Edited by J Si, A G Barto, W B Powell, D Wunsch, Wiley-IEEE press, July 2004, ISBN: 0-471-66054-X.

 

Course Contents: Estimated 12 weeks

1.      Introduction –  (estimated 3 weeks)

a.       Review of Neural Networks – CpE 358 / EE 367 (Computational Intelligence)

b.      Review of neurocontrol techniques – Chapter 3 of RR-1

c.       Review of optimization techniques - Notes

2.      Reinforcement learning (RL) – Chapters 2, 7 and 8 of PT (estimated 2 weeks)

3.      Dynamic Programming (DP) and Approximate Dynamic Programming (ADP) – Chapters 1, 4, 5 and 19 of PT, Chapter 1 of RR – 2. (estimated 1 week)

4.      Backpropagation Through Time (BPTT) [RR -5] and Chapter 15 of PT (estimated 1 week).

5.      Adaptive critics: Class of Adaptive Critics – Chapters 3, 4, 5 and 19 of PT (estimated 3 weeks)

a.       Heuristic Dynamic Programming (HDP) and Action Dependent HDP

b.      Dual Heuristic Programming (DHP)

c.       Global Dual Heuristic Programming (GDHP)

6.      Case studies on RL and ACDs –  Control Problems, Communication Systems, Power Systems – Chapters 3, 4, 5, 10, 18, 19 and 20 of PT, RR-6. In addition, any other publications relevant to case study will be used. (estimated 2 weeks)

 

 

Recommended Reading (RR)

1.      Handbook of Intelligent Control – Neural, Fuzzy and Adaptive Approaches, Eds. White and Sofge, Van Nostrand Reinhold, New York, 1992, ISBN 0-442-30857-4.

2.      D Bertsekas, Dynamic Programming: Deterministic and Stochastic Models, Prentice Hall, ISBN 0132215810.

3.      D Bertsekas, Dynamic Programming and Optimal Control, Vols. I and II, Athena Scientific, 1995, (2nd Edition Vol. I, 2000, 2nd Edition Vol. II, 2001).

4.      D Bertsekas and J Tsitsiklis, Neuro-Dynamic Programming, Athena Scientific, 1996, ISBN: 1-886529-10-8.

5.       PJ Werbos, “Roots of Backpropagation”, Wiley, ISBN 0-471-59897-6, 1994.

6.      “Adaptive Critic Based Optimal Neurocontrol for Synchronous Generator in Power System Using MLP/RBF Neural Networks”, IEEE Transactions on Industry Applications, vol. 39, no. 5, September/October 2003, pp. 1529-1540.

 

 

Prerequisites

Computational Intelligence (CpEng/EE/ME/Sys Eng 301 – known referred to as CpEng 358/EE 367) or EE 368 Neural Networks or Permission of the Instructor.

 

Project

Projects will be carried out individually. Topics will be decided in consultation with the instructor or assigned by the instructor. Projects will involve the application of adaptive critic designs.

 

Please email any questions to Dr. G K Venayagamoorthy at ganeshv@mst.edu