Stanford CS234: Reinforcement Learning | Winter 2019 | Lecture 7 – Imitation Learning

For more information about Stanford’s Artificial Intelligence professional and graduate programs, visit: https://stanford.io/ai

Professor Emma Brunskill, Stanford University
http://onlinehub.stanford.edu/

Professor Emma Brunskill
Assistant Professor, Computer Science
Stanford AI for Human Impact Lab
Stanford Artificial Intelligence Lab
Statistical Machine Learning Group

To follow along with the course schedule and syllabus, visit: http://web.stanford.edu/class/cs234/index.html


0:00 Introduction
2:15 Recap: DQN (Mnih et al. Nature 2015)
4:01 Recap: Deep Model-free RL, 3 of the Big Ideas
4:16 Recap: Double DQN
11:19 Recap: Prioritized Experience Replay
12:20 Dueling Background: Value & Advantage Function
16:13 Dueling DQN V.S. Double DON with Prioritized Replay
17:39 Deep Reinforcement Learning
19:56 Generalization and Efficiency
21:27 Class Structure
21:36 Consider Montezuma’s revenge
23:32 Reward Shaping
25:28 Learning from Demonstrations
26:23 Problem Setup
29:43 Behavioral Cloning
35:46 Problem: Compounding Errors
38:13 DAGGER: Dataset Aggregation
43:06 FeatureBased Reward Function
53:58 Linear Feature Reward Inverse RL
56:42 Feature Matching