Introduction to Imitation Learning

This presentation surveys imitation learning (IL) as learning policies from expert demonstrations, contrasting it with reinforcement learning and standard supervised learning. It clarifies structured (plan-all-at-once) versus sequential (closed-loop) decision making, then introduces Behavioral Cloning and explains distribution shift and compounding errors, motivating data aggregation methods like DAgger/Data-as-Demonstrator and confidence-aware approaches such as CAIL (inner imitation loss vs. outer performance loss). The deck connects IL to inverse reinforcement learning (IRL), framing IRL as recovering the expert’s latent reward and covering key variants including Bayesian IRL (Ramachandran & Amir) and Maximum Causal Entropy IRL. A concise medical EHR case study (Diabetes IRL) illustrates how rewards can be inferred offline and used to analyze or derive policies. Throughout, the emphasis is on when to prefer simple BC versus reward-learning methods, practical data requirements, and how to operate in fully offline settings.