Reinforcement Learning
From MDPs and dynamic programming to DQN, PPO and multi-agent systems — the complete guide to agents that learn from outcomes.
What Is Reinforcement Learning?
Agents, environments, rewards and the trial-and-error loop that powers RL.
Markov Decision Processes
The formal framework: states, actions, transitions, rewards and the Markov property.
Dynamic Programming
Solve MDPs exactly with policy evaluation, policy iteration and value iteration.
Monte Carlo Methods
Learn from complete episodes: average returns to estimate state values without a model.
Temporal Difference Learning
Learn from partial episodes with bootstrapping — the idea at the heart of modern RL.
Q-Learning
Off-policy control: learn action values with Q-learning and watch agents improve from scratch.
Exploration vs Exploitation
Balance trying new things with using what works: epsilon-greedy, UCB and optimism.
Function Approximation
Scale RL beyond tables: approximate Q-values with linear and neural models.
Deep Q-Networks (DQN)
Play Atari from pixels: experience replay, target networks and the tricks that made DQN work.
Policy Gradient Methods
Optimize the policy directly — REINFORCE and the log-probability trick.
Actor-Critic Methods
Two networks in harmony: the actor picks actions, the critic judges them with bootstrapped values.
PPO & Modern Policy Optimization
Stable, sample-efficient updates with clipping — the default choice in modern RL.
Reward Design
Rewards shape everything: sparse vs dense, shaping and the specification problem.
Offline Reinforcement Learning
Learn policies from logged data without a live environment — conservative and practical.
Imitation Learning
Learn from demonstrations: behavioral cloning and DAgger for expert-like behavior.
Multi-Agent Reinforcement Learning
Multiple learners interacting: cooperation, competition, and the games they create.
RL in Games
From Atari to AlphaGo: self-play, Monte Carlo tree search, and superhuman play.
RL for Robotics
Physical agents learning motor skills: sim-to-real, reward shaping and safety in the real world.
RL for Recommender Systems
Ranking as sequential decisions: contextual bandits and long-term user value.
Evaluating RL Agents
Measures beyond total reward: returns curves, sample efficiency and robustness.
Reinforcement Learning Roadmap
Synthesize the course, pick a domain, and plan RL projects that build real skill.