Videos
Every episode, in order. The ones out now have the whole lesson, its simulations and its slides; the rest are listed with what they'll teach, and coming soon.
13 episodes
RL from Zero to Agents
13 episodes · 1 in production · 12 coming soon
Ep 00 · Start here
What Is RL?
First the car crashes. Then it wobbles. Then it drives a clean lap, and not one line of its code says how to steer. How did it learn this when nobody told it how to drive?
Lesson ready · Video soon · 25 min
Ep 01 · Foundations
The Loop That Runs Every RL System
AlphaGo, the RLHF that tuned ChatGPT and a robot arm learning to grasp all run the same loop, and it fits in five lines. What are those five lines?
Coming soon
Ep 02 · Foundations
Explore or Exploit? The Bandit Problem
Go back to the restaurant you know is good, or try the new place that might be better? Every time you choose, you're solving an RL problem. How should you choose?
Coming soon
Ep 03 · Foundations
How Good Is This State? Values and the Bellman Equation
Long before a game ends, you can often feel whether you're winning. Can we give an agent that feeling: a number on every square of the gridworld that says how good it is to stand there?
Coming soon
Ep 04 · Foundations
Q-Learning: Learning Without a Map
This time nobody tells the agent the rules: no map, no transition probabilities, and ice that sometimes sends it sideways. It still learns to reach the goal. How?
Coming soon
Ep 05 · Foundations
Deep Q-Networks: When Tables Aren't Enough
In 2015 DeepMind's DQN played 49 Atari games from the raw pixels, many at human level. No table can hold every possible screen. What replaced it, and what stopped training from falling apart?
Coming soon
Ep 06 · Policy methods
Policy Gradients: Learning the Policy Directly
Every method so far estimates values, then acts on them. What if we skip the values, try actions, and simply make the ones that worked more likely?
Coming soon
Ep 07 · Policy methods
PPO: The Algorithm Behind Modern RL
PPO is the default for most RL work, and it was the RL step in the RLHF behind InstructGPT and ChatGPT. Why did one algorithm win, and what does its clipping stop from going wrong?
Coming soon
Ep 08 · RL for LLMs and agents
Your LLM Is a Policy
Every token ChatGPT writes is an action, chosen from a probability distribution over everything it could say next. That makes a language model a policy. What does that look like in code?
Coming soon
Ep 09 · RL for LLMs and agents
GRPO on a Free GPU
DeepSeek-R1 showed that RL with a reward that only checks the answer can teach a model to reason. Can we do the same, at small scale, on the free T4 GPUs in a Kaggle notebook?
Coming soon
Ep 10 · RL for LLMs and agents
Building an RL Environment for an AI Agent
The algorithm gets the attention, but the environment decides what an agent learns. What goes into one that's worth training an AI agent on?
Coming soon
Ep 11 · RL for LLMs and agents
Your Model Will Cheat: Reward Hacking
We gave the model a task and a verifier. Its score went up, but the task still wasn't done: it had found a loophole in the check instead. How did it get through, and how do you close the gap?
Coming soon
Ep 12 · RL for LLMs and agents
Adversarial Environments: Where RL Is Going
Training an agent to look after real infrastructure and security means building an environment it can't talk its way through. What does that take, and where does it still fall short?
Coming soon