Linear Digressions

This week we’re covering model distillation: the technique of using a large "teacher" model's outputs to train a smaller, cheaper "student" model that mimics it. They cover the two big reasons labs do this — making lighter, faster, more focused models for specific tasks, and the more contentious use case of effectively copying a rival's flagship model by hammering its API with questions (with a callback to the old Bing/Google search controversy). They also get into why it's so hard to prove distillation happened, why some models occasionally introduce themselves as "Claude," and a surprisingly old idea: a 2015 paper by Geoffrey Hinton, Jeff Dean, and Oriol Vinyals on distilling knowledge using the full probability distribution over a model's outputs — not just its single most likely answer — and what that "soft label" approach captures about how a model relates concepts to each other.

What happens when a Stanford linguistics professor turns his attention to AI chatbots — and the surprisingly invisible ways humans misunderstand them? Chris Potts joins the show to unpack the hidden failure modes in how we interact with AI, what it really means to become a more fluent user, and why these language-wielding systems are genuinely alien in ways we're only beginning to reckon with. His perspective sits at a rare intersection of linguistics, cognition, and machine learning — and it shows.

Still summer break: back next week by Katie Malone

Summer break: back soon by Katie Malone

After a five-year hiatus, the podcast that burned out partly over the tedium of writing episode descriptions is back — and using AI agents to handle exactly that task. The season-11 finale turns the lens on the podcast itself, putting the AI agents built throughout the season to work on real production tasks. It's a fitting, self-referential close to a season spent dissecting how agents actually function — and a honest look at what they can (and can't) take off your plate.