Top AI Professor Has 85% P(Doom) — David Duvenaud, Fmr. Anthropic Safety Team Lead
David Duvenaud is a professor of Computer Science at the University of Toronto, co-director of the Schwartz Reisman Institute for Technology and Society, former Alignment Evals Team Lead at Anthropic, an award-winning machine learning researcher, and a close collaborator of Dr. Geoffrey Hinton. He recently co-authored Gradual Disempowerment.We dive into David’s impressive career, his high P(Doom), his recent tenure at Anthropic, his views on gradual disempowerment, and the critical need for improved governance and coordination on a global scale.00:00 Introducing David03:03 Joining Anthropic and AI Safety Concerns35:58 David’s Background and Early Influences45:11 AI Safety and Alignment Challenges54:08 What’s Your P(Doom)™01:06:44 Balancing Productivity and Family Life01:10:26 The Hamming Question: Are You Working on the Most Important Problem?01:16:28 The PauseAI Movement01:20:28 Public Discourse on AI Doom01:24:49 Courageous Voices in AI Safety01:43:54 Coordination and Government Role in AI01:47:41 Cowardice in AI Leadership02:00:05 Economic and Existential Doom02:06:12 Liron’s Post-ShowShow NotesDavid’s Twitter — https://x.com/DavidDuvenaudSchwartz Reisman Institute for Technology and Society — https://srinstitute.utoronto.ca/Jürgen Schmidhuber’s Home Page — https://people.idsia.ch/~juergen/Ryan Greenblatt's LessWrong comment about a future scenario where there's a one-time renegotiation of power and heat from superintelligent AI projects causes the oceans to boil: https://www.lesswrong.com/posts/pZhEQieM9otKXhxmd/gradual-disempowerment-systemic-existential-risks-from?commentId=T7KZGGqq2Z4gXZstyWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com
--------
2:07:32
“AI 2027” — Top Superforecaster's Imminent Doom Scenario
AI 2027, a bombshell new paper by the AI Futures Project, is a highly plausible scenario of the next few years of AI progress. I like this paper so much that I made a whole episode about it.00:00 Overview of AI 202705:13 2025: Stumbling Agents16:23 2026: Advanced Agents21:49 2027: The Intelligence Explosion29:13 AI's Initial Exploits and OpenBrain's Secrecy30:41 Agent-3 and the Rise of Superhuman Engineering37:05 The Creation and Deception of Agent-544:56 The Race Scenario: Humanity's Downfall48:58 The Slowdown Scenario: A Glimmer of Hope53:49 Final ThoughtsShow NotesThe website: https://ai-2027.comScott Alexander’s blog: https://astralcodexten.comDaniel Kokotajlo’s previous predictions from 2021 about 2026: https://www.lesswrong.com/posts/6Xgy6CAf2jqHhynHL/what-2026-looks-likeWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com
--------
57:49
Top Economist Sees AI Doom Coming — Dr. Peter Berezin, BCA Research
Dr. Peter Berezin is the Chief Global Strategist and Director of Research at BCA Research, the largest Canadian investment research firm. He’s known for his macroeconomics research reports and his frequent appearances on Bloomberg and CNBC.Notably, Peter is one of the only macroeconomists in the world who’s forecasting AI doom! He recently published a research report estimating a “ more than 50/50 chance AI will wipe out all of humanity by the middle of the century”.00:00 Introducing Peter Berezin01:59 Peter’s Economic Predictions and Track Record05:50 Investment Strategies and Beating the Market17:47 The Future of Human Employment26:40 Existential Risks and the Doomsday Argument34:13 What’s Your P(Doom)™39:18 Probability of non-AI Doom44:19 Solving Population Decline50:53 Constraining AI Development53:40 The Multiverse and Its Implications01:01:11 Are Other Economists Crazy?01:09:19 Mathematical Universe and Multiverse Theories01:19:43 Epistemic vs. Physical Probability01:33:19 Reality Fluid01:39:11 AI and Moral Realism01:54:18 The Simulation Hypothesis and God02:10:06 Liron’s Post-ShowShow NotesPeter’s Twitter: https://x.com/PeterBerezinBCAPeter’s old blog — https://stockcoach.blogspot.comPeter’s 2021 BCA Research Report: “Life, Death and Finance in the Cosmic Multiverse” — https://www.bcaresearch.com/public/content/GIS_SR_2021_12_21.pdfM.C. Escher’s “Circle Limit IV” — https://www.escherinhetpaleis.nl/escher-today/circle-limit-iv-heaven-and-hell/Zvi Mowshowitz’s Blog (Liron’s recommendation for best AI news & analysis) — https://thezvi.substack.comMy Doom Debates episode about why nuclear proliferation is bad — https://www.youtube.com/watch?v=ueB9iRQsvQ8Robin Hanson’s “Mangled Worlds” paper — https://mason.gmu.edu/~rhanson/mangledworlds.htmlUncontrollable by Darren McKee (Liron’s recommended AI x-risk book) — https://www.amazon.com/dp/B0CNNYKVH1Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com
--------
2:14:58
AI News: GPT-4o Images, AI Unemployment, Emmett Shear's New Safety Org — with Nathan Labenz
Nathan Labenz, host of The Cognitive Revolution, joins me for an AI news & social media roundup!00:00 Introducing Nate05:18 What’s Your P(Doom)™23:22 GPT-4o Image Generation40:20 Will Fiverr’s Stock Crash?47:41 AI Unemployment55:11 Entrepreneurship01:00:40 OpenAI Valuation01:09:29 Connor Leahy’s Hair01:13:28 Mass Extinction01:25:30 Is anyone feeling the doom vibes?01:38:20 Rethinking AI Individuality01:40:35 “Softmax” — Emmett Shear's New AI Safety Org01:57:04 Anthropic's Mechanistic Interpretability Paper02:10:11 International Cooperation for AI Safety02:18:43 Final ThoughtsShow NotesNate’s Twitter: https://x.com/labenzNate’s podcast: https://cognitiverevolution.ai and https://youtube.com/@CognitiveRevolutionPodcastNate’s company: https://waymark.com/Watch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Doom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com
--------
2:19:51
How an AI Doomer Sees The World — Liron on The Human Podcast
In this special cross-posted episode of Doom Debates, originally posted here on The Human Podcast, we cover a wide range of topics including the definition of “doom”, P(Doom), various existential risks like pandemics and nuclear threats, and the comparison of rogue AI risks versus AI misuse risks.00:00 Introduction01:47 Defining Doom and AI Risks05:53 P(Doom)10:04 Doom Debates’ Mission16:17 Personal Reflections and Life Choices24:57 The Importance of Debate27:07 Personal Reflections on AI Doom30:46 Comparing AI Doom to Other Existential Risks33:42 Strategies to Mitigate AI Risks39:31 The Global AI Race and Game Theory43:06 Philosophical Reflections on a Good Life45:21 Final ThoughtsShow NotesThe Human Podcast with Joe Murray: https://www.youtube.com/@thehumanpodcastofficialWatch the Lethal Intelligence Guide, the ultimate introduction to AI x-risk! https://www.youtube.com/@lethal-intelligencePauseAI, the volunteer organization I’m part of: https://pauseai.infoJoin the PauseAI Discord — https://discord.gg/2XXWXvErfA — and say hi to me in the #doom-debates-podcast channel!Don’t miss the other great AI doom show, For Humanity: https://youtube.com/@ForHumanityAIRiskDoom Debates’ Mission is to raise mainstream awareness of imminent extinction from AGI and build the social infrastructure for high-quality debate.Support the mission by subscribing to my Substack at https://doomdebates.com and to https://youtube.com/@DoomDebates This is a public episode. If you would like to discuss this with other subscribers or get access to bonus episodes, visit lironshapira.substack.com
Canada - english