ForeCast

Lukas Finnveden is a Research Analyst at Redwood Research. We talk about making deals with early scheming AIs: paying them to cooperate rather than take over. You can read a full transcript here. To see all Forethought’s published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.

Rose Hadshar (Forethought) and Nora Ammann (ARIA) talk about power concentration, checks and balances, coups, and post-AGI political economy. You can read a full transcript here. This is the audio version of a video podcast. Watch the video on YouTube: youtu.be/PB855Xpx1kk?si=v05LVM-2VQ3XiT7N Prompted in part by Rose's recent article for 80,000 Hours on ‘Extreme power concentration’ — 80000hours.org/problem-profiles/extreme-power-concentration Timstamps: 00:00:00 Intro 00:01:26 What is ‘power concentration’? Is power concentration the right framing? 00:05:25 When can it be good for power to be centralised? 00:09:40 Is ‘checks and balances’ a better framing? 00:11:16 How AI undermines existing checks and balances 00:15:25 Economic power, meme complexes, and cultural influence 00:28:50 AI companies vs governments as centres of power 00:31:25 The difficulty of knowing where power actually lies 00:40:05 Do humans and AIs concentrate power differently? 00:48:38 Should we be trying to imagine a better post-AGI political economy? 00:56:28 Concrete actions: transparency & whistleblower protections 00:59:07 Human-AI teaming and collaborative intelligence 01:01:16 AI agent economies and proving trustworthiness 01:13:16 Can we train AI to follow the law? 01:16:33 Building resilient coalitions of trustworthy agents 01:18:42 Closing reflections

Joe Carlsmith is a writer, researcher, and philosopher. He works at Anthropic on the character/constitution/spec for Claude. Before that, he was a senior advisor at Open Philanthropy. You can read a full transcript here. To see all Forethought’s published research, visit forethought.org/research. To subscribe to our newsletter, visit forethought.org/subscribe.

This is a bonus episode to say that Forethought is hiring researchers. After an overview of the roles, we hear from Research Fellow Mia Taylor about working at Forethought. You can read a full transcript here. The application deadline has been extended to November 1st 2025. Apply here: forethought.org/careers/researcher Chapters (00:00:00) Forethought hiring overview and roles (00:03:21) Interview with Mia begins (00:03:34) Why Mia joined Forethought (00:05:59) Daily work, how work at Forethought was different from expected (00:08:38) Research examples (00:10:58) Who should and shouldn't apply (00:14:11) Disagreements and closing thoughts Links Apply for researcher positions Referral form (if you know someone who might be a good fit)

This is a narration of ‘Introducing Better Futures ’ by William MacAskill; published 3rd August 2025. Narration by Perrin Walker (@perrinjwalker).