I sat down with @arimorcos and @_RobToews for our recurring AI vibe check. We got into the tenuous future of open source models, the latest in the lab wars and some really interesting future implications of the compute crunch. It's been far too long since we did one of these and I always have fun jamming with these two. This time we hit on: ▪️ Why near-frontier open weight AI may be disappearing ▪️ Will compute constraints push labs to suspend their own APIs ▪️ Frustrations around Fable ▪️ OpenAI’s future ▪️ Where we are with Recursive Self Improvement and its implications We also hit on Cursor/xAI, ASML competitors and a ton more. 0:00 Intro 1:40 Coding Agents Cross a Threshold 3:29 Is Open-Weight AI in Retreat? 7:37 Cost Crunch & Scaffolding 12:13 The "Apps Are Cooked" Debate 16:37 Sam Altman Under Scrutiny 19:44 Anthropic's Fable Backlash 23:24 How Big a Step Change Is Fable? 26:50 What's Going On at Google? 33:20 Could the APIs Go Away? 34:11 Breaking the Semiconductor Bottleneck 35:42 Beyond EUV: Atom & X-Ray Lithography 37:23 Implications of a Compute Shortage 40:20 Do Alt Chips Actually Help? 43:43 SpaceX, xAI & the Cursor Acquisition 48:50 How Close Are We to RSI? 52:21 Quickfire YouTube: youtu.be/W_iO8XxgD_I Spotify: bit.ly/4envbk8 Apple: bit.ly/4eoD4FT

I sat down with @lukaszkaiser to get into whether the architecture he helped invent is actually enough, and what's next in generalization, coding agents, RL and more. Lukasz co-authored "Attention Is All You Need," the paper that introduced the transformer and worked on reasoning models at OpenAI so he’s been a key part of major shifts in the field. We hit on: ▪️ The case for and against a new architecture coming after the transformer ▪️ What’s required for model generalization in the physical world ▪️ How much coding agents have improved his AI research productivity ▪️ The next domains for RL ▪️ Why Anthropic initially won coding ▪️ Future research directions he’s excited about 0:00 Intro 1:12 Transformers vs. Human Learning 8:37 How Do We Get Physical World Generalization? 10:52 What Comes After Transformers 13:59 How Much Have Agents Improved Lukasz's AI Research Productivity? 17:21 How Close Is an AI Research Intern? 26:06 RL Beyond Verifiable Tasks 35:38 App Companies: Build Models or Lean on Labs? 46:21 Multimodal Is Still Missing Something 49:46 OpenAI's Bet on Reasoning 55:26 The AI Coding Wars 59:26 Focus vs. Keeping Embers Burning 1:02:09 Open Source vs. Closed Source Gap 1:05:15 Quickfire YouTube: youtu.be/N1geOimmdDo Spotify: bit.ly/4foudX0 Apple: bit.ly/4uGUhkO

I sat down with @lukaszkaiser to get into whether the architecture he helped invent is actually enough, and what's next in generalization, coding agents, RL and more. Lukasz co-authored "Attention Is All You Need," the paper that introduced the transformer and worked on reasoning models at OpenAI so he’s been a key part of major shifts in the field. We hit on: ▪️ The case for and against a new architecture coming after the transformer ▪️ What’s required for model generalization in the physical world ▪️ How much coding agents have improved his AI research productivity ▪️ The next domains for RL ▪️ Why Anthropic initially won coding ▪️ Future research directions he’s excited about 0:00 Intro 1:12 Transformers vs. Human Learning 8:37 How Do We Get Physical World Generalization? 10:52 What Comes After Transformers 13:59 How Much Have Agents Improved Lukasz's AI Research Productivity? 17:21 How Close Is an AI Research Intern? 26:06 RL Beyond Verifiable Tasks 35:38 App Companies: Build Models or Lean on Labs? 46:21 Multimodal Is Still Missing Something 49:46 OpenAI's Bet on Reasoning 55:26 The AI Coding Wars 59:26 Focus vs. Keeping Embers Burning 1:02:09 Open Source vs. Closed Source Gap 1:05:15 Quickfire YouTube: youtu.be/N1geOimmdDo Spotify: bit.ly/4foudX0 Apple: bit.ly/4uGUhkO

I sat down with @lukaszkaiser to get into whether the architecture he helped invent is actually enough, and what's next in generalization, coding agents, RL and more. Lukasz co-authored "Attention Is All You Need," the paper that introduced the transformer and worked on reasoning models at OpenAI so he’s been a key part of major shifts in the field. We hit on: ▪️ The case for and against a new architecture coming after the transformer ▪️ What’s required for model generalization in the physical world ▪️ How much coding agents have improved his AI research productivity ▪️ The next domains for RL ▪️ Why Anthropic initially won coding ▪️ Future research directions he’s excited about 0:00 Intro 1:12 Transformers vs. Human Learning 8:37 How Do We Get Physical World Generalization? 10:52 What Comes After Transformers 13:59 How Much Have Agents Improved Lukasz's AI Research Productivity? 17:21 How Close Is an AI Research Intern? 26:06 RL Beyond Verifiable Tasks 35:38 App Companies: Build Models or Lean on Labs? 46:21 Multimodal Is Still Missing Something 49:46 OpenAI's Bet on Reasoning 55:26 The AI Coding Wars 59:26 Focus vs. Keeping Embers Burning 1:02:09 Open Source vs. Closed Source Gap 1:05:15 Quickfire YouTube: youtu.be/N1geOimmdDo Spotify: bit.ly/4foudX0 Apple: bit.ly/4uGUhkO

I sat down with @lukaszkaiser to get into whether the architecture he helped invent is actually enough, and what's next in generalization, coding agents, RL and more. Lukasz co-authored "Attention Is All You Need," the paper that introduced the transformer and worked on reasoning models at OpenAI so he’s been a key part of major shifts in the field. We hit on: ▪️ The case for and against a new architecture coming after the transformer ▪️ What’s required for model generalization in the physical world ▪️ How much coding agents have improved his AI research productivity ▪️ The next domains for RL ▪️ Why Anthropic initially won coding ▪️ Future research directions he’s excited about 0:00 Intro 1:12 Transformers vs. Human Learning 8:37 How Do We Get Physical World Generalization? 10:52 What Comes After Transformers 13:59 How Much Have Agents Improved Lukasz's AI Research Productivity? 17:21 How Close Is an AI Research Intern? 26:06 RL Beyond Verifiable Tasks 35:38 App Companies: Build Models or Lean on Labs? 46:21 Multimodal Is Still Missing Something 49:46 OpenAI's Bet on Reasoning 55:26 The AI Coding Wars 59:26 Focus vs. Keeping Embers Burning 1:02:09 Open Source vs. Closed Source Gap 1:05:15 Quickfire YouTube: youtu.be/N1geOimmdDo Spotify: bit.ly/4foudX0 Apple: bit.ly/4uGUhkO

I sat down with @lukaszkaiser to get into whether the architecture he helped invent is actually enough, and what's next in generalization, coding agents, RL and more. Lukasz co-authored "Attention Is All You Need," the paper that introduced the transformer and worked on reasoning models at OpenAI so he’s been a key part of major shifts in the field. We hit on: ▪️ The case for and against a new architecture coming after the transformer ▪️ What’s required for model generalization in the physical world ▪️ How much coding agents have improved his AI research productivity ▪️ The next domains for RL ▪️ Why Anthropic initially won coding ▪️ Future research directions he’s excited about 0:00 Intro 1:12 Transformers vs. Human Learning 8:37 How Do We Get Physical World Generalization? 10:52 What Comes After Transformers 13:59 How Much Have Agents Improved Lukasz's AI Research Productivity? 17:21 How Close Is an AI Research Intern? 26:06 RL Beyond Verifiable Tasks 35:38 App Companies: Build Models or Lean on Labs? 46:21 Multimodal Is Still Missing Something 49:46 OpenAI's Bet on Reasoning 55:26 The AI Coding Wars 59:26 Focus vs. Keeping Embers Burning 1:02:09 Open Source vs. Closed Source Gap 1:05:15 Quickfire YouTube: youtu.be/N1geOimmdDo Spotify: bit.ly/4foudX0 Apple: bit.ly/4uGUhkO

I sat down with @lukaszkaiser to get into whether the architecture he helped invent is actually enough, and what's next in generalization, coding agents, RL and more. Lukasz co-authored "Attention Is All You Need," the paper that introduced the transformer and worked on reasoning models at OpenAI so he’s been a key part of major shifts in the field. We hit on: ▪️ The case for and against a new architecture coming after the transformer ▪️ What’s required for model generalization in the physical world ▪️ How much coding agents have improved his AI research productivity ▪️ The next domains for RL ▪️ Why Anthropic initially won coding ▪️ Future research directions he’s excited about 0:00 Intro 1:12 Transformers vs. Human Learning 8:37 How Do We Get Physical World Generalization? 10:52 What Comes After Transformers 13:59 How Much Have Agents Improved Lukasz's AI Research Productivity? 17:21 How Close Is an AI Research Intern? 26:06 RL Beyond Verifiable Tasks 35:38 App Companies: Build Models or Lean on Labs? 46:21 Multimodal Is Still Missing Something 49:46 OpenAI's Bet on Reasoning 55:26 The AI Coding Wars 59:26 Focus vs. Keeping Embers Burning 1:02:09 Open Source vs. Closed Source Gap 1:05:15 Quickfire YouTube: youtu.be/N1geOimmdDo Spotify: bit.ly/4foudX0 Apple: bit.ly/4uGUhkO

.@scmallaby spent 30 hours with Demis Hassabis in the back of a British pub. I sat down with Sebastian to chat about his recent book, The Infinity Machine, where he provides a fascinating history of DeepMind, Demis and the broader AI landscape. We hit on: ▪️ Whether the race between labs was inevitable ▪️ Project Mario: the secret plan to spin DeepMind out of Google ▪️ Why Google missed scaling the transformer despite inventing it and wasn’t first to ChatGPT and Claude Code ▪️ The personal history between Demis, Elon and Sam ▪️ The quasi-spiritual conviction Sebastian didn't expect to find underneath the science 0:00 Intro 2:04 Was the AI Race Inevitable? 4:03 The 2015 Safety Summit Backfire 11:27 Why Google Doesn't Make As Concentrated Bets 15:51 Project Mario: The Secret Spinout Plan 19:43 What Demis Actually Regrets 23:46 Venture Startups vs. Tech Behemoths 34:08 David Silver and the RL True Believers 38:21 Demis, Elon, and the Evil Genius Feud 42:39 Great Man Theory vs. Inevitability 45:00 What Demis Didn't Want Published YouTube: youtu.be/WC_embCiwgU Spotify: bit.ly/4wRqvLE Apple: bit.ly/4dDnnvq

.@scmallaby spent 30 hours with Demis Hassabis in the back of a British pub. I sat down with Sebastian to chat about his recent book, The Infinity Machine, where he provides a fascinating history of DeepMind, Demis and the broader AI landscape. We hit on: ▪️ Whether the race between labs was inevitable ▪️ Project Mario: the secret plan to spin DeepMind out of Google ▪️ Why Google missed scaling the transformer despite inventing it and wasn’t first to ChatGPT and Claude Code ▪️ The personal history between Demis, Elon and Sam ▪️ The quasi-spiritual conviction Sebastian didn't expect to find underneath the science 0:00 Intro 2:04 Was the AI Race Inevitable? 4:03 The 2015 Safety Summit Backfire 11:27 Why Google Doesn't Make As Concentrated Bets 15:51 Project Mario: The Secret Spinout Plan 19:43 What Demis Actually Regrets 23:46 Venture Startups vs. Tech Behemoths 34:08 David Silver and the RL True Believers 38:21 Demis, Elon, and the Evil Genius Feud 42:39 Great Man Theory vs. Inevitability 45:00 What Demis Didn't Want Published YouTube: youtu.be/WC_embCiwgU Spotify: bit.ly/4wRqvLE Apple: bit.ly/4dDnnvq

.@scmallaby spent 30 hours with Demis Hassabis in the back of a British pub. I sat down with Sebastian to chat about his recent book, The Infinity Machine, where he provides a fascinating history of DeepMind, Demis and the broader AI landscape. We hit on: ▪️ Whether the race between labs was inevitable ▪️ Project Mario: the secret plan to spin DeepMind out of Google ▪️ Why Google missed scaling the transformer despite inventing it and wasn’t first to ChatGPT and Claude Code ▪️ The personal history between Demis, Elon and Sam ▪️ The quasi-spiritual conviction Sebastian didn't expect to find underneath the science 0:00 Intro 2:04 Was the AI Race Inevitable? 4:03 The 2015 Safety Summit Backfire 11:27 Why Google Doesn't Make As Concentrated Bets 15:51 Project Mario: The Secret Spinout Plan 19:43 What Demis Actually Regrets 23:46 Venture Startups vs. Tech Behemoths 34:08 David Silver and the RL True Believers 38:21 Demis, Elon, and the Evil Genius Feud 42:39 Great Man Theory vs. Inevitability 45:00 What Demis Didn't Want Published YouTube: youtu.be/WC_embCiwgU Spotify: bit.ly/4wRqvLE Apple: bit.ly/4dDnnvq

.@scmallaby spent 30 hours with Demis Hassabis in the back of a British pub. I sat down with Sebastian to chat about his recent book, The Infinity Machine, where he provides a fascinating history of DeepMind, Demis and the broader AI landscape. We hit on: ▪️ Whether the race between labs was inevitable ▪️ Project Mario: the secret plan to spin DeepMind out of Google ▪️ Why Google missed scaling the transformer despite inventing it and wasn’t first to ChatGPT and Claude Code ▪️ The personal history between Demis, Elon and Sam ▪️ The quasi-spiritual conviction Sebastian didn't expect to find underneath the science 0:00 Intro 2:04 Was the AI Race Inevitable? 4:03 The 2015 Safety Summit Backfire 11:27 Why Google Doesn't Make As Concentrated Bets 15:51 Project Mario: The Secret Spinout Plan 19:43 What Demis Actually Regrets 23:46 Venture Startups vs. Tech Behemoths 34:08 David Silver and the RL True Believers 38:21 Demis, Elon, and the Evil Genius Feud 42:39 Great Man Theory vs. Inevitability 45:00 What Demis Didn't Want Published YouTube: youtu.be/WC_embCiwgU Spotify: bit.ly/4wRqvLE Apple: bit.ly/4dDnnvq

Excited to announce @getgarner has raised $100M at a $2.74B valuation. 2.5M beneficiaries. 800 employers. $1B saved for our customers. 12% lower plan costs on average. With this funding, we continue to build the new AI front door to healthcare - starting with two new features that we are announcing today that transform our member experience and accelerate our understanding of clinical performance: Garner Assistant and Garner Research Agent. Grateful to @ikeswetlitz and @Bloomberg for covering our story! bloomberg.com/news/articles/…