🆕 The Full Story of Notion AI
latent.space/p/notion
We're so excited to chat with
@simonlast and
@sarahmsachs about Notion's "Token Town" - the crack team of AI Engineers and Model Behavior Engineers entrusted with building AI for Silicon Valley's most beloved knowledge work collaboration platform - and their latest launch of Custom Agents!
We talked:
• The full history of the 5 major rebuilds of Notion AI — and the key lessons from each
• How to eval agent *usefulness* not just correctness
• MCP vs CLI pros and cons
• What "work" looks like when agents are coworkers — why they build for the "top of the class" rather than dumb down AI for everyone
• Simon's take on the ideal "software factory" of the future
and so much more!
Timestamps:
00:00:00 Introduction and launching Notion Custom Agents
00:01:17 Why Notion rebuilt agents four or five times
00:03:35 Building for where models are going, not just where they are
00:05:32 The Agent Lab thesis, wrappers, and product intuition
00:08:07 User journeys, leadership, and low-ego AI teams
00:13:16 The Simon Vortex, hackathons, and bringing security in early
00:16:39 Team structure, demos over memos, and building for agents
00:20:25 Evals, Notion’s Last Exam, and the Model Behavior Engineer role
00:27:37 Evals as an agent harness and the changing role of software engineers
00:30:42 The software factory: specs, verification, and agent workflows
00:32:18 Live demo: a custom agent for coworking space applications
00:35:08 Composing agents, manager agents, and memory as pages
00:38:15 Notion Mail, Gmail, native integrations, and tools
00:39:43 MCP vs CLI and the cost of capability
00:44:13 When Notion uses MCP vs building its own integrations
00:47:43 The history of Notion’s agent harness rebuilds
00:55:35 Power users, public tools, and the setup agent
00:58:01 Self-fixing agents, permissions, and “flippy”
01:01:13 Pricing, credits, and choosing the right model automatically
01:09:01 Why Notion isn’t training its own frontier model
01:14:07 Retrieval, ranking, and search built for agents
01:17:27 Meeting Notes as data capture and workflow automation
01:21:18 Wearables, hardware, and Notion as the system of record
01:23:45 Outro