Here is our current plan for OpenAI: openai.com/index/built-to-be…

It is actually quite hard to make good evals… More than half the benchmarks you see online are pure slop.

Hi, I'm Wyatt Thompson, a member of the Human Data team at OpenAI. We recently had the chance to chat privately with Tyler Cowen and Alex Tabarrok about the impact of AI on the future of labor. One theme stood out: economic growth matters. It drives opportunity, innovation, and human flourishing. Here's the conversation:

cdn.openai.com/papers/22265b… I often think about a surprise takeaway of PaperBench being that many academic papers are not reproducible even by humans (which makes them bad evals)—the data quality required for AI research at times surpasses academic peer review!