A year ago, I predicted that we would enter The Era of Evals, but it's now happening much faster than I anticipated. Frontier Labs have scaled their Eval production with us by more than 10X in the last 12 months, on what was already a 9-figure base. Every tech-forward enterprise is rapidly building evals for its agents. @mercor_ai is spending over $10M / month on inference and now has Evals for every agent deployment. We need Evals to know (1) what model to use, (2) what context or tools we should add to improve the model, and (3) whether it's working in production. One CTO of a $25B enterprise told me he used to have a product roadmap, but replaced his entire product roadmap with an Eval roadmap.

lots more of this coming

excited for this one

A day later and I feel even more confident in this: Siri is going to be the AI that most consumers end up using most of the time (if they have an iPhone). It's the AI you have with you, with access to everything. And yes, it's finally good enough. spyglass.org/siri-ai/

We partnered with @trajectorylabs to post-train NVIDIA Nemotron 3 Ultra for legal. Here’s what we found: 1) Open-weight models can reach frontier legal performance. On our Legal Agent Benchmark (LAB), Nemotron 3 Ultra started at a 0% all-pass rate. After post-training, it reached 5.8%, placing it between Sonnet 4.6 at 4.2% and Opus 4.6 at 6.6%. 2) Post-training dramatically improves reliability. Before training, many held-out tasks missed enough rubric dimensions to land around ~70% pass rates. After training, those tasks shifted toward ~95% pass rates. 3) Open-weight performance comes at much lower cost. Post-trained Nemotron 3 Ultra reached a similar quality band to leading closed models while running at roughly 1/8th to 1/50th the per-token price of Sonnet 4.6 and Opus 4.6. Most importantly: we post-trained this model on the @trajectorylabs platform less than 24 hours after Nemotron 3 Ultra launched, using the same harness, data, and recipe we used for Nemotron 3 Super. More to come as we continue to experiment with open-weight legal agents. Read more on post-training with Trajectory below:

Fable is a very impressive model. But it is also - Expensive - Slow - Rate limited - Nerfed for important R&D tasks - Not private (retains prompts/responses even for enterprise users) The difference between owned and rented intelligence gets clearer by the day.

incredible

such an incredible read excited to see what rob cooks up

This new research on US unicorn startups is really interesting. Some key facts from the report: 1. Immigrants founded or cofounded 455 of America’s 775 privately held billion-dollar startups, equal to 59% of all US unicorns. 2. 66% of all US unicorns were founded or cofounded by immigrants or the children of immigrants. 3. 79% of US unicorns have either an immigrant founder or an immigrant in a key leadership role. 4. The 455 immigrant-founded US unicorns have a combined valuation of $5 trillion. 5. That $5 trillion valuation is larger than the total stock-market value of companies listed in all but 7 countries. 6. Including immigrant-founded unicorns that went public since 2016 pushes the total value above $5.8 trillion. 7. The number of immigrant-founded US unicorns rose from 50 in 2018 to 455 in 2026. 8. 24% of US unicorns have a founder who first came to America as an international student.

Adding Harvey to the list of app layer companies, joining Ramp, Sierra, Decagon, etc, who are devoting some level of dedicated effort to join Cursor in approaching positive gross margins decoupling themselves from frontier model providers as the marginal cost of post training goes down. The marginal unit of intelligence's value from the next model release, while being valuable, is being supplanted by harness-level differences in model performance, increasing ability to traverse the performance-cost-latency pareto curve with post-training infra advancements, and the simple fact that when one runs practical benchmarks (and not all the toy benchmarks around nowadays), GLM and latest MiniMax are at Parity or exceed Frontier Models on an absolute basis on many tasks (more on this in state of data May, a bit delayed). Ofc ik Harvey has been toying with rlaas vendors for a while and their finetuning efforts pre big rl wave weren't incredibly well received, but I generally find that most app layer ai companies with some elite engineering talent will be seriously exploring post training their own small models, at least in conjunction with systems that use frontier models as above head orchestrators. That some of them reach out to me on advice for procuring rl datasets from rl env companies is reifying evidence of that.

macOS California Release Location Map

congrats nick!

We’re open-sourcing Stem Studio, our 3JS game engine today. This is a browser-based 3D multiplayer game engine and dev studio based on the idea that game dev should become more open, remixable, and web-native. AI will make it easier to create games. But shared building blocks will make it easier for developers to build on top of each other. Stem Studio is MIT licensed, JavaScript-based, and built for browser multiplayer 3D worlds. Code is here: buildwithstem.com Fork it, break it, remix it, and show us what you make.

Demis Hassabis, Elon Musk, and Jensen Huang all have one thing in common: THEY LOVE VIDEO GAMES. If you don't play video games growing up, you're NGMI.

What happened to TBPN? seems to have vanished from tech twitter…

the future of journalism vibe code the news

congrats @Bencera !

does this mean Tirzepatide isn’t any better — it’s really all dose dependent?

We’ve spent a lot of time on the framework underneath Codex, so it can move quickly on routine work while stopping for review when the risk changes. Here’s how we use sandboxing, approvals, network policy, and telemetry to run Codex safely @OpenAI: openai.com/index/running-cod…