🫡 tons of respect for Teddy Collins et al!

Nathan's a real OG of AI investing, is friends with a bunch of legit researchers, and did a lot to help me during the Adept days. Respect!

apparently this is what the chinese AI ecosystem thinks our grand american AI master plan is! (this is not a joke! forwarded to me by an attendee at a real chinese ai conference!)

Excited to release PostTrainBench v1.0! This benchmark evaluates the ability of frontier AI agents to post-train language models in a simplified setting. We believe this is a first step toward tracking progress in recursive self-improvement 🧵:

I’ll be leaving Amazon at the end of this week to cook up something new! Thanks to the Adept deal, I’ve spent the last ~2 years learning from @ajassy et al while leading Amazon’s agents R&D effort and our San Francisco AI lab. As a childhood EC2 Micro Instance fanboy, it was fun to speedrun launching our own tier-1 AWS service. We scaled up the Adept agent recipes, did new RL research, and shipped it to AWS customers like Hertz, 1Password, and Amazon.com itself. And it's cool to see Nova Act on top of realevals.xyz (at least for now). There’s incredible work to be done at Amazon and I'm grateful for the opportunities to take on more here. But with AGI so close, I want to spend 100% of my time on teaching AI systems brand new capabilities. At OpenAI, I was lucky to incubate the first GPTs; at Adept, we went all-in on agents before anyone else–our tech/people now drive computer-use efforts at every major lab. I have a bet for what's next. ;) This wasn't an easy decision, and I'm sad to leave this wonderful team. I’m grateful for the trust our execs placed in me during an important moment for Amazon and the field. I'm excited to swing at the next idea!

Reiner is perhaps the strongest chip LLM person I have ever gotten to work with, and he and @MikeGunter_ have built just as legit of a team. I'm stoked to have been an investor in an early round. Tapeout in under a year! 😅

Tried to maximize hot takes per unit time -- including one that @HarryStebbings asked @Benioff about ;) Really fun talking with Alex!

Stoked about the first release from our new lab: our browser use agent lets you MapReduce over the web! This early preview moves us closer to reliable agents that learn from rewards across a wide range of digital and physical environments. Love our Adept Amazon team so much!

Incredible what happens when you bring the existing world knowledge and intuition of VLMs to the physical world... only a few more missing pieces in the recipe for AGI... really feeling the agi with this one

I’m excited to announce Tolan, our first Embodied Companion. With no launch or press we’ve quietly hit 500,000 downloads, over $1m in ARR, and a #1 app store category ranking. Today I’m also announcing our $10m seed round (more on that below) and sharing some of what we’ve learned building an AI companion for consumers.

1. Breakdown of DeepSeek V3 efficiency vs Llama 3: - Better: 11x fewer FLOPs per token, thanks to MoE [37B vs 405B activated params] - Better: 2x faster numerics [fp8 vs bf16 training] - Worse: 0.5x flops utilization [16% vs 33% end-to-end MFU*] - Neutral: similar hardware platform [H800 and H100 both have 2Pflops/s dense fp8] - Neutral: same training data volume [14.8T vs 15T tokens] Llama 3’s design was obviously and intentionally conservative: dense model (not MoE), bf16 training (not fp8), GQA attention (not cheaper alternatives). DeepSeek benefited by being aggressive on all these fronts, at the cost of being later to market. 2. The core algorithmic improvements were already known; the closed source LLM labs were probably already doing similar things. DeepSeek’s improvements are real, but far more modest than the Llama comparison would suggest; my wild guess is closer to 1.5x improvement. MoE was published in 2017; in 2021 Switch Transformer reported 7x speedups vs dense models, similar to DeepSeek’s 11x. OpenAI is widely rumored to have been using MoE models for years. NVIDIA published their fp8 training paper in 2022. 3. NVIDIA’s stock price is down 15% after DeepSeek. Should it be? LLM compute is like a gas: it expands to fill the available budget. Over the last 3 years the labs have grown their budgets, despite algorithms and hardware improving. There’s no reason to expect this to change now: you win by making the best model, not by shrinking your budget. The more meaningful question: do algorithmic improvements like DeepSeek’s mean that margins will shift from hardware vendors to labs? Hard to see why. Algorithmic improvements are quickly copied from one lab to another, making it hard for them to maintain technological differentiation. Hardware improvements take much longer to copy.

new sf ai lab at amazon led by @jluan (ex-adept) and @pabbeel (ex-covariant/berkeley)! exciting to see :)

Ending Twitter hibernation to congratulate @mnaficy and team on @arcade_ai's launch--I've loved playing with it, turning natural language into physical objects. First product shape I've ever seen that could only be possible with diffusion models! Proud to be a (small) backer!