Super excited about this work! This paper was driven by a claim I've been making to anyone who'll listen: "interpretability is the language of data". (1/3)

There’s a lot of weird and surprising correlations in large scale datasets, this research helps unravel them to make posttraining less of a black magic

Neural networks do math by rotating shapes. We found a shape-rotating calculator hidden inside an LLM – and it’s used for more than just math! (1/6)

This work is extremely cool, LLMs have learned from raw text data an incredibly elegant geometrical algorithm and applied it in multiple different settings

Excited that Silico, the core platform behind our research results, is finally being announced! Here's one example of how I've used it: To interpret the EchoJEPA model and visualize different attribution methods onto a temporally aligned 3D heart mesh. More results to share on this soon!

Silico is a lab for AI scientists to dissect their own brains. It already runs autonomously for days, performing frontier AI research finding and fixing model pathologies.

if you have goblins in your model, silico will find them or your money back

loved the piece on interpretability in the @nytimes this morning! the field has accomplished some pretty cool things in recent years. still, there is much work to do as @davidbau put it elegantly, interpretability is now where biology was in 1930: “The cell was a black box for biologists. They were slow to get off the starting block to start studying heredity. But once they did, the problem fell.” there is so much we can learn from these alien intelligences.

At my last job, we often got calls from parents frantically asking for their child's genetic test results. Too often, the results were inconclusive. Variant effect prediction sounds abstract but can be life-or-death for genetic disorders. Proud of the team for narrowing this gap!

there is much that we can learn from these alien intelligences. i'm excited to see what the community can do with the tools we are open sourcing interpretability is at the center of our success here. not only do these explanations offer potential discoveries, understanding the model was critical to the entire research path here. very proud of the team's work and to our amazing partners at @MayoClinic. excited for all the great things still to come

interpretability is hands down the coolest subfield of AI research and will change the world

Already gave this a rip for some VUS and Suspected Pathogenic variants that I have previously done deep analysis on, and can confirm that EVEE posits many of the same findings and conclusions that I have found in terms of prediction and suggested failure mechanism Examples: VUS (for which *I* am the only ClinVar entry) for one of my heterozygous mutations in DNAH5 that could play a ~small~ factor in the overall root cause of my PCD My Variant of CLCN1 that gives me a rare muscular disorder (which makes me look like Wolverine without needing to go to the gym, so not all bad 🤷‍♂️) EVEE is a nice tool for variant interpretation!

We achieved state-of-the-art performance in predicting which of 4.2 million genetic variants cause diseases by interpreting a genomics model, in a new preprint with @MayoClinic. We're now releasing an open source database for all variants in the NIH's clinvar database. 🧵(1/8)

Introducing self-correcting search: a technique to let diffusion models self-correct mid-trajectory. Working with @RadicalAI, we gave MatterGen a feedback loop from its own activations, improving viable on-target candidates by ~30%. (1/8)

All watched over by machines of loving grace

New blog post: how we built infrastructure to enable interp at trillion-parameter scale with minimal inference overhead. In a couple short years, interpretability has gone from toy models to the frontier. (1/6)

New Paper! RL can teach our models to solve math or code, but open-ended tasks — which make verification expensive or even impossible — remain difficult to optimize. LLMs-as-Judges help, but often struggle to retrieve information even when it is present. Reinforcement Learning from Feature Rewards (RLFR) provides a solution. Extracting model beliefs via interpretability reveals a well-calibrated reward signal that permits scalable training.

We’re putting more computation (in the form of intelligence) into the most general object in neural network training: backprop. This essay describes how I think we can do this, why interp is key, the relevance to alignment, and how we should do it right.