I quite like this line of work! blends "mechinterp" and "learning mechanics" approaches to fundamental science of deep learning.

I thoroughly enjoyed reading this recent paper by @yasamanbb et al (arxiv.org/pdf/2602.15029) that derives analytically why certain latent variables must lead to geometry in word embeddings. (getting Fourier modes even with open boundary but exponential kernel is neat!) I think it would be great to compare this to some of @prfsanjeevarora et al's work on this (eg tinyurl.com/4az4325n) More broadly, I have been thinking about the right data generating process for language. For vision, we have latent spaces with great manifold structure (eg the SO3 pose of an object) and nonlinear mixing functions. But for language? Are there really any continuous latent variables? What is the "DSprites" of language? Is it all just co-occurrence stats or is there something more in LLM word embeddings?

Leaked Sam Altman messages (2023)

Mechanistic interpretability aspires to be the biology of deep learning. @KuninDaniel and @learning_mech say that an emerging theory of deep learning they and their team call 🛠️ learning mechanics 🛠️ will be the physics.

crazy how you can pinpoint the exact curvature of a trillion dimensional model based on how wiggly the loss curve is

did you know that with a few modifications, you can get the Ising model to simulate cells fighting to the death? one of my favorite side projects of all time: jamiesimon.io/blog/cell-figh…

for the bright-eyed and bushy-tailed: there's a Learning Mechanics discord! young academics who want to do research in this area should especially consider joining starting convos. discord.gg/GTHfUnf7hz

Excited to share that our paper “Sequential Group Composition: A Window into the Mechanics of Deep Learning” was accepted to ICML 2026 in Seoul! Co-led with @giovannimarchet and @AdeleMyersPhD @hopfbifurcator @ninamiolane Paper: arxiv.org/abs/2602.03655

Palantir office speedrun

ditto. props to @justanotherlaw for taking a second look :) (though I also found merit in the criticisms in the first version.) hopeful we can eventually (hopefully soon enough...) make contact w/ AI alignment governance, whose noble causes we would v much like to aid.

It seems we're at a stage where deep learning is evolving from alchemy into an engineering discipline; this is an exciting paper which lays out that a scientific theory is emerging for Deep Learning. Paper: arxiv.org/abs/2604.21691 Tweet: x.com/learning_mech/status/2…

yeah, totally! I once messaged everyone on facebook with my first and last name. I eventually made a big group chat! v ethnically geographically diverse. probs the closest I've gotten as an adult to meeting a truly random slice of the US.

Aren’t diffusion models explicitly derived from a correspondence with physics and entirely consistent with how physics says you should model systems over a range of scales ( ie mori zwanzig theory: langevin dynamics with a fitted vector field ? ) what more do you want?

thanks for the flag. comments on all pages of learningmechanics.pub should now be working! don't be shy :)

From "Mathematical theory of deep learning: Can we do it? Should we do it?" to "There Will Be a Scientific Theory of Deep Learning". It's respectively the title of a talk I gave four years ago, and the title of an arxiv paper from four days ago. I really like the "learning mechanics" perspective (think of it as a continuation of "statistical mechanics", "quantum mechanics", and so on). Several of my last academic papers can be viewed under that lens (e.g. Learning threshold neurons via the “edge of stability”; or LEGO). I'm not as optimistic as the authors of the recent arxiv paper that we will EVER be able to reach what the "physics mechanics" field have achieved, but it's certainly worth trying. Talk: youtu.be/3uRD_lg701k?si=yjLY… Paper: arxiv.org/abs/2604.21691

Organizers organizing

David Bau is talking about "Reading science back out of AI" @davidbau