Timaeus is joining ⊢ Sequent Research!

Please reach out if you’re interested in working with us! Sequent will have a large in-person presence in Berkeley, as well as researchers remote from London, Melbourne, and elsewhere. 🇺🇸🇬🇧🇦🇺 1. Full post: sequent.org/launch 2. Express interest: sequent.org/apply

RT @jesse_hoogland: I’m incredibly proud of what we’ve built with Timaeus and all the research we’ve accomplished. But given how much resea…

This is why a strong human component is still necessary in alignment research at Sequent. Maybe that’s you. You should consider dropping what you’re doing and helping. A lot of other theoretical and empirical research can be left to the ASI, but alignment can’t (responsibly).

“Neural networks are grown, not programmed” We’re changing that. Mechinterp investigates how models generalize beyond their training data by studying the resulting internal structure. We introduce patterning as the dual: given desired structure, determine what data produces it.

RT @jesse_hoogland: Well, we haven't solved all the problems yet, so the offer stands... If you want to solve deep scientific problems in…

Training Data Attribution (TDA) should account for learning dynamics! The same data can influence model behavior in dramatically different ways at different time points of training. We call for a shift towards stagewise data attribution and the study of influence dynamics. 1/11

How does training data shape model behavior? Well, it’s complicated… 1/10

1/ AI is accelerating. But can we ensure that AIs truly share our values and follow our goals? We argue that aligning advanced AI systems requires cracking a core scientific challenge: how data shapes AI's internal structure, and how that structure determines behavior.

Our paper has been accepted to ICLR as a spotlight paper! We introduced the refined Local Learning Coefficient, which measures “how much structure” there is in particular parts of the model associated to particular datasets or behaviors.

1/8 How do transformers learn? In our new work, we find that transformers develop in-context learning in discrete stages that can be automatically discovered. 🧵 arxiv.org/abs/2402.02364 Joint work w/ @georgeyw_, Matthew Farrugia-Roberts, @lemmykc, Susan Wei, @danielmurfet