We're training the largest open-source, open-development, AND open-weight base LLMs of any (actual) non-profit. The latest on our journey to the frontier is a 129B-A16B 1T tokens (1e23 FLOPs) MoE. We've improved our training efficiency (i.e. loss per FLOP) by >5x in just the past couple months. Follow along the day to day work in Discord, watch us make mistakes and discuss tradeoffs on GitHub, see our loss curves on Weights & Biases, and watch us consistently hit our preregistered loss targets within 1% @ marin.community

I wanted to run some old protein structure analysis software (ConFind) at scale... written in C , custom build instructions, slow. A slog of a task a year ago. But now easy to just reimplement it github.com/timodonnell/pycon… . Yes to agents rescuing good ideas from bit rot!

When you come into biology from CS you scoff at the obscurity of everyone's research, look at all these underpaid postdocs churning out papers on ridiculously niche topics. They don't have the right (startup-adjacent) cultural traits to tackle ambitious goals. Then...

To train better open models, we need predictable scaling. Delphi is Marin’s first step: we pretrained many small models with one recipe, then extrapolated 300× to predict a 25B-param / 600B-token run with just 0.2% error. Getting there took some work 🧵

Anyone willing to donate GPU time to re-train MHCflurry?

MHCflurry 2.2.0rc2 is on PyPI: pypi.org/project/mhcflurry/2… Try it out and let us know if you spot any problems in our transition from TensorFlow to PyTorch

I suppose I am coming back to this platform... bsky never quite hit critical mass for me

Thanks to @iskander 's robot army for rewriting mhcflurry in torch! Available now as "pip install mhcflurry==2.2.0rc1" github.com/openvax/mhcflurry…

I'm rebuilding AlphaFold2 from scratch in pure PyTorch. No frameworks on top of PyTorch. No copy-paste from DeepMind's repo. Just nn.Linear, einsum, and the 60-page supplementary paper. The project is called minAlphaFold2, inspired by Karpathy's minGPT. The idea is simple: AlphaFold2 is one of the most important neural networks ever built, and there should be a version of it that a single person can sit down and read end-to-end in an afternoon. Where it stands today: - ~3,500 lines across 9 modules - Full forward pass works: input embedding → Evoformer → Structure Module → all-atom 3D coordinates - Every loss function from the paper (FAPE, torsion angles, pLDDT, distogram, structural violations) - Recycling, templates, extra MSA stack, ensemble averaging — all implemented - 50 tests passing - Every module maps 1-to-1 to a numbered algorithm in the AF2 supplement The Structure Module was the most satisfying part to build. Invariant Point Attention is genuinely beautiful — it does attention in 3D space using local reference frames so the whole thing is SE(3)-equivariant, and the math fits in about 150 lines of PyTorch. What's next: - Build the data pipeline (PDB structures MSA features) - Write the training loop - Train on a small set of proteins and see what happens The repo is public. If you've ever wanted to understand how AlphaFold2 actually works at the level of individual tensor operations, this is meant for you. Repo: github.com/ChrisHayduk/minAl…

new preprint alert! tl;dr we made a global tokenizer for proteins

Re-posting Nick Boyd's Mosaic blogpost after his incredible results in the Nipah G competition. blog.escalante.bio/180-lines…

🚀 Just released: Protein Hunter on GitHub! github.com/yehlincho/Protein… Now supports Boltz and Chai with more models coming soon! Use it to: 1️⃣ Design binders from scratch 2️⃣ Optimize your own designs 🔗 Boltz: shorturl.at/0s5Ih 🔗 Chai: shorturl.at/NqrHZ

OpenFold3-preview (OF3p) is out: a sneak peek of our AF3-based structure prediction model. Our aim for OF3 is full AF3-parity for every modality. We now believe we have a clear path towards this goal and are releasing OF3p to enable building in the OF3 ecosystem. More👇

I’ve been testing BoltzGen a bit recently and while I haven’t done any experimental testing yet, the quality of the software is very clear. It installs, runs, logs everything, has tons of options. Very excited to test out the designs irl!

Excited to release BoltzGen which brings SOTA folding performance to binder design! The best part of this project has been collaborating with many leading biologists who tested BoltzGen at an unprecedented scale, showing success on many novel targets and pushing its limits! 🧵..

🧵1/ We @tahoe_ai just published a new post on the Tahoe blog—a story of how we used Tahoe-100M, the world’s largest drug-perturbed single-cell dataset, to find compounds that upregulate MHC-I and make tumors more visible to the immune system. Here’s how 🧬🔍👇

Protein templates are finally supported in Boltz-2! Next up: updating BoltzDesign1 to use Boltz2 for template-based motif scaffolding and more