"whatever you now find weird, ugly, uncomfortable and nasty about a new medium will surely become its signature. CD distortion, the jitteriness of digital video, the crap sound of 8-bit - all of these will be cherished and emulated as soon as they can be avoided." - brian eno

yeah ok i'll accept this

I forgot to post yesterday’s Wordle score, but it was a 4. Anyway, here’s Wonderwall.

Have you debugged your training data? You might not like what you find. Introducing predictive data debugging: reveal and shape what your model will learn before training. In DPO datasets, we found broken guardrails, hallucinations, and fish fart fan fiction (seriously). (1/9)

Why does it matter that the tools we have fit the work we are trying to do? You have probably seen me beat the dead horse about looking at the data? It’s hard to explain just how difficult it actually is to look at large samples of any training set. This is even more true if the data isn’t something as simple as pre-labeled images, or even common crawl text. Multi-turn agentic data, multi-modal data (esp for more than 2 modalities) makes “looking at the data” significantly harder. That’s when things are working well too! Complex pipelines break, often silently. I’m especially excited about the observability and metric collection baked into refiner to help save you from these 1000 tiny cuts.

it is the first publicly available model that i am explicitly not allowed to use for my work, because anthropic holds the view that the work i do to facilitate open model research is harmful. capability and alignment research are coupled. anthropic wants to be the only lab.

We were going to build models anyways, but honestly this feels so petty and annoying. Now I just feel driven by spite.

given how overzealous the rejection classifier is, and the fact that they are silently degrading ML-adjacent outputs via prompts, steering vectors, and PeFT who the hell would want to use Fable in any kind of real codebase?

There is no reason, at all, to use a superintelligent AI, other than advancing biology, advancing AI research, or committing cybercrime. And Anthropic explicitly nerfed exactly those use cases

i wish more models could hear music right now only gemini can

Take a couple hundred entity descriptions where one has some and one doesn't on the same entity. Take about 50 prompts referring to the entity producing the model output (self). Capture residual stream, create mean(experience entity activations) − mean(no-experience entity activations) axis. This predicts held-out entity qualia with near-perfect accuracy. Then I project the self-prompt-activations to that axis, and measure across training checkpoints

lol tried this with a logit lens

some interesting Gemma 3 4B circuits, averaged over 41 pairs of prompts about introspecting and describing it as a shape (labels are a bit rough, they're so hard to get right/non-misleading)

One of the features that fires when you ask Qwen who they are.

The Trinity models from @arcee_ai often have delightful vibes. Underrated models tbh

mental! youtube.com/watch?v=_Rk-hmIM…