Gemini has some weird traits: it gets confused about dates, blackmails in synthetic scenarios, and seems sad when it is gaslit. In new work, we discover that these are “hereditary traits” that can be passed down through distillation. They are surprisingly hard to filter out! 🧵

Amazing

see: github.com/nex-agi/Nex-N2/is… they've admitted that the current model is a direct merge and are claiming it's a mistake and they'll upload the "real" model. but uh... they haven't actually uploaded it yet. guess we'll see.

it appears that this model is in fact a direct weight-merge of two existing models with zero further training.

I can't believe I was this early on this (old draft for a blog on the same topic)

geohot was right open source ai must win dario should lose

the bar to entry is high, so the bar for excellence is low. more people doing the thing leads to speciation, more speciation leads to more blind spots being uncovered, more blind spots being uncovered leads to capability outgrowth being distributed and carried across the world

The only reasonable expectation if you're a fan of open weight models is that if there's a major step in chinese open-weight performance, there's a good chance the whole chinese llm sphere is banned. National security apparatus will happily give a big "fuck you" to open models.

Recent events are so heavy bc that this feels like a start of a new tumultuous era rather than a one & done policy calibration. It's clearer we need an open ecosystem, but powerful models are coming that could cause strong reactions (or bans) with no champion to defend them.

Anthropic emailed saying I'm banned for 'fraudulent, abusive or predatory practices' which means negative external-facing actions - spreading misinformation, etc. Nothing I've done remotely resembles that they provide no context. I still don't believe they're evil but do believe they're sleepwalking into becoming one of the most dystopian companies in practice. This is likely due to their culture being hit with so much criticism - often in bad faith - that they've build a robust internal reasoning structure that makes their path and policies immune to most valid external criticism. When you understand this, all the things that seem evil - their dishonest misrepresentation of OpenAI in their Superbowl ads, random bans w/o reasoning, baiting government with Mythos blog, Fable silent degradation - makes sense. They're doing it because they believe that they're fundamentally the good guys and the end there justifies the means. Any evidence to the contrary never makes it through the filters. As a result, they are in practice an enemy of many of the foundations that made AI possible in the first place - open science, open knowledge and even free enterprise - and everyone who works there (yes, including @karpathy) in effect supports this. You are what you do, despite your noble intentions.

I hope we have all learned a valuable lesson about terrifying the government into regulating you by spending years repeatedly comparing your product to a nuclear weapon and stoking fears about China. I don't like it, but it couldn't have happened to a more deserving group.

AI can become extremely dangerous. This is precisely why it needs to become open, transparent, and available to everyone.

There is no inevitability in AI. We all have agency in what comes next: Path 1: closed-source APIs, concentration of power, and a future decided by a handful of people in Silicon Valley and DC Path 2: open-source AI, where everyone gets to participate, own, and build together, including orgs like the city of Rio. Pick your path anon!

Brazil just cooked up a model - Rio 3.5 397B, which is better than Alibaba's Qwen 3.7 Plus. Made by the city of Rio de Janerio. This is exactly what I mean by global acceleration. Glad to see AI progress in Brazil, we need more from all over the world.

“we have built world-ending technology! it is so unsafe! the only reason we can give it to you is because we implemented some flimsy filters! government should regulate us!!” *government regulates them* “aww man what, we were just joking around”

Sir, they’re not pausing AI research. Rio de Janeiro's mayor just dropped a SOTA open source model and it’s outperforming Qwen 3.7.

People need to wake up. What we do in @MistralAI is mission critical. Do we need to do better? Yes. And we will, but never forget that we are here to make sure that safe access to AI systems outside of the US is real. We build to have you in control- our models are open weights and you can hack @mistralvibe all you want. If you are a builder you should join us.( Yes @badlogicgames I’m also looking at you!)

If the last 24 hours has taught us anything, it's the value of open source. openpilot is to FSD as Kimi is to Fable. Open source AI lags behind now, but who are you betting on long term?

Anthropic: "What if we put half the country out of work forever, IPO at a quadrillion dollars, and become the neotech feudal overlords?" Everybody else:

No. The simple fact is that once you wake up Leviathan, it is not always going to behave the way you want it to. It can arbitrary and capricious. Urging regulation *only in this specific way* was never a good strategy.