building open ai infrastructure @pleiasfr
Joined April 2011
- Tweets 46,255
- Following 4,048
- Followers 23,359
- Likes 160,162
3,747 Photos and videos
Pinned Tweet
May 25
After months of delay, the successor post to "The model is the product": the AI decoupling. vintagedata.org/blog/posts/t…
6
49
335
37,850
1. The decisive question right now is whether open-endedness/recursivity requires different concepts of training more than scale.
I would love for Zephyr to be correct but the issue is that the gap will come from aspects that aren't distillation availability. Americans have brought up really large clusters, at least Anthropic has an idea of what to do with them, some RSI has started, and there's a data moat
18
1,193
Non-US pretraining team now trying to babysit a run without reading outputs
JUST IN: Anthropic says a “huge percentage” of its own employees are now barred from accessing Fable 5 & Mythos 5 under U.S. restrictions.
5
5
79
3,751
Ok if the only answer to all this is billions of Mistral subsidies, I’m out.
Jun 13
Not having any EU competitive labs (plural intended) is about to get very painful.
13
2
135
15,225
walking around with sovereign glasses near sovereign artworks to show anthropic how disappointed i am
3
1
89
3,819
Jun 13
i guess the blogpost is timely again.
May 25
After months of delay, the successor post to "The model is the product": the AI decoupling. vintagedata.org/blog/posts/t…
2
8
46
4,911
Jun 13
🫠
Jun 13
Légitime fierté française en matière d’emplois dans l’intelligence artificielle: à l’échelle européenne, Paris mène largement dans la compétition. lesechos.fr/travailler-mieux…
7
46
9,914
Jun 13
There are still multiple paths to make this happen. They all involve nuking two common delusions in outer space:
1. We can win at the application layer
2. Let’s give up on LLM and do 15m LeWorldModel.
Jun 13
Not having any EU competitive labs (plural intended) is about to get very painful.
12
13
152
11,273
Jun 13
Yes and yes. About 10 billion tokens multilingual, good enough for the gpt-2 range.
can someone train an LLM only on data from before like 1600 or something
would it sound like a person from back then
3
26
3,425
Jun 13
Not having any EU competitive labs (plural intended) is about to get very painful.
Jun 13
The US government, citing national security authorities, has issued an export control directive to suspend all access to Fable 5 and Mythos 5 by any foreign national, whether inside or outside the United States, including foreign national Anthropic employees.
The net effect of this order is that we must abruptly disable Fable 5 and Mythos 5 for all our customers to ensure compliance.
Access to all other Claude models is not affected.
We apologize for this disruption to our customers. We believe this is a misunderstanding and are working to restore access as soon as possible.
Read our full statement: anthropic.com/news/fable-myt…
20
12
168
23,182
Jun 12
and new position opened at pleias: not research, but very much ai development.
1
24
3,049
Jun 11
A few good points, unfortunately still circling on the one bad narrative: still a ton to do on V/LLMs *especially* for Physical AI.
Jun 11
Here's a project I've been working on recently: a vision of what happens if Europe doesn't take AI seriously, inspired by AI 2027 europe2031.ai/
2
2
18
2,145
Jun 10
Not surprised. It's totally true that we haven't nailed the most efficient arch for many modalities/data representation, just it isn't JEPA.
Jun 10
You may have recently heard claims that video generation models are "dumb" about physics, and only "world models" (V-JEPA, specifically) have a valid internal model of physics.
This turns out to be false. In a recent paper, researchers show that a LINEAR probe of diffusion videogen models predict various "physics" very well, significantly better than V-JEPA or VideoMAE (and plain VAE just sucks).
This is noteworthy, because a *linear* probe being this accurate shows that the model has a pretty explicit internal representation of the physics!
1
1
67
7,457
Maybe less labs than large companies. This is already happening in China and the tokenonomics crisis has created the incentives.
Jun 9
I’m curious who the labs are that Anthropic is actually worried that Mythos/Fable will accelerate. OpenAI doesn’t use Claude. DeepMind?
1
27
3,099
Fables do pass my new hard memorization tests (opus is just hedging: " I can't, from memory, confidently [state] without risking fabrication")
1
2
19
1,699
Currently the one loop that works is my agents asking me for more data annotation (they can’t do it)
Jun 7
Here’s your monthly reminder that you shouldn’t be prompting coding agents anymore.
You should be designing loops that prompt your agents.
4
2
38
3,194
Thanks Anthropic for automating middle management, I guess.
14
891
casual saturday
don't agree tbh. data labeling sounds low status but it's actually incredibly valuable work and no one is above it
4
87
11,258
So I guess we’ll just endlessly cycle through hubris and cope.
5
1
34
2,164
A few months ago, frontier models were clearly undersold, but now, usual reminder there are many things they can’t do. Recursive self-improvement looks very jagged.
1
1
25
1,187