4 Photos and videos
New paper from Basis' Project MARA team and collabs. The ability to learn and use world models is a key aspect of human intelligence, but evaluating this ability remains elusive. In this work we propose WorldTest, a representation-agnostic, behavior-based agent eval framework.
1
11
21
3,881
We're hiring research scientists in PL and other areas. Join us! basis.ai/join-us/#careers
2
406
Basis retweeted
11 Nov 2025
New preprint on learning abstract world models for robotics planning. Paper code below. š¤š
Must an agent plan by simulating pixels frame by frame, or can it think in abstractions? Consider planning an international flight: we can reason about buying tickets, changing airplanes, and crossing borders without committing to the color of the airplane or the milliseconds before takeoff. Absent abstraction, planning over long time horizons would be intractable, because every minute detail of the world would need to be simulated.
[1/7]
2
11
24
4,526
Basis retweeted
28 Oct 2025
šØ MIT and Basis Research just dropped a new way to measure if AI actually understands the world and the results are brutal.
Itās called "WorldTest", and it doesnāt just check how well an AI predicts the next frame or maximizes reward.
It checks whether the model can build an internal model of reality and use it to handle new situations.
They built 'AutumnBench', a suite of 43 interactive worlds and 129 tasks where AIs must:
ā¢ Predict hidden parts of the world (masked-frame prediction)
ā¢ Plan sequences of actions to reach a goal
ā¢ Detect when the environmentās rules suddenly change
Then they tested 517 humans vs. top AI models Claude, Gemini 2.5 Pro, and o3.
Humans crushed every model. Even massive compute scaling barely helped.
The takeaway is wild... current AIs donāt understand environments; they pattern-match inside them.
They donāt explore strategically, revise beliefs, or run experiments like humans do.
WorldTest might be the first benchmark that actually measures understanding, not memorization.
The gap it reveals isnāt small itās the next grand challenge in AI cognition.
Paper: Benchmarking World-Model Learning (arxiv. org/abs/2510.19788)
56
210
921
109,947
Basis retweeted
30 Oct 2025
like i have been saying since 2019, world models are the next key step.
29 Oct 2025
š„ MIT just exposed every top AI model and itās not pretty.
They built a new test called WorldTest to see if AI actually understands the worldā¦ and the results are brutal.
It doesnāt just check how well a model predicts the next frame or maximizes reward it tests whether it can build an internal model of reality and use it to handle new situations.
They built AutumnBench 43 interactive worlds, 129 tasks where AIs must:
ā¢ Predict hidden parts of the world (masked-frame prediction)
ā¢ Plan sequences of actions to reach a goal
ā¢ Detect when the environmentās rules suddenly change
Then they tested 517 humans vs. Claude, Gemini 2.5 Pro, and o3.
Humans crushed every model. Even massive compute scaling barely helped.
The takeaway is wild.. todayās AIs donāt understand environments; they just pattern-match inside them.
They donāt explore strategically, revise beliefs, or run experiments like humans do.
WorldTest might be the first benchmark that actually measures understanding, not memorization.
The gap it reveals isnāt small itās the next grand challenge in AI cognition.
(Comment āSendā Iāll DM you the paper)
10
29
215
32,587
Basis retweeted
30 Oct 2025
"Todayās AIs donāt understand environments; they just pattern-match inside them."
Literally what critics have been saying for years now.
29 Oct 2025
š„ MIT just exposed every top AI model and itās not pretty.
They built a new test called WorldTest to see if AI actually understands the worldā¦ and the results are brutal.
It doesnāt just check how well a model predicts the next frame or maximizes reward it tests whether it can build an internal model of reality and use it to handle new situations.
They built AutumnBench 43 interactive worlds, 129 tasks where AIs must:
ā¢ Predict hidden parts of the world (masked-frame prediction)
ā¢ Plan sequences of actions to reach a goal
ā¢ Detect when the environmentās rules suddenly change
Then they tested 517 humans vs. Claude, Gemini 2.5 Pro, and o3.
Humans crushed every model. Even massive compute scaling barely helped.
The takeaway is wild.. todayās AIs donāt understand environments; they just pattern-match inside them.
They donāt explore strategically, revise beliefs, or run experiments like humans do.
WorldTest might be the first benchmark that actually measures understanding, not memorization.
The gap it reveals isnāt small itās the next grand challenge in AI cognition.
(Comment āSendā Iāll DM you the paper)
37
1,072
6,804
151,602
New paper from Basis' Project MARA team and collabs. The ability to learn and use world models is a key aspect of human intelligence, but evaluating this ability remains elusive. In this work we propose WorldTest, a representation-agnostic, behavior-based agent eval framework.
1
11
21
3,881
All open roles: jobs.ashbyhq.com/basis-reseaā¦
1
1
249
We'll also be at NeurIPS; come talk to us! Visit our booth or register for our social: luma.com/ivw952te
1
371
Our Project MARA team who led this work is looking for research scientists to join us! Link to apply below.
29 Oct 2025
š„ MIT just exposed every top AI model and itās not pretty.
They built a new test called WorldTest to see if AI actually understands the worldā¦ and the results are brutal.
It doesnāt just check how well a model predicts the next frame or maximizes reward it tests whether it can build an internal model of reality and use it to handle new situations.
They built AutumnBench 43 interactive worlds, 129 tasks where AIs must:
ā¢ Predict hidden parts of the world (masked-frame prediction)
ā¢ Plan sequences of actions to reach a goal
ā¢ Detect when the environmentās rules suddenly change
Then they tested 517 humans vs. Claude, Gemini 2.5 Pro, and o3.
Humans crushed every model. Even massive compute scaling barely helped.
The takeaway is wild.. todayās AIs donāt understand environments; they just pattern-match inside them.
They donāt explore strategically, revise beliefs, or run experiments like humans do.
WorldTest might be the first benchmark that actually measures understanding, not memorization.
The gap it reveals isnāt small itās the next grand challenge in AI cognition.
(Comment āSendā Iāll DM you the paper)
3
2
295
And we're hosting a social at NeurIPS. If you want to come chat with us, RSVP: luma.com/ivw952te
3
323