ML PhD breaking down the latest #DeepLearning and #LLMs papers. Adventures in latent space.
Joined January 2015
- Tweets 458
- Following 319
- Followers 124
- Likes 31
78 Photos and videos
Pinned Tweet
"Bangers" essentially live in the tails of the distribution.
Modern post-training is designed to chop those tails off to prevent high-perplexity "weirdness." We optimized for Safety > Vibes, so now we have models that are perfectly polite and statistically incapable of being interesting.
Jan 3
It happened again! I brainstormed a new name for a library, and gpt-5.2 and Opus 4.5 were both kinda mid. They had "logical" names but none fitting quite my taste, being too cookie cutter.
Each one had one OK suggestion among many. Surprisingly, both models had the same name idea as their favourite one, independently.
Then I asked GPT 4.5. And damn, it spit out banger after banger name ideas, to the point where now I can't decide between multiple great names.
This model has a special place in my heart. Sorry for somewhat vague-posting again :)
1
13
3,785
Feb 24
Video models generate stunning visuals, but can they actually *reason*? This massive new work from @hokindeng @caizhongang @Williamiumli and 50 collaborators tackles a question I've been curious about: what happens when you study video
reasoning at unprecedented scale? 𧡠1/8
1
3
81
Feb 24
The pipeline supports continuous expansion: new tasks can be added and scaled automatically. This could be transformative if it enables the community to rapidly test hypotheses about what reasoning skills matter most. Data toolkit at video-reason.com 7/8
1
22
Feb 24
This is infrastructure work that could enable a new research direction. The scale systematic eval early scaling laws make it more than just "another benchmark."
π arxiv.org/abs/2602.20159
20
Feb 22
1/ When does sparse attention *break*? SpargeAttention2 from @Jintao_Zhang_, @xiangxiaoyuaw, @HaochengXiUCB, @jianfei_chen team digs into why Top-k and Top-p masking fail at high sparsity (>90%) and proposes a surprisingly simple fix.
1
34
Feb 22
10/ What I really like: they asked *why* existing methods fail before proposing a fix. The analysis of masking failure modes (uniform vs skewed distributions) feels like the kind of insight that will apply beyond this specific paper. Thoughts on other failure modes?
1
12
Feb 10
FedEx is officially the worst. My "International Priority" is apparently on a European vacation. π Route: NL π³π± β‘οΈ Paris π«π· β‘οΈ Cologne π©πͺ ...and now BACK to Paris π«π·?! π€‘ Itβs been 5 days for a next-day delivery.
1
60