@unregularized profile picture
Sidak Pal Singh @unregularized
Joined October 2022
  • Tweets 51
  • Following 106
  • Followers 485
16 Photos and videos
Pinned Tweet
Sidak Pal Singh@unregularized
26 Jul 2024
📢I'll be presenting two posters, at #ICML2024 HiLD workshop (Straus 2) today (assuming no further ✈️ delays): - Closed form of the Hessian spectrum for some neural networks openreview.net/forum?id=gW30… - Landscaping Linear Mode Connectivity openreview.net/forum?id=OSNM…
1
1
12
3,503
quite good actually. coding’s so smooth! x.com/GoogleDeepMind/status/…

Google DeepMind
@GoogleDeepMind
18 Nov 2025
This is Gemini 3: our most intelligent model that helps you learn, build and plan anything. It comes with state-of-the-art reasoning capabilities, world-leading multimodal understanding, and enables new agentic coding experiences. 🧵
1:57
188
sure the numbers are great, but screw that! it’s your phd, try wild and original ideas even if you fail once or twice at least. x.com/gabriberton/status/194…

Gabriele Berton
@gabriberton
29 Jul 2025
A few numbers from my PhD: 8 first-author top-conference (CVPR/ICCV/ECCV) papers 100% acceptance rate per paper 80% acceptance rate per submission 1 invited long talk at CVPR tutorial 5 top-conf demos (acceptance rate 100% vs ~30% average) ~2k GitHub stars
9
923
when you go beyond linear mode connectivity, interesting things happen 😮👇 x.com/Theus__A/status/194300…

Alexander Theus@TheusResearch
9 Jul 2025
1/ 🚨 New paper alert! 🚨 We explore a key question in deep learning: Can independently trained Transformers be linearly connected in weight space — without a loss barrier? Yes — if you uncover their rich symmetries. 📄 arXiv: arxiv.org/abs/2506.22712
1
6
614
Belated life update: 🎓 PhD — done 🔬 Joined Google in NYC 🗽as a Research Scientist ♊️ Gemini: now more than just my star sign :)
25
11
551
29,144
🚀 TOMORROW afternoon at ICLR: Learn about the directionality of optimization trajectories in neural nets and how it inspires a potential way to make LLM pretraining more efficient ♻️ (Poster# 585, hall 2b)
Sidak Pal Singh@unregularized
12 Jul 2024
Ever wondered how the optimization trajectories are like when training neural nets & LLMs🤔? Do they contain a lot of twists 💃 and turns, or does the direction largely remain the same🛣️? We explore this in our work for LLMs (upto 12B params) ResNets on ImageNet. Key findings👇
1
6
2,110
Don't miss out our spotlight ✨paper at ICLR 🇸🇬 about the loss landscape of Transformers and their special heterogeneous structure, done together with great collaborators! x.com/wormaniec/status/19145…

Weronika Ormaniec@wormaniec
22 Apr 2025
Ever wondered how the loss landscape of Transformers differs from that of other architectures? Or which Transformer components make its loss landscape unique? With @unregularized & @f_dangel, we explore this via the Hessian in our #ICLR2025 spotlight paper! Key insights👇 1/8
2
16
1,384
Sidak Pal Singh retweeted
Alice Bizeul@AliceBizeul
12 Feb 2025
✨New Preprint ✨ Ever thought that reconstructing masked pixels for image representation learning seems sub-optimal? In our new preprint, we show how masking principal components—rather than raw pixel patches— improves Masked Image Modelling (MIM). Find out more below 🧵
17
61
524
48,467
Sidak Pal Singh retweeted
Yann N. Dauphin@ynd
10 Dec 2024
Don’t miss our poster shedding more light on sharpness regularization at NeurIPS tomorrow neurips.cc/virtual/2024/post…
4
6
2,678
Reinventing things has a bad rep in today's age. But is it really that bad? Maybe it's something to be even cultivated, like selectively? The second post in this series of blogs is now out. Let's have a deeper look at this overused trope! wovencircuits.substack.com/p…
2
310
I’m exploring a new form of writing—threads of human curiosity woven through the circuits of AI, crafting reflections that are, in the end, fully machine-generated, yet in a way profoundly human. wovencircuits.substack.com/p…
214
Come, let's scale up the building one floor, And, layer up the neural networks once more. Soon our buildings will touch the sky, And, our computers will bear AGI. A quaint little hut in the mountains is out of fashion, Satisfaction has no gradients for backpropagation. ~Fitoor
5
309
At this paper count, recalling all the paper names would already be a big feat :) x.com/peter_richtarik/status…

Peter Richtarik
@peter_richtarik
4 Oct 2024
Source: papercopilot.com/paper-list/…
2
353
“Hypotheses are nets: only he who casts will catch.” - Novalis
4
332
At last some attempts to change the status quo: authors with three or more papers are obligated to review for ICLR x.com/PreetumNakkiran/status…

Preetum Nakkiran@PreetumNakkiran
7 Aug 2024
Review requirements! (And 10pg limit!)
1
6
606
📢I'll be presenting two posters, at #ICML2024 HiLD workshop (Straus 2) today (assuming no further ✈️ delays): - Closed form of the Hessian spectrum for some neural networks openreview.net/forum?id=gW30… - Landscaping Linear Mode Connectivity openreview.net/forum?id=OSNM…
1
1
12
3,503
Poster 1: Sharpness/Flatness are much talked about: better minima, Sharpness aware minimization, Edge-of-Stability, and so on. But what really is sharpness? What exactly does it quantify, besides the surface-level definition? How are the eigenvalues and eigenvectors really like?
2
279
Come to our posters today at 3:30 pm (Straus 2) to know more! :)
159
Poster 2: Linear Mode Connectivity (LMC) is yet another popular feature of neural loss landscapes. But how does LMC arise in the first place? How should the landscape be structured to allow LMC? Are barriers present just at the end, or do they start much early?
1
230
There goes away my Austrian flight to #ICML2024 🥲
1
352
Sidak Pal Singh retweeted
Yuhui Ding@yuhui_ding
22 Jul 2024
Tuesday 1:30pm-3pm, Hall C 4-9 #515. Drop by our poster if you are interested in SSMs for graphs👇! Code: github.com/skeletondyh/GRED
4
3
11
1,862
Load more