3 Photos and videos
Alexander Theus retweeted
Apr 23
#ICLR2026
Into mode connectivity, model merging, or permutation invariance? We show how optimization dynamics shape the loss landscape of merged weights. Come check it out!
π
23/04 10:30AM β 13:00PM
π Pavilion 3 P3-1809
w/ @TheusResearch @DamienTeney @orvieto_antonio
1
1
6
170
Alexander Theus retweeted
π’ Submissions are OPEN for the Weight Space Symmetry Workshop @icmlconf!
β° Deadline extended β April 30 (23:59 AOE)
Consider submitting any work related to weight symmetries: optimization, model merging, weight space learning, and so on!
#ICML2026 #weightsymmetry2026
1
6
19
6,067
Alexander Theus retweeted
π’Excited to announce the Workshop on Weight-Space Symmetries @icmlconf! We welcome 4-page submissions analysing symmetries, their effects on training and model structure, and practical methods to utilize them.
Submission Deadline: April 24 (23:59 AoE)
#ICML2026
3
37
56
21,774
19 Sep 2025
Excited to announce that our paper has been accepted as an Oral at NeurIPS 2025! π₯³
9 Jul 2025
1/ π¨ New paper alert! π¨
We explore a key question in deep learning:
Can independently trained Transformers be linearly connected in weight space β without a loss barrier?
Yes β if you uncover their rich symmetries.
π arXiv: arxiv.org/abs/2506.22712
1
1
12
1,131
9 Jul 2025
1/ π¨ New paper alert! π¨
We explore a key question in deep learning:
Can independently trained Transformers be linearly connected in weight space β without a loss barrier?
Yes β if you uncover their rich symmetries.
π arXiv: arxiv.org/abs/2506.22712
2
8
59
5,700
9 Jul 2025
9/ π Takeaway:
Transformers can be linearly connected β but only if you exploit richer network symmetries.
We show that general symmetry alignment (not just permutations) unlocks low-loss paths across ViTs and GPT-2.
1
522
9 Jul 2025
10/ π Paper: arxiv.org/abs/2506.22712
By: @Theus__A , Alessandro Cabodi, @SAnagnostidis , @orvieto_antonio , @unregularized , and @val_boeva
π Huge thanks to my amazing co-authors for this collaboration!
#Transformers #LMC #MachineLearning #DeepLearning
1
1
557