Giving LLMs too much RoPE: A limit on Sutton’s Bitter Lesson — Bradley C. Love
Introduction Sutton’s Bitter Lesson (Sutton, 2019) argues that machine learning breakthroughs, like AlphaGo, BERT, and large-scale vision models, rely on general, computation-driven methods that...
bradlove.org