If we already have AGI — why can't it solve a sudoku puzzle?
According to any reasonable definition, a "reasoning" AI should be able to arrange a few numbers in a 9x9 grid without much trouble. But frontier LLMs are really bad at this.
By "this" we mean Constraint Satisfaction Problems — stuff like chip design, energy grid optimization, HFT. This is because token-by-token generation tends to commit to bad choices very early on, and can't revise them later.
However, there is a whole other approach — Energy-Based Models, and researchers like
@YannLeCun have been pushing for it since the 1980s. Now it finally seems to be feasible, and, among other things, can give us cheap and efficient formal software verification — which is very much needed, given just how messy ALL of humanity's software turned out to be.
EBMs minimize an energy function in latent space. High energy means high constraint violation (something's wrong); low — you're close to the truth. Unlike LLMs, they optimize the entire trace at once — and can refine it iteratively. This allows for much more precision with much less compute 🧵