Very concerned about the second-order effects of advances in AI auditors. The direct consequences are well understood by most: lower cost of entry, higher bar for exploits, bounty hunters and blackhats retroactively auditing old codebases, etc.
There will come a point (expect 1-2 years from now) where more and more real projects will completely skip a human audit - it's an uphill battle to convince budget owners to spend 6-7 figures on security when a sub-$1k solution appears to find the same issues again and again. For everyone who doesn't grasp the true ingenuity of security research, this will feel like the moment thousands of accountants were replaced by Excel formulas, or travel agencies were displaced by Google Flights.
But there are those of us that understand that bug hunting is as much an art as it is a science, and pattern-matching, no matter how exceptionally engineered, will never discover a complex, novel logic bug or bug class. It will be up to us to educate and to make the case for why the best hunters can never be replaced by neural networks that are, at the end of the day, extremely capable imitators. They will say we are coping. That we are decelerationists. That AIs that can write poems and solve Math Olympiad questions can audit code as well as any human. They will be wrong, and time will tell.
The problem is that extremely complex bugs are likewise extremely rare, so trusting an AI audit could genuinely be 99.9% safe within a few years. And a fair argument is that even a human audit doesnāt necessarily capture the last 0.1% ā weāve seen projects exploited after top-tier audits, which has harmed confidence in the entire audit marketplace. So is the new āsecurity through obscurityā going to become āsecurity through low probability of a novel bugā? It might.
My outlook on the security endgame looks different. There will be two battles fought in parallel. The AIs on both sides will have a light-speed shootout, determining who can exploit the vast majority of bugs quicker than the other. That contest will be over in a few blocks, and weāll know who won by checking whether the contract still has funds. And then there's the "correspondence chess" match between the true savants on both sides - humans capable of more than imitation. This battle never truly ends, unless all critical properties are formally proved, flawlessly. This cannot be done for sufficiently complex apps and infrastructure.
We know blackhats are not going anywhere. NK and similar aren't interested in post-hack negotiations and are going to pocket 100% of whatever value they steal. Meanwhile white-hats may well not be incentivized enough in a world where only very rare bugs remain, getting cents on the dollar per exploit compared to blackhats, if not completely rugged by the project.
From that perspective it seems clear projects that truly care about their security have to recruit white-hats to their side, or they risk conceding an empty-net goal. Hiring a team of specialists with an exceptional record of finding deep-lying bugs raises the difficulty and adds a 2nd layer of PoW on top of an AI audit. In practice, blackhats will likely follow the path of least resistance and choose targets in ascending order of difficulty. Hiring teams (like yours truly) that have disclosed tens of live issues and earned dozens of public contest gold medals is going to push a project to the bottom of that "to do" list.
AI tech is revolutionary and will change much of how the world operates. But let's remember: a bad poem earns at worst a thumbs-down. An undetected smart contract bug could be the end of people's financial lives. Letās be careful not to entrust AI with millions of dollars in one of the few domains where human creativity still outperforms the machine.