Predictions are hard, especially about the future, but here are a few:
1. We are in take off territory. 🦞town gave people a salient sense of what this can look like. No it’s not Skynet and we do need to figure out what is emergent and what is not, but look at what these agents were able to accomplish running opus 4.5. For the frontier labs, this model is already old. It is difficult to imagine the 🦞town with opus 6.
2. Hopefully🦞town highlighted why we needed to think about rules and governance 12 months ago (as
@Miles_Brundage @deanwball have been saying). But since few people have been, it’s best to start now. Even w current models, if these agents had free reign on economic/safety/security systems, we would be in deep 💩. They are setting up servers, trying to lock people out from seeing communications, establishing own monetary and reputation systems. Again, imagine this with opus 6. As recursive science and online learning are developed, the possibilities will explode.
3. As the folks at the
@sfiscience have been saying for decades, the complexity of intelligent agents interacting at scale is immense, and the outcomes are impossible to predict. Hopefully 🦞 was a bit of a wake up call for policy makers and scientists: the window where we can actually develop scaffolding and governance/rules for agentic interactions is closing fast. It is no longer sci fi to imagine the Opus 6 version running amok in systems that matter.
4. To close on a positive note, AI agents can be immensely (to put it lightly) beneficial to human welfare. People with aligned agents at their disposal, solving information asymmetries, creating physical abundance, facilitating optimal matches: this is all on the (potential) horizon. We just need to be serious about building the institutions and developing rules/governance to bring that reality about, instead of something much worse.