I think one of the biggest risks to humanity posed by the new AI/ML future is stagnation. The machines are only as smart as the training data. The moment actual humans stop producing novel inputs the entire system becomes self referential.
The "society with advanced technology but no knowledge of how it actually works" is a classic science fiction trope. We can get there with ML if we work hard enough.
I was fired from Twitter this morning because during a code review, I had a procedure called "upDo.g" and when Elon asked me "what's upDo.g?" I said "nothing much, what's up with you." He walked away and then my badge stopped working & security escorted me out of the building.
Story to be updated soon with more: Am hearing that several “critical” infra engineering teams at Twitter have completely resigned. “You cannot run Twitter without this team,” one current engineer tells me of one such group. Also, Twitter has shut off badge access to its offices.