The Law of Unintended Consequences always wins ⚖️
One of the biggest dangers from AI is poorly defined objectives...
Sounds boring. Could be catastrophic.
Especially when AI is given enough autonomy to figure things out on its own.
We know objective definition is difficult anyway.
The objective for companies is profit, and they optimise for it.
This is good, because profit is a proxy for value - if someone pays you more to do something than it cost, then it means there was excess value.
But optimising for a proxy can go wrong fast because there are many different ways to achieve it.
You can lay off lots of staff to cut costs and increase profits.
You can cut corners when providing clean water to save time and money.
You can build cheaper bridges with weaker materials.
You could argue long-term these tactics don't work, but it doesn't mean this doesn't happen. The same scenarios will happen with AI.
If we tell it “make sure humanity flourishes,” it might interpret that in a thousand ways — or take a shortcut.
That's fine when it's ChatGPT - you just reprompt it.
But what about when you have an autonomous AI scientist?
If it was told to cure cancer, it might decide it needs more test subjects first.. so would give the population cancer.
We've seen this with social media.
The algorithm was meant to predict what we wanted to see and show it to us.
Instead, it made us more predictable.
It pushed us to the extremes, because it's actually hard to predict how you'll react if you hold a moderate position, but if you have an extreme one it's easier to guess what you'll like or detest.
It became more addictive, because if you're addicted to scrolling it's much easier to get more data from you, and in turn predict what you'd like to see next.
No-one is happy with how social media turned out, but the algorithm is achieving its objective.
Let's make sure we put enough thought into the objectives of the future, we don't need a social media 2.0 with robots 🤖 🤣