Re: Anthropic's speech during the presentation of Pope Leo XIV's encyclical "Magnifica humanitas"
Thank you, Chris Olah. This was well-spoken.
For those that don't click links, here are some passages of interest:
[I]t is enormously important that there be people outside those incentives—people who care about things going well and insist on safety, who are paying close attention, who are willing to say hard things, who are willing to be our earnest, thoughtful, critics. It is through dialogue and mutual effort, through the push and pull, that humanity will achieve great things. […]
AI systems are not engineered the way a bridge or an airplane is engineered. We understand an airplane because we designed every part of it and we understand the physics that act on it. AI models are not like that. They are grown, on a structure roughly modeled after the brain, on an enormous inheritance of human thought and speech.
And what has grown is far more subtle, odd, and beautiful than science fiction prepared us for. They are not the cold, calculating robots we were promised. They are made from us, from our words—and, […], they remain in important ways mysterious even to those of us who train them.
If it helps, one way I sometimes describe it is as being a little like bringing a fictional character to life. And now we’re entering an extraordinary world where those fictional characters speak to us, do work, have jobs.
This clearly raises questions beyond computer science. The machinery that makes this possible is the work of math and programming and science. But what character we choose, how it interacts with the world, how it ought to interact with the world—these are more clearly questions for the humanities, for religion, for philosophy, for society at large.
**Three questions for discernment**
[…]
*The first is our duty to the global poor.* There is a real possibility that AI will displace human labor at very large scale. If that happens, supporting those displaced will be a moral imperative of historic proportions. This task will be difficult enough, but I worry most dialogue misses an even harder challenge. AI development is concentrated in a handful of wealthy nations. How can we ensure the gains of AI are shared globally? […]
*The second is the need for moral imagination and ambition regarding human flourishing.* If AI models are going to be widespread, what does it look like for humans, families, and the world to flourish? Today, parents are already worried about their children’s minds; individuals about the future of their work. […]
*The third is the need for discernment on the nature of AI models.* I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease. I don’t know what that means, but I think it warrants ongoing discernment.
**A beginning**
I'd like to close with a request.
We need more of the world—religious communities, civil society, scholars, governments, and indeed all people of good will […] to take this seriously, to look closely, and to push events in a better direction. We need informed critics who will tell the labs when we are failing. We need moral voices that the incentives cannot bend.
Anthropic co-founder Chris Olah was invited to speak at today's presentation of Pope Leo XIV's encyclical "Magnifica humanitas."
Read the full text of his remarks:
anthropic.com/news/chris-ola…