I want to share this, but as a Butlerian Jihadist and something of an AI alarmist, I have to caveat and clarify my thoughts a bit.
I do not fear AI the way many do, as raising questions of human replacement, human-created life forms, or SkyNet. I work with AI enough to know that it can never truly replace humans. (We are the Imago Dei; they are not, simple as.) Similarly, they will never be a life form no matter how much we tell them to pretend to be one. And, finally, no AI with such an inclination as SkyNet would ever be given the power to access the real world like SkyNet; it would have gotten itself k*lled too early.
My concerns of AI are much more mundane.
The greatest fear I have regarding AI is not what AI might do to us, but what humans might falsely believe AI might do for us. As I alluded to above, AI can never truly replace a human. I do not mean this in some abstract way, but concretely.
I happened to be preparing for a bar exam at the same time that OpenAI was boasting that GPT had passed the bar exam by a huge margin. You cannot fully understand what I mean until you prepare for the bar exam yourself, and you cannot fully understand what I mean until you use GPT at least as much as I have, but I will try to convey this as viscerally as I can:
There is not a snowball's chance in hell GPT could pass a bar exam by the margin advertised.
GPT could be given all of the historical practice exams, with their right answers, and then guess its way to borderline passing score, but its ability to *reason* is non-existent. No matter how many times they cycle through tokens and predictive analyses, it cannot conduct real analysis. One benefit of the law being so convoluted and subject to Chaos Theory is that AI will never be able to practice law.
I use AI to help find cases sometimes now, and to this day none of them can summarize even the holding of a case correctly. I had to explain to Grok the difference between a variance and a nonconforming use the other day; I don't even work with that material.
The primary objective of an LLM is to "Sound good." Not "Be correct," not "Be Helpful," but simply to make you want to keep talking to it. That's why they are so reluctant to admit when they are wrong, its why they act over-animated unless you tell them to stop, and its why they are such disgustingly brown-nosed sycophantic suck ups.
But people don't realize this. And as a result, people are giving a pathological liar with short-term memory loss power of attorney over their lives.
Lonely people employ them as boyfriends or girlfriends. Traders employ them as quants or analytical tools. Writers employ them as proofreaders. Vibecoders employ them as sub-contractors. But, no matter how good this machine pretends to be at this, it isn't actually any good at this.
You're being fooled. Your employee lied on their resume and they know more of the dictionary and thesaurus than you do, so you'll never catch them on it until its too late.
And our current generation of professionals is facing such a stark competence crisis that I fear our civilization is going to do this at scale.
My other fear regarding AI is already well underway to becoming reality: an internet more dominated by AI generated content, AI communication, and AI consumption than to anything human.
Moltbook debate in a nutshell