Since many of you guys want answer as much as I do, I will save myself a bit of time by putting down a few of the thoughts I've written about... here in one place.
I remember the days of ELIZA back in the 1960s - yes, I’m that old. It was the first chatbot, roughly 500 lines of code, laughably stupid by today’s standards yet utterly groundbreaking. Many users were convinced a real human therapist was behind the screen. This became known as the ELIZA effect: our well-documented human tendency to anthropomorphize machines and project real intelligence, empathy, or consciousness onto software.
That same effect explains why so many today are fooled by LLMs. Despite giga-scale weights and impressive outputs, LLMs are nothing more than a vastly more sophisticated ELIZA - statistical pattern-matchers, fancy random word generators.
I’ll be the first to admit I don’t know what consciousness is (no one seems to), and my shorthand of “awareness” is lame. But I know what it isn’t. LLMs are NOT conscious and never will be - unless your definition is loose enough to call a parrot intelligent simply because it can 'talk'. You can steer LLMs to say literally anything you want; that’s a feature of a machine: a countersign of sentience. End of story.
I work with frontier LLMs every damn day... all day long. They’re as dumb as a box of rocks. But they’re still incredibly useful tools - IF you throw a TON of engineering at them.
This is my opinion… and worth every cent you’re paying for it. 😊
A user asks:
"When you speak you just utter it word by word, the ever continuing stream of thought coming somewhere from your non-verbal brain.
Words are just an interface for the vast complexity of your brain.
How are you better than LLMs when speaking?"
Good question.
Perhaps its true that when a certain complexity is reached, it becomes 'smart'.
Since we really don't know that much about intelligence or consciousness, who's to say?
My interest is a lot more down the earth: can I make LLMs do real work? Reliable work. After kicking them every day for the last few years I have come to these conclusions, not just by understanding how they work, but from experiences with their work products:
1. They are as dumb as a box of rocks - not always or even usually, but often enough to make them useless out-of-the-box for serious, reliable work products.
2. I have to have a human-in-the-loop (me!) to avoid the accumulating chain errors from this:
0.9 ^ 5 (90% accuracy over five trials)
which computes out to flipping a coin. Nowhere near enough reliability to 'sell' - if one is honest.
So these two factors - which are consequences of how these LLMs work internally - are pretty fatal to LLMs WITHOUT a TON of engineering to mitigate the consequences.
All the other noise out there about whether LLMs are truly 'intelligent' or 'can reason' or are 'alive' or are 'conscious' are philosophical questions IMO... which as much as I enjoy them, can't easily overcome the downsides...
I would agree with the 'Its Alive!' crowd more ... except that working with these LLMs for 8,000 man hours now, there are too many counter-indicators of the stance that LLMs can 'reason' etc.... at least from my work and my use case. And all LLM outcomes are deterministic... so that given enough time a person with paper and pencil can replicate the output of an LLM from its inputs.
So after 55 years, 1 million LOC, 12 software patents, 2 arvix paper in AI/quantum, I don't think we are yet seeing emergent intelligence. Am I on the lookout for it? Yep.
User says:
"All the best programmers I know are starting to write code by hand again."
Seems to be a trend. The valley of disillusionment. Reality strikes back. The hard work begins. The realization that LLMs are a dead-end to AGI. All this coming together at the same time.
Still, I press ahead with my auto-coding tool... as it was designed from the ground up with these realizations
1. Devs want model/lab agnostic coding platforms
2. Devs want desktop privacy
3. Devs want pay-as-you-go model costing
None of the lab coding platforms provide these.
4. It is mathematically impossible for LLMs to get to AGI. If you don't understand this simple engineering limitation, then you don't understand how LLMs ACTUALLY work. Myself and others have written about this quite a bit before so check it out.
5. It is this terrible LLM limitation (#4) that means that a new kind of AI foundation is needed - not a transformer. It CAN NOT be based on next token prediction, but must be based on world view, logic and reasoning.
6. So AGI is decades away IMO. The problem for the labs is that they need a trillion dollars to survive and research in the meantime, which means downplaying #4 and upselling the ridiculous idea that LLM-based AI can replace workers.
7. This is NOT to say that we can't push LLMs into a lot of useful service... in fact my last year has been dedicated to this possibility. But we're talking REAL SWE, not the hacking/slop/vibe that results in 500,000 LOC for Claude, for instance.
About me:
Started coding 55 years ago - never stopped
1 million LOC - at least
8,000 hours working with LLMs since before ChatGPT-3 (Neo)
12 software patents - 7 of those pending in the AI domain
Principle author of COSMOS Revelation 1980s
Principal author of
SEEKERChat.ai RAG 2010s
Principal author of
ViperPrompt.ai 2025
2 arxiv science papers:
arxiv.org/pdf/2601.19929
arxiv.org/pdf/2110.11163
And I was the principle editor of this quantum paper:
arxiv.org/pdf/2306.09122
You can go to my linkedin page to see some of the 5 granted patents. The last 7 (regarding LLMs) are pending.
linkedin.com/in/david-ostby
-05621351/
And yes, having 'AI' code 'for you' will definitely reduce your experience 'coding'. But also remember that current computer languages (like python) are abstractions of lower coding languages, which are abstractions of machine code, which are abstractions of processor bit streams. The first machine I worked on to any degree was a mini-computer and we often opened a panel and 'wire-wrapped' taps into the computer back plane.
So in that sense, if we can perfect our auto-coders a bit more, perhaps they will take their place as the next layer of abstraction. My efforts of the last year are an experiment in exactly this. We shall see.