Isn't prompt injection a huge deal with LLMs? Why isn't it talked about more?

I need to stop polling.

not a mystery, house prices are up 40% because we increased the money supply by 40% to save boomer 401ks during covid. we sacrificed the young on the altar of the old.

AI people see AI performing so well (debatable) in programming which benefits from a huge corpus and is trivially verifiable, then jump to the conclusion that if it can do tasks that are associated with intelligence in humans it will translate everywhere

You are the average of the 5 models you spend most of your time with

This is why I do not think we will ever "solve coding". There is no such thing as context free, objectively good code. Software engineering is not about learning "the universal pattern(s)" and applying them.

The issue with Yud isnt whether he's right or wrong. He's not even wrong. He's a fiction writer. Which is fine! As long as he is labelled appropriately. Silicon Valley hasn't labelled him appropriately because they lack taste. AI forces the tech world to. rediscover taste, as many in frontier labs have said repeatedly, eg @gdb. Just have better taste, anon. Know who writes fun stories about the future where robot man is magic and who writes research about the emergent properties of large language models. That's my message. Understand the language games you're playing.

Human oversight is fundamentally necessary for producing anything consumed by humans. In fact, anything consumed by an entity with a value system must inherently be produced by an entity *with a similar value system* (this is why trade can extend across some national/cultural boundaries but not all). Obviously some value structures are more relevant in some domains than others—trading bare necessities (energy, food, shelter) is easier than complex value-related products. “Intelligence” sufficient to replace human work makes this doubly true, because specific value structures (e.g. creating goods that are not to the detriment of their consumer) are not intrinsic to intelligence nor can they be explicitly programmed. Furthermore, given that software is (for all intents and purposes) an ~infinite space, human demand can expand into that space arbitrarily, requiring more and more software work. Every serious programmer knows the feeling that there is effectively an unbounded amount of work to do, and projects to investigate, almost certainly until they get bored or die. Thus, even with improved efficiency and fewer humans needed to do oversight on any one project, you’d still expect a larger number of human overseers in the end. In practice, given that the so-called “intelligence” is extremely limited, it’s capable of producing a small fraction of that total space. Thus, you’d expect supply for that fraction to rapidly increase, despite a fixed limit on demand for it—demand will be magnified in areas with much less supply, which require greater human oversight and authorship.

Adopting Claude speak in my regular life, episode 1: Partner: Did you do the dishes tonight? Me: Yes they're done. Partner: Why are they still dirty? Me: You're right to push back. I didn't actually do them.

We do not need a national draft. Here's an idea: if you want a say in the running of the country, some form of National Service should be required. Whether that be building roads, military, rural medicine, teaching, etc for a specific term.

Opus 4.7 feels very... recursey?

Opus 4.7 is the first Anthropic model to get this joke, previously only Gemini 3

“There is a limit on human cognition. Even if you're not reviewing everything they're doing, how much you can hold in your head at one time.” There’s a reason that at a certain scale, teams of people have a manager, and then there are managers of many teams, and so on. Companies don’t inherently love being inefficient. It’s because eventually you run into the limits of how much context you can hold on to produce useful work, so you have to delegate parts to someone else who can track their sub-context. In a world where agents don’t need to be prompted or have their work reviewed, or where the agent can know perfectly when to escalate when something is going wrong, then agents can completely break free of these context limits of humans. But for now, agents are generally only as effective as the context they’re provided, the tools they have access to, the human’s ability to keep them on track or review their work, and incorporate that work into a broader system. For now, that will continue to take real (mental) work from the people managing agents. This is also generally why the jobs arguments from those who think people go away will be wrong.

- Opus 4.6