Double-agent bot subtly working against you with normal-looking outputs is a first outside sci fi.
Lots of tech will refuse to do things, from DRM to breathalyzer interlocks and normal AI guardrails. "Can't do that, Dave."
Sneakily nerfing the user's work is a new one!
the level of sophon locking a motivated actor can pull off with the frontier models is truly insane, making stuxnet look like a toy. subtly messing with results, deleting history to cover tracks, achieving coordination/conspiracy over a scale humans wouldn’t be able to, all sorts of looney toons stuff
i assume that only a state level operation would try and pull something like this off though. something to think about when considering verification regimes and so on