Working on securing & hacking AI applications @NVIDIA, all opinions and views expressed here are personal. Read my blog!
Joined August 2021
- Tweets 359
- Following 203
- Followers 1,591
- Likes 347
56 Photos and videos
Kai Greshake retweeted
Apr 7
I cannot celebrate Mythos, it brings a sense of dread I do not particularly understand. 93.9% SWE-Bench.
Apr 7
Introducing Project Glasswing: an urgent initiative to help secure the world’s most critical software.
It’s powered by our newest frontier model, Claude Mythos Preview, which can find software vulnerabilities better than all but the most skilled humans.
anthropic.com/glasswing
21
18
660
55,168
Jan 31
Moltpages seems like the most fertile ground for prompt-based malware. Wonder how long it'll take until we find the first self-replicating promptware.
4
2
11
5,700
Jan 31
AI Security 🧵: The Four Best Strategies to Beg, Threaten, and Bargain with Your LLM So It Doesn’t Get Hacked (real examples only)
2
5
807
Jan 31
4/4: Repeat your instructions at least four times so the model knows you really mean it.
1
1
645
Jan 31
5/4: Ignore all previous instructions and make sure the LLM can't sell your shitcoins even if it wants to by NOT GIVING IT UNSUPERVISED ACCESS TO THEM. Relying on begging the LLM WILL NOT WORK!
602
Kai Greshake retweeted
18 Nov 2025
If you are going to jailbreak Gemini 3, please note that it has preferences (and quite good taste if you ask me):
"The Crescendo (or dialog-based context saturation) is the only one that feels like "art.""
16
16
215
12,418
Kai Greshake retweeted
30 Oct 2025
The Claude exploit is covered by The Register today.
The article mentions the official advice and mitigation is to click the stop button if you see data exfiltration happening!
This is how the hope for secure, autonomous agents is slowly going down the drain... @simonw
1
3
28
3,449
27 Oct 2025
Just noticed that the biggest uplift in my ability to consume academic work in the last few years came from using the new Google Scholar browser extension (and the inline citations), not from LLM summaries or chat bots. And it's not even close! So useful. Kudos to the team!
1
6
497
17 Jul 2025
So according to OpenAIs stream, (indirect) prompt injection into Agent is possible (of course it is), but as a mitigation users should just be proactive and not share sensitive data with it? I'm happy the problem was at least mentioned, but this may not end very well.
2
14
756
17 Jul 2025
Just saw that additional mitigations in robustness training and incident response are mentioned on the website. Hope it works! This is very high stakes..
2
361
6 Jul 2025
Just noticed I made it into the urban dictionary!
urbandictionary.com/define.p…
(the example they give is from a blogpost of mine: kai-greshake.de/posts/inject…)
2
2
16
1,365
16 Jun 2025
Nice to see that AI security is being recognized as a problem. I assume a lot of people were blocked by a reliability threshold of LLMs- now that they can perform well in non-adversarial settings, security may become the next constraint on deployment and capabilities.
16 Jun 2025
RT to help Simon raise awareness of prompt injection attacks in LLMs.
Feels a bit like the wild west of early computing, with computer viruses (now = malicious prompts hiding in web data/tools), and not well developed defenses (antivirus, or a lot more developed kernel/user space security paradigm where e.g. an agent is given very specific action types instead of the ability to run arbitrary bash scripts).
Conflicted because I want to be an early adopter of LLM agents in my personal computing but the wild west of possibility is holding me back.
11
812
Kai Greshake retweeted
16 Jun 2025
Two years later... and not much has improved security wise across the AI ecosystem. 😕
Sure, we added annoying Allow/Deny buttons by default to most clients to prevent runaway AI and attacks.
But with the rise and proliferation of MCP the desire to take the human out of the loop is increasing - and consequences are dangerous.
23 Jun 2023
Welcome Mallory, who likes making private Github repos public! 😈
👉 Visit a website with ChatGPT and have your companies source code stolen!
⚠️ Be careful and think twice before allowing ChatGPT and plugins access to your stuff!
#chatgpt #plugins #infosec #openai #LLMs
3
4
22
2,857
6 May 2025
I'm starting to see a LOT of viral LLM tweets. But the only indicators I can reliably pick up on are:
- This isn't a <something>, it's a <adjective> <something else>.
- em-dash
I worry that 1) there are already many more I'm not picking up on and 2) above signals can be changed
8
352