Red teaming is essential for AI chatbot security. Explore how we uncover vulnerabilities and boost safety. Check out our latest insights! #RedTeaming#ChatbotSecurity#AIChatbots
Eurostar AI vulnerability: when a chatbot goes off the rails - pentestpartners.com/security… by @PenTestPartners
TL;DR
> Found four issues in Eurostar’s public AI chatbot including guardrail bypass, unchecked conversation and message IDs, prompt injection leaking system prompts, and HTML injection causing self XSS.
> The UI showed guardrails but server side enforcement and binding were weak.
> An attacker could exfiltrate prompts, steer answers, and run script in the chat window.
> Disclosure was quite painful, despite Eurostar having a vulnerability disclosure programme. During the process, Eurostar even suggested that we were somehow attempting to blackmail them!
> This occurred despite our disclosure going unanswered and receiving no responses to our requests for acknowledgement or a remediation timeline.
> The vulnerabilities were eventually fixed, hence we have now published.
> The core lesson is that old web and API weaknesses still apply even when an LLM is in the loop.
#Eurostar#PenTestPartners#LLMSecurity#ChatbotSecurity#PromptInjection#GuardrailBypass#SystemPromptLeak#APISecurity#WebSecurity#XSS#InputValidation#ResponsibleDisclosure
Tinker, Tailor, LLM Spy: Investigate & Respond To Attacks On GenAI Chatbots - youtube.com/watch?v=XpokqHFK… at @BSidesTO
Allyn Stott claims that coming, and you aren’t ready; Your first generative AI chatbot incident. GenAI chatbots, leveraging LLMs, are revolutionizing customer engagement by providing real-time, automated 24/7 chat support. But when your company’s virtual agent starts responding inappropriately to requests and handing out customer PII to anyone that asks nicely, who are they going to call? You.
You’ve seen the cool prompt injection attack demos and may even be vaguely aware of preventions like LLM guardrails; but are you ready to investigate and respond when those preventions inevitably fail? Would you even know where to start? It’s time to connect traditional investigation and response procedures with the exciting new world of GenAI chatbots.
In this talk, you’ll learn how to investigate and respond to the unique threats targeting these systems. You’ll discover new methods for isolating attacks, gathering information, and getting to the root cause of an incident using AI defense tooling and LLM guardrails. You’ll come away from this talk with a playbook for investigating and responding to this new class of GenAI incidents and the preparation steps you’ll need to take before your company’s chatbot responses start going viral—for the wrong reasons. - @whyallyn#BSidesTO#AISecurity#LLMSecurity#ChatbotSecurity#PromptInjection#AIIncidentResponse#DFIR#PIIExposure#LLMGuardrails#AISOps#ThreatHunting#RootCauseAnalysis
Grok data leak: hundreds of thousands of private chats indexed by Google
xAI faces a major privacy breach as over 370,000 links to private user chats with the Grok chatbot were exposed and indexed by Google, according to Forbes.
What happened?
The leak traces back to the “Share” feature, which generates unique URLs.
These URLs were crawled by search engines, making confidential data publicly accessible.
Exposed information includes routine conversations as well as sensitive medical data, passwords, uploaded images, and tables.
Impact and response
This is a significant breach of users’ privacy and security on Grok.
xAI has not yet issued an official statement on the incident.
The event echoes a similar situation with ChatGPT, where chat links were also indexed but fixed rapidly.
What users should do
Avoid using the “Share” function until the issue is resolved.
Review and restrict sensitive content in conversations.
Stay updated on official xAI announcements and security news.
#xAI#Grok#DataLeak#Privacy#CyberSecurity#ChatbotSecurity#InfoSecurity#Forbes
This is your last call to join our live free webinar "AI Chatbot Security Frameworks for Business Success" happening today at 9 AM PDT | 7 PM EEST!
Don’t miss the opportunity to explore real-world chatbot vulnerabilities and learn how to secure your AI solutions from the ground up. You'll also have a chance to ask your questions live to our leading AI experts:
🔷 Vadym Nevidomy, AI Solution Architect for Cybersecurity Projects
🔷 Olha Kolomoiets, VP of AI Engineering and Integration
Join us and take your chatbot security strategy to the next level. Register here ➡️ cutt.ly/WrU21vQ9#AIChatbots#CyberSecurity#LiveWebinar#apriorit#AIExperts#ChatbotSecurity
Cyber : ChatGPT पर भरोसा करने से पहले जान लें कुछ रिस्क
डिजिटल के दौर में ChatGPT जैसे AI चैटबॉट्स भले ही काम को आसान बना रहे हों, लेकिन जरूरत से ज्यादा भरोसा आपकी प्राइवेसी और सिक्योरिटी को खतरे में डाल सकता है। देखिय ये खास रिपोर्ट
#AIandPrivacy#ChatbotSecurity#DigitalSafety#PrivacyMatters#TechRisks
As we embrace the power of chatbots, it's crucial to prioritize data security & user privacy. Centralized chatbots are vulnerable to single points of failure and misuse of authority. Decentralized chatbots offer a robust alternative, #PrivacyMatters#ChatbotSecurity