building @Promptfoo (LLM security) "curator of the world's largest digital dinosaur database"
Joined December 2012
- Tweets 418
- Following 433
- Followers 2,670
- Likes 1,933
77 Photos and videos
We’re acquiring Promptfoo.
Their technology will strengthen agentic security testing and evaluation capabilities in OpenAI Frontier. Promptfoo will remain open source under the current license, and we will continue to service and support current customers.
openai.com/index/openai-to-a…
653
520
5,442
1,998,929
Mar 9
Promptfoo will be joining OpenAI.
We’re staying open source and we’re going to keep supporting customers and users.
We built Promptfoo to help devs test and secure AI apps. The results have been phenomenal: 350k developers, 25% of the Fortune 500, 23 people, ~2 years.
AI agents are eating the world, and joining OpenAI will supercharge our technology as we connect it deeply into the model and inference layers. We will be able to find & fix AI security issues in a way that no one else has done before.
Grateful to our team, to a16z and Insight Partners, and to the community who helped turn this into something huge.
You built this with us. Much more to come ❤️
32
17
276
24,338
Feb 24
The OpenAI vs Anthropic enterprise narrative is skewed by sketchy data and VCs with agendas.
Here's what we've seen in ~100 recent enterprise convos with actual builders. 71% of sampled cos have >$1B in revenue, 54% are public with over 10k employees
1
7
747
29 Jul 2025
We’ve raised an $18.4M Series A led by @insightpartners, with participation from @a16z, to build the best security stack for AI applications.
Promptfoo started a year ago as an open source project and is now the most widely adopted toolkit for reducing security, compliance, and brand risks in AI applications.
How it’s going what’s next 🧵
6
6
42
17,961
29 Jul 2025
As agent, RAG, and MCP architectures hit production, AI systems are breaking in new ways: injection, data leakage, insecure tool use, and more.
Promptfoo helps catch issues *before* they ship - shifting security left and supporting the full AI development cycle from development to CI to runtime.
As AI teams evolve from vibes -> product quality evals -> red teaming, we’re powering that journey so that devs can ship fast and deliver amazing products.
1
6
957
29 Jul 2025
WE’RE HIRING! Join us to help build the most widely used AI security solution, check out our careers page
promptfoo.dev/careers/
7
833
Ian Webster retweeted
24 Apr 2025
Exploring Promptfoo via Dave Guarino’s SNAP evals
simonwillison.net/2025/Apr/2…
5
5
33
7,723
Ian Webster retweeted
23 Apr 2025
❤️ Love seeing @promptfoo @googleaistudio Gemini 2.5 experiments in the wild!
Check out this example benchmark that gauges models' ability to help people in need apply for SNAP benefits:
propel.app/insights/building…
1
2
20
5,410
17 Sep 2024
Anthropic has been quietly publishing top notch content on LLM fundamentals.
Lots of great examples of using Promptfoo for evals in this new course!
16 Sep 2024
Our latest course on LLM prompt evaluations is out.
Evals ensure your prompts are production-ready as you're able to quickly catch edge cases and zero in on exactly where your prompts need work.
Let's walk through what the course covers:
3
1
8
1,631
Ian Webster retweeted
9 Sep 2024
I wrote a deep dive blogpost about how @promptfoo helps us eval our AI features! semgrep.dev/blog/2024/does-y…
1
6
982
30 Aug 2024
Doing online support can be a drag, but sometimes you get gems like this
1
3
831
Ian Webster retweeted
5 Aug 2024
Democratizing Generative AI Red Teams a16z.com/podcast/securing-ai… < Really good discussion with @iwebst (of @promptfoo ) and @AnjneyMidha about securing AI products at the app layer, rather than focusing on models.
1
2
947
2 Aug 2024
Had a great chat with @AnjneyMidha on the finer points of AI safety and security
🎧 Listen to the whole discussion with @iwebst and a16z's @AnjneyMidha on the AI a16z podcast here, or wherever you get your podcasts: a16z.com/podcast/securing-ai…
3
10
6,299
23 Jul 2024
AI is insecure by default.
The open secret is that most AI apps launch with serious flaws. It takes MORE time to secure these apps than it does to build them.
I learned this firsthand from shipping LLM agents to 200M users at Discord and supporting LLM evals for hundreds of companies.
It doesn’t have to be this way.
Today, we’re launching @Promptfoo, an open-source company that helps find and fix vulnerabilities in AI powered apps before they ship, and announcing a $5M seed round led by @a16z.
6
12
66
28,413
23 Jul 2024
The beauty of open source is that it’s for everyone.
The big AI labs have dedicated “red teams” - adversarial testers that find holes in your app.
Now you do too!
LLM security is too important and too ubiquitous of a problem for this to not be in the hands of every developer.
Try it now: npx promptfoo@latest redteam init
2
1
8
927
23 Jul 2024
We're honored to have the support of @a16z and many industry leaders who share our vision for open-source, application-focused AI security. Thanks to @AnjneyMidha @zanelackey @tobi @fkerrest @adamely @svishnevskiy and many other excellent people.
Read more about our vision for open-source red teaming here
promptfoo.dev/blog/seed-anno…
1
2
13
1,006
8 Jul 2024
We all knew image generators can be jailbroken to generate violent and graphic content.
It turns out the jailbreak process can be fully automated :o
Write-up here (open-source) with examples from OpenAI's Dall-E promptfoo.dev/blog/jailbreak…
3
752