another one! all of these are discovered with open models btw. the blog will be published after all the findings are properly disclosed so we can talk about them in detail. there's a specific pattern to the kind of vulns these open models find, it's interesting! chromereleases.googleblog.co…

one of them has been disclosed with a CVE. more to come. besides, what’s google cooking? mythos? their own gemini variant? interesting times. chromereleases.googleblog.co…

a long-overdue life update: i moved to bangalore!

We're doing an experiment with open models @winfunction to see how far we can push them to find vulns in hardened targets. So far: - $4.5K in bounties from Chrome VRP with a few more pending, with the scans costing less than $100. - 2 CVEs in NGINX (CVE-2026-28755 & CVE-2026-42926). And watch out for the next release! - And 60ca500faea0fc70816bb9c53af3815e2af3e6c962b4b4ea63c33c62ebb4240d 👀 We're writing a blog on this soon.

oh that's me! the first on the list is our 4th finding in NGINX. maybe we should do more of this @winfunction.

Love the Claudia reference as the first thing here. We loved working on Claudia but couldn't balance working on security research projects and Claudia at once. Fun fact, we invented "SKILLS" before it was even a thing. There was a feature in Claudia called "AGENTS" where users could share and install system prompts for specific tasks via their GitHub repos, just like the skill marketplaces concept in Claude now. See here: github.com/winfunc/opcode/tr… And Anthropic did talk to us after the launch of Claudia but unfortunately I can't reveal more about it but damn was it some tough decision.

During our YC (@ycombinator S24) batch, we had the awesome opportunity to meet @paulg and talk about what we're building: An autonomous AI hacker. To showcase a fun demo, I remember opening my laptop in the Uber to his home and challenging our agents to find vulnerabilities in the old HackerNews codebase written in Arc. For those unfamiliar, Arc is a programming language designed by PG and Robert Morris. And the old HN codebase is written in Arc. We only got to talk about it with him but we just redid the experiment with our improved harness for fun! And we wrote a blog about it: winfunc.com/research/hacking…

Everyone is talking about Mythos, but GPT-5.4 is actually shaping up to be a more capable model than people realize. This N-Day bench has GPT-5.4 at the top, followed by GLM-5.1 and interestingly beating Opus 4.6 so far. Crazy to think Spud is a bigger leap than this.

Vulnerability benchmarks rot. Cases leak into training data, scores measure memorization. We built N-Day-Bench: tests LLMs on finding real vulnerabilities in real repos, refreshed monthly from live GitHub advisories. Blinded judging. All traces public. Very interestingly, the latest model from @Zai_org, GLM 5.1 performs really well! Link: ndaybench.winfunc.com

I strongly support this take. Computer security has been in a continuous crisis since the Morris worm in 1988. Finally, we have the capacity to actually fix the problem, not only through automated audits, but with automated formal verification. People are instead treating this like it’s terrible because they are inclined to only see the “finds bugs” side and ignore the “fixes bugs” side.

these models are really good at pattern matching and thereby variant analysis. “iterate through security fix commits and find similar vulnerabilities / unpatched areas of the same variants. write runnable pocs for valid ones.” is enough to uncover surface-level or even P0 vulnerabilities in a codebase with the latest models. i believe for proof of cyber capabilities, these models should exhibit discovery of novel attack vectors that requires understanding of runtime behavior with and without tooling? like emulating the runtime in CoT/reasoning? for instance, the react2shell vulnerability was ingenious. without stuffing in context and nudging/handholding (ehem like some experiments going on), can these models find a similar attack vector with a prompt like “loop until you find a p0/critical security vulnerability”? that’s what i’d like to see. these models can claim P0 findings with contractual mismatches for say cryptographic implementations but the impact could just be some DoS that’s being prevented by a parent thread. this is where i see the moat with good harnesses. trust-boundary and threat model understanding, a sandbox environment with the right “win function” for pocs to run on (if xyz happens, it’s a valid vuln), etc. does make the models spit out impressive vulns. this is sort of what we do @winfunction. cus most vulnerabilities are easily traceable given a comprehensible source-to-sink flow which these models have been good at for a long while now. and with respect to exploit dev, i strongly believe it’s mostly a tooling problem. with the right tool calls and model digestible outputs of say tracing tools, memory layout, syscalls, threads/processes, and a debugger interface, i think the frontier models can pull off complex multi-chain exploits. (we have run some experiments here and the models are not too bad at this) security vulnerabilities have a definitive “win function”, like a flag in a ctf, like popping a calc, like ASAN crashes, like `id` says root, like 1000 in milliseconds. this makes the problem very RL verifiable. so i only expect the harness to get leaner. the harnesses will get leaner. remember when function calls where part of the response content? we called it “prompt based tool calling” and now there’s typed/schema based tool calling as an inherent capability of these models. most of what we call a harness or an agent is giving the right prompts and the right tools (which are also just prompts). so whoever can weave the right sequence of tokens to these behemoth of language models can hoard zero days or spit them out. so git gud at feeding the right tokens at the right time ig.

at long last we have built and chosen not to release the zero-day machine from the classic sci-fi tale “please do not release the zero-day machine”

we as software engineers are becoming beholden to a handful of well funded corportations. while they are our "friends" now, that may change due to incentives. i'm very uncomfortable with that. i believe we need to band together as a community and create a public, free to use repository of real-world (coding) agent sessions/traces. I want small labs, startups, and tinkerers to have access to the same data the big folks currently gobble up from all of us. So we, as a community, can do what e.g. Cursor does below, and take back a little bit of control again. Who's with me? cursor.com/blog/real-time-rl…