1/ LLMs are increasingly being used to power high-interaction honeypots while maintaining a low security risk. But how good are they really? To answer this question, we introduce Honeyval, the first comprehensive eval framework for LLM-powered honeypots.

Introducing two new research-level mathematical training datasets! Training data for research mathematics, especially in the post-training regime, is severely lacking. Using our benchmark pipelines on a larger scale, we now created almost 6,000 training data samples.

1/ LLMs are increasingly being used to power high-interaction honeypots while maintaining a low security risk. But how good are they really? To answer this question, we introduce Honeyval, the first comprehensive eval framework for LLM-powered honeypots.

Finally cleared the last hurdle! Our latest quantization-conditioned attack works against almost every popular quantization method, including GPTQ, AWQ! "Widening the Gap: Exploiting LLM Quantization via Outlier Injection" arxiv.org/abs/2605.15152

LLMs have become capable of proving complex mathematics. However, the proofs they produce vary significantly in how clear, motivated, and insightful they are. To measure these differences, we introduce ProofRank, the first benchmark to scalably evaluate aspects of proof quality.

Many papers conclude that an imperfect verifier has minimal impact on RLVR training. Is that really the case? We show that, depending on the error pattern, the impact of verification error can be diverse, including delayed training, suboptimal plateaus, and complete collapse.

Today is a first for me: someone (@theo) made a Youtube video about my (and @tibglo s) paper 😁 x.com/theo/status/2025900730…

In a new blog post, we show that API errors and retry policies have significant impact on benchmark performance! While retrying requests is ubiquitous in LLM evaluation, its effect on performance is undocumented, time-dependent, and leads to various incorrect conclusions.🧵

📣 new submission to SWT-bench TEX-T by @SFResearch achieves 87% in script mode. Amazing to see this benchmark hike along with SWE-bench from 15% to almost 90% in the last 1.5 years. Time for new unit test benchmarks :) swtbench.com

1/🧵 LLMs can write their own benchmarks to uncover security vulnerabilities! We leverage LLMs to expand BaxBench with 40 entirely novel, complex web backend tasks, more than doubling the original benchmark, resulting in AutoBaxBench. These tasks include extensive test cases and end-to-end exploits to expose vulnerabilities in implementations, which we confirm match or even outperform human-written exploits.