Official SWE-bench Account. Follow for updates to the SWE-universe
Joined May 2025
- Tweets 26
- Following 17
- Followers 316
- Likes 65
5 Photos and videos
19 Dec 2025
how do I join SWE-bench slack John
2
2
4
7,219
More SWE-bench environments, tasks, trajectories, and training recipes for all!
Jan 26
SWE-smith is going multilingual!
We have expanded our task synthesis pipeline to JavaScript!
This release includes:
• 6,099 new JS tasks
• Coverage across 34 popular repos
• End-to-end Modal pipeline for fast task synthesis
Scaling agentic training data just got easier.
1
1
803
Join us in SWE-bench slack if you're interested in contributing and using these new datasets! (bottom left of swebench.com)
Expect a lot more to come in the following weeks :)
1
2
416
🚀🚀🚀
Jan 22
PyPI downloads last month
- swebench: 3.1 Million (10M Total)
- swesmith: 1.9M (2.8M Total)
- mini-swe-agent: 164k (636k Total)
We're incredibly grateful ❤️ to the worldwide SWE-* community who continue to build on our work!
New releases on all fronts coming soon
1
538
SWE-bench retweeted
5 Nov 2025
New eval! Code duels for LMs ⚔️
Current evals test LMs on *tasks*: "fix this bug," "write a test"
But we code to achieve *goals*: maximize revenue, cut costs, win users
Meet CodeClash: LMs compete via their codebases across multi-round tournaments to achieve high-level goals
31
99
417
102,938
SWE-bench retweeted
6 Nov 2025
🏆Glad to know that our #ASE25 paper about automated bug repair using MMLM just got the ACM SIGSOFT Distinguished Paper Award🎉 And it is still ranked top #1 in @SWEbench Mulmimodal Track! Thank Kai, Xiaofei @xfxie312, and Jian for the great work!
8 Sep 2025
Excited to announce Kai's latest ASE'25 work, let LLMs not only see bugs, but also fix them:
📄 “Seeing is Fixing: Cross-Modal Reasoning with Multimodal LLMs for Visual Software Issue Repair”
🔗arxiv.org/abs/2506.16136
Ranked #1 on @SWEbench Multimodal!
1
6
1,157
SWE-bench retweeted
10 Sep 2025
Congrats to @Zai_org GLM-4.5 on getting the 7th spot on our SWE-bench Verified [Bash Only] leaderboard!
w/ @KLieret @_carlosejimenez @jyangballin
2
1
12
2,360
SWE-bench retweeted
10 Sep 2025
Super excited to have @anyscalecompute use mini-swe-agent for their large scale runs!
w/ @KLieret @_carlosejimenez @jyangballin
1
4
16
2,750
SWE-bench retweeted
6 Sep 2025
3 out of the top 6 most downloaded datasets on @huggingface are SWE-bench related.
Thanks!!! ♥️
1
7
65
18,458
SWE-bench retweeted
12 Aug 2025
Recent open model scores on SWE-bench Bash Only:
🥇Qwen3-Coder 480B/A35B Instruct - 55.40%
🥈Kimi-K2-Instruct - 43.80%
🥉gpt-oss-120b - 26.00%
See the full leaderboard below! 👇
6
27
211
66,426
SWE-bench retweeted
20 Aug 2025
What if your agent uses a different LM at every turn? We let mini-SWE-agent randomly switch between GPT-5 and Sonnet 4 and it scored higher on SWE-bench than with either model separately. Read more in the SWE-bench blog 🧵
ALT GPT-5 or Sonnet 4. Flipping a coin at every step of your agent beats both
18
20
269
31,881
SWE-bench retweeted
21 Aug 2025
Deepseek v3.1 chat scores 53.8% on SWE-bench verified with mini-SWE-agent. Tends to take more steps to solve problems than others (flattens out after some 125 steps). As a result effective cost is somewhere near GPT-5 mini. Details in 🧵
8
22
156
24,030
SWE-bench retweeted
7 Aug 2025
GPT-5 gets 74.9 on SWE-bench. Wonder what the budget per task is.
3
1
17
4,061
SWE-bench retweeted
31 Jul 2025
What happens if you compare LMs on SWE-bench without the fancy scaffolds?
Our new leaderboard “SWE-bench (bash only)” shows you which LMs are the best at getting the job done with just bash.
More on why this is important 👇
14
27
203
33,013
SWE-bench retweeted
28 Jul 2025
Super exciting to have 3 new open-weight models that all obtain more than 60 on SWE-bench Verified! Looking forward to the results on SWE-bench Multimodal when these models obtain vision capabilities :)
5
3
22
3,019
SWE-bench retweeted
24 Jul 2025
Releasing mini, a radically simple SWE-agent: 100 lines of code, 0 special tools, and gets 65% on SWE-bench verified!
Made for benchmarking, fine-tuning, RL, or just for use from your terminal.
It’s open source, simple to hack, and compatible with any LM! Link in 🧵
ALT mini-SWE-agent: 100 lines is all you need. 65% on SWE-bench verified with Claude Sonnet 4. No special tools, no tool calls, no shell session.
12
74
782
112,121
🎉 Congrats @Alibaba_Qwen @huybery @JustinLin610 and the Qwen team!
Incredible progress in the last year, love to see Qwen continue championing open models for SWE-bench!
22 Jul 2025
>>> Qwen3-Coder is here! ✅
We’re releasing Qwen3-Coder-480B-A35B-Instruct, our most powerful open agentic code model to date. This 480B-parameter Mixture-of-Experts model (35B active) natively supports 256K context and scales to 1M context with extrapolation. It achieves top-tier performance across multiple agentic coding benchmarks among open models, including SWE-bench-Verified!!! 🚀
Alongside the model, we're also open-sourcing a command-line tool for agentic coding: Qwen Code. Forked from Gemini Code, it includes custom prompts and function call protocols to fully unlock Qwen3-Coder’s capabilities. Qwen3-Coder works seamlessly with the community’s best developer tools. As a foundation model, we hope it can be used anywhere across the digital world — Agentic Coding in the World!
💬 Chat: chat.qwen.ai/
📚 Blog: qwenlm.github.io/blog/qwen3-…
🤗 Model: hf.co/Qwen/Qwen3-Coder-480B-…
🤖 Qwen Code: github.com/QwenLM/qwen-code
2
1,282
4
1,156