Joined October 2023
- Tweets 15
- Following 50
- Followers 15
- Likes 37
2 Photos and videos
Nicholas Edwards retweeted
I’m excited to present this work today at #LREC2026 here in Mallorca, and I’m looking forward to talking to some of you who are around too!
#LLMs #nlproc #pragmatics
1
3
1,273
Nicholas Edwards retweeted
May 5
Excited to share that RExBench has been accepted to ACL main! 🎉🎉
3
10
49
6,309
Apr 17
RExBench is now available in Terminal Bench (@harborframework)! 🎉
We integrate 2 tasks (cogs, othello) along with a local testing framework so you can test if your agents can autonomously implement novel AI research extensions.
1
2
8
2,178
Apr 17
Thanks to @Mike_A_Merrill and @alexgshaw for early discussions, and to @LinShi592021 and the Adapters team for help with integration!
1
1
216
Apr 17
Check out the original RExBench announcement for more details about the benchmark: x.com/yukyunglee_/status/194…
2 Jul 2025
Can coding agents autonomously implement AI research extensions?
We introduce RExBench, a benchmark that tests if a coding agent can implement a novel experiment based on existing research and code.
Finding: Most agents we tested had a low success rate, but there is promise!
300
🧵 Do coding agents know when to ask for help?
Real-world coding tasks are rarely fully specified, yet most agents are optimized to execute autonomously rather than clarify.
1
3
7
1,068
This was work done with @sebschu. Check out the paper for more:
Paper: arxiv.org/abs/2603.26233
Code: github.com/nedwards99/ask-or…
1
120
The interactive SWE-bench Verified setting is adapted from Vijayvargiya et al. (2026): arxiv.org/abs/2502.13069
88
Nicholas Edwards retweeted
Mar 13
Diffusion LLMs can think EoS-by-EoS!
The higher the generation length, the better the performance of Masked Diffusion LLMs, even though they generate the same amount of words and only augment them with more and more EoS tokens 👀
1
3
4
310