This week we had the pleasure of @romovpa presenting "Claudini: Autoresearch Discovers State-of-the-Art Adversarial Attack Algorithms for LLMs" arxiv.org/pdf/2603.24511
He showed us how auto-research (i.e. Claude Code in a loop, equipped with a simple benchmark) can discover attack algorithms on LLMs better than hand-crafted SOTAs. "Attack" here is a prefix added to a prompt that would lead to a fixed string in the output of an LLM with high probability; LLM is a white box with known weights.
Some takeaways:
1. Without seeding this autoresearch with multiple hand-written attacks, it does not work — it combines ideas, but does not come up with novel ideas.
2. Autoresearch does much better than hparam search only.
3. Plenty of options for reward hacking — need to design benchmark with autoresearch in mind.
4. Kimi performed no worse than Claude or Gemini in this task.
5. Always useful to run autoresearch if you have a benchmark to optimise as it is so low effort and powerful.
Lesson titles describe a topic. Outcomes describe what you'll actually do.
We added outcomes to the draft view so you can make more informed decisions about which lessons to focus on before you build.
grasp.study
Grasp Paper Club reviewed Prolonged Reasoning Is Not All You Need: Certainty-Based Adaptive Routing for Efficient LLM/MLLM Reasoning (arxiv.org/pdf/2505.15154) last week.
- Our concise summary -
Sometimes it’s enough to have a short answer. We can train a model to provide short or long answers. Based on the perplexity of the short answer, we can decide if we should switch to long answer. Their solution and literature review of SoTA could be improved.
"If you're not making progress, it's just that the quality of what you're learning isn't up to the mark. It's not that you as a person are not capable." Raj, a web developer in Bengaluru, at our first meet up. More from him and four others in our new film. Come watch the full video on grasp.study.
The most useful learning goals describe what you'll be able to do, not just what you'll know.
Three choices that can help you shape good ones with Grasp:
3. Build on what you've already learned.
New courses work better when they pick up from existing ones rather than starting from scratch. With Grasp you can reference previous courses to create new ones.
If you're not sure how to frame your goal, talk it through with Grasp's mentor before you generate the course. It can help you narrow things down, factor in a target date if you have one, and figure out what's realistic given the time you have.
paths.grasp.study
imo there's fair chance the web develops a parallel markdown universe for agents to read
so you should buy your .md domain for $25 before the Moldovans catch on and start extorting us all
my plan for grasp.md/ is to have the same routes but ~plaintext replication
Introducing Grasp Meetups ☕
A chance to share what you're working on, swap notes with other learners, and meet the team behind Grasp.
We're starting in India:
• Bengaluru: Mon, April 27
• Hyderabad: Sat, May 2
In either city? Send us a message and come join us.
If you're somewhere else in the world, tell us where we should go next. We're picking cities based on where our learners are. form.typeform.com/to/XZkt9cY…
You can now bring any link into Grasp.
Drop in a URL to build a full course around it, or share it with your mentor to discuss.
Anything you find online, now part of your learning. grasp.study
Grasp lessons just got better, every reading and video in your now comes with time estimates and sharper guidance
The best part? One click to reformat any lesson you've already built.
grasp.study
"Can I just pick up where I left off?"
Yes. Now you can.
Shipped this week on Grasp: Continue Learning. Open your course, you're right back in it.
grasp.study