I hand-wrote a 500-LoC RL stack to make hacking on RL research much easier. Most RL stacks are either massive and unhackable, or duct-taped research scripts. I am open-sourcing Mithrl, a modular RLVR stack. Next items on my checklist: adding more complex environment examples, supporting multi-gpu async RL, and QoL fixes. I might scrap external runtime dependencies (Huggingface PEFT vLLM) and write purpose-built, simpler versions from scratch if I feel the need. If you want to experiment with RL and are looking to own sovereign tools, I’d love to get on call, understand your requirements and help integrate for free.

all the AI replies focused on degraded battery and energy usage, maybe all same model?

mirror mirror on the wall, who's the sparsest of them all

Grateful to be a small part of this! Rishi and team were so fun to work with

KV cache at openai actually means Khosla Ventures cash

greg's blogs are awesome

how is there a data structure for everything

bullish!

pessimists are right, optimists make money cautious optimists??

good long-horizon work

in case this helps someone, my claude.md was simple. something along the lines of: """ {brief summary of eval task} {what is being measured and ideally what the agent does to get 1.0 reward} In a loop, run the eval on our harness, sift through the logs of our new run in a subagent and try to find any evidence of reward hacking. once you do, reason about what change to the environment can fix this. Patch it and run the eval again. """ Long-horizon evals are tedious to get right, often the biggest blocker is how long the eval runs itself. meta loops are nice. Sandbox companies are going to make so much money because evals will only run longer

if anyone asks who the best ios dev I know is, it's kabir...

a world where everyone writes beautiful and interesting blogs like this, is a fantastic world to live in

helping out w a long-horizon eval and my env kept getting reward-hacked. I started an overnight claude meta process of patching reward hacks as they come and it worked wonderfully.

henceforth has officially become my favourite word

I built omkaark.com/model-viz/ as an internal tool. its open source, feel free to take and change it without attribution! its a DSL to make model diagrams cc: @rasbt

happy to help, have thus far helped quite a few teams w/ this! it's simple, build a strong eval set (the hard part), setup runners to parallelize across @modal (setup containers w harness) and collect per-step results. the other interesting part is attributing commits to regressions which is a harder one but solvable. put all this as part of your CI and don't merge without looking at the report.

for my next adventure, @michael_trbo and I will be working together to build a tinyLPU! for our first checkpoint, we reinvented the MXM: the language processing unit's matrix multiplication engine. here's how we did it

I find this insane... ran out of modal credits, so codex got it working on docker instead?? actual agency