Exploring Direct Tensor Manipulation in Language Models: A Case Study in Binary-Level Model Enhancement: areu01or00.github.io/Tensor-…

My RLM agent can effortlessly process ~80k lines of service logs from CloudWatch in a single go. that's worth like 8 million tokens. The cool part is, after 53 steps, it had spent only 32k "active" tokens* (not through the full 8MM yet atp, more like half). That's nothing for Claude Fable 5 (rip), and weeell within effective context window, so its very "context-efficient". It can go VERY far and I dont even have to handhold it or anything, i'm not worrying about context running out or compactions either. I'm saying I kicked this thing off, almost without any context, and it was able to infer the service architecture based on logs alone, and spot issues my team didn't. In this particular case it was able to narrow down on a specific slice and find a couple issues that flew under the team's radar (AgentCore's throttles, Slack's user_not_found) Very handy. I'll release this as OSS soon (my first release on llm tooling!)

Very important.

I think there's some confusion about what on-policy distillation (OPD) loss actually optimizes. So here's some math: If you write down the *sequence-level* reverse KL between a student and teacher model, the gradient would have two terms (see the image below). - MiniLLM backpropogates through the student's sampling distribution, which makes it akin to policy gradient with dense rewards (but noiser). - OPD does not add this term, thus it akin to supervised learning, simpler to implement and similar to what DAGGER does. As such, almost all of the OPD codebases (e.g, Tinker, verl, huggingface) use the implementation suggested by the GKD paper (one on the left). We did try what MiniLLM suggested but just couldn't get it to work in practice -- overall, my impression back then was that it adds a bunch of complexity and needs stabilization tricks, without clear gains. That said, it is an interesting direction worth studying that why treating distillation as an RL problem doesn't work well or bring benefits (maybe it does in really long-horizon tasks?) PS: MiniLLM changed their paper title (!) after Thinking Machines blog post LOL to catch the on-policy distill cites ;) PPS: GKD is a terrible name

It's now possible to compile Python extensions (C, C , Rust etc) to WebAssembly and distribute them through PyPI such that Pyodide can install them directly simonwillison.net/2026/Jun/1…

Wtf

> what can be done in 100 hours with Opus can done with Fable in 1 Stfu retard

Decided to go to DC next week to talk directly with policymakers. Not sure how impactful it will be but with everything happening, feels like a good time to share more about open-source AI, transparency, concentration of power, the real risks vs the real benefits. Who do you think I should meet there (Congress members, WH people, public orgs,...)?

90-day review period prior to release by the government still applies so it doesn’t really matter. We have decelerated before acceleration even took off.

🤣🤣🤣🤣🤣🤣

Ahhahahaha this was my first thought too but government will allow the release eventually maybe something like 90 days as per their earlier directive to frontier labs to get their models evaluated by them before release.

How do you give a code LLM knowledge of an entire repository without paying for it at every single query? We introduce Code2LoRA: a hypernetwork that turns a repository into its own LoRA adapter. Repo knowledge baked into weights → zero inference-time token overhead.

This graph captures what’s broken about AI evals: they structurally favor closed-source APIs that can route, fallback, ensemble, and optimize behind the scenes with no transparency. No offense, @ArtificialAnlys, but how is comparing one model to two models fair?

What if you could shrink a language model’s memory by 50x in seconds without losing performance? MIT researchers present Fast KV Compaction via Attention Matching. They build compact key-value caches in latent space that preserve attention outputs per head, avoiding slow end-to-end training. Result: up to 50x compaction in seconds on some datasets with minimal quality loss – outperforming prior methods on the speed vs. quality tradeoff. Fast KV Compaction via Attention Matching Paper: arxiv.org/pdf/2602.16284 Code: github.com/adamzweiger/compa… Our report: mp.weixin.qq.com/s/3wDmtR64s… 📬 #PapersAccepted by Jiqizhixin

With the MaxProof framework, M3 exceeded the human gold-medal threshold on both sets. In this paper, we go deeper into the technical path behind our progress in mathematical proof: improving the base model, aligning a verifier, building refinement capability, and designing the test-time scaling framework MaxProof. Here it is 《MaxProof: Scaling Mathematical Proof with Generative-Verifier RL and Evolutionary Search》 huggingface.co/papers/2606.1…

Fable is the new protobuf king

Our Scientific Agent Skills now run on OpenClaw, NemoClaw, and Hermes Agent! Already in use by 160,000 scientists worldwide with 28k GitHub stars. Please let us know how we can improve them for you! github.com/K-Dense-AI/scient…