Code2LoRA distills an entire codebase into model weights, rather than injecting it into a context window. The same drawback of all coding agents is that they don’t know your actual codebase. The usual fixes are to retrieve files into the context window or fine-tune an adapter per codebase, and both run into trouble at scale. RAG pays retrieval cost on every query and degrades as the index grows, while per-repo fine-tuning is expensive to train and brittle, since every commit can force a retrain. Code2LoRA encodes the repo into a vector and uses a hypernetwork to generate a full set of LoRA adapters in one forward pass, with zero inference-time token overhead and no per-repository training. It comes in two variants: 1. Code2LoRA-Static maps a single repository snapshot to an adapter for stable codebases. 2. Code2LoRA-Evo adds a GRU that aggregates commit diffs into a hidden state so the adapter updates as the code evolves rather than going stale. Each update is one GRU step on a stored diff embedding rather than a full re-encode of the repository. Results on RepoPeftBench (604 Python repos, 300K tasks): - Static: 63.8% cross-repo exact match, 9.9 pts over the strongest baseline - Evo: 60.3% cross-repo, 5.2 pts over a single shared adapter, exceeding the per-repo LoRA upper bound with no per-repo training The takeaway is that repository knowledge can be stored parametrically rather than fed through context, and that adapting to a codebase as it changes is a separate problem from conditioning on a single snapshot. You generate the adapter, update it as the code changes, and pay nothing at inference.

iLTM (arxiv.org/abs/2511.15941) is a recent success in tabular learning different from PFN/ICL approaches; it uses a meta-trained hypernetwork pre-trained on thousands of datasets to generate network weights for new tasks.

Check out Programs As Weights (PAW), it does the same hypernetwork to LoRA

Hey! I’m building Code2LoRA - hypernetwork that dynamically compresses code repositories into adapters, instead of spending extra tokens on large context for LLMs! Check it out here:

How do you give a code LLM knowledge of an entire repository without paying for it at every single query? We introduce Code2LoRA: a hypernetwork that turns a repository into its own LoRA adapter. Repo knowledge baked into weights → zero inference-time token overhead.

creating a hypernetwork of satellites or just conquest

LatentSkill turns agent text skills into LoRA weights A hypernetwork compresses reusable procedures into plug-and-play adapters, storing knowledge in weight space instead of context. It cuts prefill tokens by up to 72% while improving ALFWorld success by over 21 points.

I've built out 10's of research papers using CPU inference with tiny models. Equivariant Encrypted Inference, LoRA in the loop Recursive Self Improvement, Hypernetwork LoRA production. Maybe education is the problem.

What? TOGA prunes quantizes LLMs jointly via a hypernetwork trained on real loss. Not greedy layer-by-layer. Result: 21% lower perplexity, 2x faster prefill, 6.5x less memory. geepity.com/2606.07819/

we actually chatted about hypernetwork for continual learning two weeks ago on stream

hey boss really nice thought! I think the idea you found is the concept of a hypernetwork where one model generate the weights of a target model

haha thanks I’m glad you liked it (and the paper). i still think it’s a relevant problem but our solution at the time (the hypernetwork training method) feels a bit outdated by now. we’ve been working on some new stuff in a similar direction though so I hope to give it a refresher.

Code2LoRA Hypernetwork-Generated Adapters for Code Language Models under Software Evolution

For coders (developers, software engineers), Code2LoRA is a big step toward truly personalized, low-friction AI coding assistants. It solves one of the biggest current pain points with tools like Cursor, GitHub Copilot, or other code LLMs: they often lack deep, up-to-date understanding of your specific repository without huge slowdowns or costs. What It Enables in Practice Repo-specific knowledge without the usual trade-offs: No need to stuff entire codebases into massive context windows (which eats tokens, slows things down, and hits limits). No expensive per-repo fine-tuning or retraining. Instead, a small hypernetwork quickly generates a custom LoRA adapter (tiny adapter weights) from your repo snapshot or commit history. This adapter injects project-specific details (imports, APIs, coding style, architecture, conventions) directly into the frozen base model. Static mode (for stable/mature codebases): Feed it a repo snapshot → get a tailored adapter in under 10 ms. The AI then "knows" your project at a deep level for code completion, refactoring, bug fixing, etc. Benchmark results: Matches the performance of full per-repo fine-tuning (63.8% exact match on cross-repo tasks). Evo mode (for active development): The adapter updates incrementally with each commit/diff (using a GRU to track changes). Your AI coding helper stays perfectly in sync as the codebase evolves — no staleness. • • Especially useful for fast-moving projects, monorepos, or teams with frequent changes.

すごい👀　Code2LoRA だって LLM くんにコード書いてもらう時、普通は long prompt 渡すことになるけど、 そうじゃなくて PEFT しようぜ的な話。 で、その PEFT を per-repository でやるのは大変だから、hypernetwork で LoRA の param を直接出そうぜ的な👀 repo を読んで静的に LoRA 出すものと、 repo の diff に追従して動的に LoRA 返す感じのもの（詳細は中身見たほうが良さげ）があり、 なんかどっちもすごい👀 これデフォになってくれるととてもありがたいかも。 CLAUDE.md とかに書くこと減らして良さそうな気がするので！