Introduce Differential Transformer V2 (DIFF V2), an improved version of Differential Transformer. This revision focuses on inference efficiency, training stability, and architectural elegance. We verify the design on production-scale LLMs.

These LLM-Pro-Finance models will be presented at the AGEFI AMTech Event on Nov, 4th. Join us to see live demonstrations and learn how these models can transform financial operations in both English and French-speaking markets.

You see: - a new arch that is better and faster than full attention verified with Kimi-style solidness. I see: - Starting with inferior performance even on short contexts. Nothing works and nobody knows why. - Tweaking every possible hyper-parameter to grasp what is wrong. - Trying to find the efficient chunkwise parallelizable form to squeeze juice out of the GPU - RoPE or NoPE, a question haunting for nights. - Fighting buggy implementation that causes one of the long-context benchmarks drops ~20 pts. - RL diverging. Aligning training-inference numerics. - Dedicated efforts to make sure comparisons are solid and fair. - Going back-and-forth in a pool of adversarial gate-keeping tests, and finally it survives. Great teamwork!

THINGS YOU MISSED ON DIGITAL: 1️⃣ AI Uptake: New EU strategies to accelerate AI in industry & science 2️⃣ Cable security: Securing Europe’s digital backbone 3️⃣ Online Safety: Platforms must protect minors online 4️⃣ Dragon: The LLM powered by @EuroHPC_JU supercomputers