DeepSeek V4 by @deepseek_ai just dropped! SGLang is ready on Day 0 with a full stack of optimizations from architectures to low-level kernels. We also deliver a verified RL training pipeline in Miles (by @radixark) for V4 at launch: 1️⃣ Native "ShadowRadix" Design: DeepSeek V4's hybrid attention is complex. Our new ShadowRadix engine is the first to provide native prefix caching for SWA and compressed KV pools, making 1M context retrieval seamless and memory-efficient. 2️⃣ High-Performance Kernels: - Flash Compressor: IO-aware fused kernels, 10x faster than naive implementations. - Lightning TopK: High-speed indexing for 1M context in just 15µs. - Integrate FlashInfer trtllm-gen MoE, FlashMLA, and MegaMoE kernels 3️⃣ Rich Features: Speculative decoding, HiSparse, Attention DP/TP/CP and MoE TP/EP, and multi-platform support 4️⃣ Verified RL: The open-source RL pipeline: full parallelism (DP/TP/EP/PP/CP), tilelang kernels, tensor-level checked precision, verified with growing reward. Get started immediately with our out-of-the-box Cookbook 👇 Enjoy! #DeepSeekV4 #SGLang #LLM

The standout feature in SGLang v0.5.13 is BCG (Breakable CUDA Graph). It delivers prefill efficiency comparable to PCG (Piecewise CUDA Graph), while being significantly more flexible and compatible with advanced optimizations. BCG is also a powerful debugging tool — it enables eager execution inside CUDA graph replay, so you can easily print debug info or inspect intermediate states. Prototype by @csy_789. @Oasis_a19 turned it into a production-ready prefill optimization. I was fortunate to pick the name for this awesome technique. 😊

🎉 SGLang v0.5.13 is out! First, new model support! Nemotron 3 Ultra, Step-3.7-Flash, Command A , plus new diffusion models: Cosmos3, FLUX.2-Klein, Ideogram 4, LingBot-World, SANA-WM, and Ernie-Image. Here are the highlights for this release: - Speculative Decoding V2 is now the default! Tree drafting (topk>1) for faster generation - Breakable CUDA Graphs now make prefill faster - Qwen 3.5 runs faster on NVIDIA Blackwell with new GDN kernels - HiCache with UnifiedTree on by default for hybrid SWA/Mamba models - SGLang-Diffusion now supports realtime generation! Plus progressive resolution - Multiple performance and feature updates for DeepSeek V4 Thanks to our amazing partners and model makers: @NVIDIAAI @AMD @intel @awscloud @boson_ai @cohere @bfl_ml @ideogram_ai @deepseek_ai @Kimi_Moonshot @Alibaba_Qwen @StepFun_ai @Baidu_Inc @robbyant_brain

🎉 Congratulations to @lmsysorg for setting a new record on NVIDIA GB300 NVL72!

🚀New record on GB300 NVL72: SGLang exceeds 12K tok/s per GPU on DeepSeek V4 Pro 1.6T (FP4, 8K/1K), orchestrated with NVIDIA Dynamo (SGLang) and MTP. Per @SemiAnalysis_ InferenceX benchmarks, performance stays strong across the entire interactivity curve. More to come with @NVIDIAAIInfra 🤝

🎉 Meet Kimi-K2.7-Code from @Kimi_Moonshot, their latest open-source coding model, day-0 support is now live in SGLang! ⏳ More reliable on long, multi-step coding tasks 🧠 30% fewer reasoning tokens ⚡️ Same architecture as K2.5/K2.6, deploy with your existing setup Cookbook: docs.sglang.io/cookbook/auto… Run it now with SGLang!

🎉 SGLang has Day-0 support for MiniMax-M3 from @MiniMax_AI, a native-multimodal MoE reasoning model of ~428B total params (~23B active), 60 layers, 1M context across text, image & video. ✅ Native multimodality: text-image-video fusion from step 0 ✅ MiniMax Sparse Attention (MSA): 9× prefill / 15× decode speedup over M2 at 1M context, 1/20 per-token compute ✅ Frontier-level agentic performance: coding & cowork ✅ MXFP8 across vendors: native on NVIDIA Blackwell & AMD MI350X/MI355X; bf16 build for H200 Cookbook: docs.sglang.io/cookbook/auto… Run it now with SGLang!

SGLang Office Hour 6/11 Higgs Audio V3 TTS x.com/i/broadcasts/1nKOLLdYN…

Hot take: most of your denoising steps don't need full resolution👌 SPEED proves it with progressive resolution growth during denoising, up to 2× speedup, and quality untouched. Now shipping in SGLang Diffusion: FLUX.1 & 2, Z-Image, Qwen-Image, Wan. Ideogram 4 next Congrats @BrianCChao @howard_xhc @YarivLior @GordonWetzstein 👏 and thanks to the support from @hsu_byron and @mick_qian Don't forget to check out this doc: docs.sglang.io/docs/sglang-d…

📣 SGLang Office Hour: How to Build Fast, Modern Voice Cloning with @boson_ai Our Office Hour is back! This week we're joined by Ke and Huapeng from the Boson AI team to talk about Higgs Audio V3 TTS, how to build fast, modern voice cloning that actually sounds human, and their work optimizing SGLang-Omni for TTS production serving. Bring your questions for Ke and Huapeng, see you tomorrow!

Heard about Speculative Decoding? Come hear how Speculative Speculative Decoding (SSD) pushes inference speed to a new ceiling. RSVP 👇

Thank you, New York! We'll see you next time🫡

🙌The community account @sgl_project is back! If you're building with SGLang, this is your space. Follow for releases, Office Hours, tutorials & events👇

Sleeper hit no more 😄 304M downloads and 744% in 3 months — humbled to see SGLang growing alongside this incredible community. Thank you all for building with us!

We're excited to see Vortex built on SGLang! By integrating directly into the serving stack, Vortex turns theoretical sparse-attention speedups into real end-to-end gains. It's a great example of why a programmable, production-grade serving layer matters. Excited to build this with the community! 👏