7 Photos and videos
10x π«‘π
Eagle3 draft model here: huggingface.co/Inferact/Miniβ¦
DAY 0 ALERT: @MiniMax_AI M3 is now available on HuggingFace & has been added to InferenceX. The M3 architecture has ~428B parameters and ~23B activated parameters. Due to the 10x engineers from @inferact, M3 is already delivering pretty well-optimized performance on @NVIDIAAI B300 Blackwell Ultra on Day 0 @vllm_project! Furthermore, Inferact released their EAGLE3 heads, which enable even greater performance.
Looking forward to Day 1, 2, and 3 performance & the team is grinding on benchmarking Day 0 MI355X performance on InferenceX too.
12
1,889
π Proud of the team's work to land day-0 MiniMax M3 support in vLLM!
Day-0 M3 in vLLM: 1M context, MSA sparse attention, native multimodal, and tool calling for agentic workloads.
Huge effort and partnership across @MiniMax_AI , @NVIDIAAI , @AIatAMD. This is what open-source inference at the frontier looks like. π
Jun 12
π Congrats to @MiniMax_AI on releasing MiniMax M3! Frontier coding and agentic capabilities, native image and video input, computer use, and a 1M-token context window, all in a single open model.
At the heart of M3 is MSA, a new sparse attention architecture: instead of attending densely over the full KV cache, each query scores 128-token KV blocks and runs attention only over the top blocks. That is what makes 1M-token context practical to serve.
M3 runs in vLLM with day-0 support, verified on NVIDIA and AMD hardware:
β¨ MSA sparse attention with dedicated prefill and decode kernels
β¨ 1M-token context serving with prefix caching and chunked prefill
β¨ BF16 and MXFP8 checkpoints, with MoE backends for both Hopper and Blackwell
β¨ Native multimodal input (image video)
β¨ Tool calling, reasoning parsing, and thinking-mode control for agent workloads
Day-0 support like this is a true team effort. Grateful to the teams at @MiniMax_AI, @NVIDIAAI, @AIatAMD, and @inferact, and to the vLLM community for making it happen. π
Deep dive into the implementation, kernel work, and deployment recipes:
π vllm.ai/blog/2026-06-12-miniβ¦
6
1,159
Proud of the vLLM team for shipping day-0 support on Nemotron 3 Ultra!
550B / 55B active, hybrid Mamba-Transformer, 1M context β servable today.
Jun 4
π Day-0 support for NVIDIA Nemotron 3 Ultra on vLLM!
Ready to be served with the latest vLLM stable release, the new open frontier reasoning model is built for long-running autonomous agents:
π§ 550B total / 55B active β Hybrid Transformer-Mamba MoE
π Up to 1M token context
β‘ NVFP4 BF16
π οΈ Tool calling, coding, deep research, orchestration
Read our detailed model launch blog and recipes! recipes.vllm.ai/nvidia/NVIDIβ¦
1
14
2,799
π Excited to collab with @NVIDIARTXSpark pushing local AI agents forward across RTX DGX Spark!
Sharing our hands-on #vLLM #DGXSpark blog with the @vllm_project community.
We showed it off with a live 20 Questions gameβfirst at our office warming, then at #MLSys2026, where curious attendees took turns stumping the model.
Why vLLM DGX Spark? You get a familiar serving workflow on local hardware: streaming responses, memory-efficient KV-cache management, runtime controls for unified memory, and the metrics to deploy on real workloads. βοΈπ
Read the full blog and try it on your Spark π
vllm.ai/blog/2026-06-01-vllmβ¦
Local AI Agents are leveling up across DGX Spark & RTX PCs.
NVIDIA OpenShell is coming to Windows alongside new agentic AI optimizations and creator app updates β including NVIDIA Broadcast 2.2, plus upcoming RTX acceleration for Adobe apps and Blender.
More π
1
1
2
2,095
Honored to be on this list! π
Cheers to AI infrastructure, @vllm_project, and building the future of inference π
The Redpoint InfraRed 100 is now live.
These are the companies building the infrastructure that powers everything happening in AI right now, from world models and agent runtimes to the sandboxes, databases, and security tools agents depend on.
Congratulations to this year's honorees!
Read the full 2026 InfraRed Report: our state of the union on AI and cloud infrastructure π redpoint.com/reports/the-infβ¦
2
21
2,571
π Proud to see the Rust frontend land upstream in @vllm_project!
Huge congrats to @BugenZhao for driving this work and introducing it at @PyTorch Meetup Singapore last week.
A great milestone for the team and the vLLM community. π¦
PR: github.com/vllm-project/vllmβ¦
May 26
π¦ The Rust frontend is officially merged into vLLM!
As GPUs get faster, the frontend has become a real share of CPU time. The new Rust frontend is a drop-in alternative to the Python API server β same engine, same ZMQ boundary. Opt in with VLLM_USE_RUST_FRONTEND=1.
Early numbers: on a preprocess-heavy workload, ~837 req/s vs ~162 req/s for default Python β ~5x in a single process.
A few design choices we're excited about:
β’ Layered crates with clear boundaries
β’ Stream-native pipeline β non-streaming for free
β’ Builds on stable Rust
Huge thanks to @BugenZhao from @inferact for introducing the work at @PyTorch Meetup Singapore.
github.com/vllm-project/vllmβ¦
1
2
20
1,984
That's a wrap on #MLSys2026 in Bellevue! π’
It was great meeting so many of you this past week β researchers, contributors, and friends of @vllm_project. The energy around inference systems right now is something else, and the conversations reminded us why this community matters.
A few highlights from our team:
π€ @rogerw0108 (co-founder, vLLM core maintainer) gave an invited talk, "Rethinking Open Source Contribution in the Age of AI Agents" β a maintainer's-eye view of how AI-generated PRs are reshaping the economics of open source, with concrete examples from vLLM.
π€ @yifandotqiao gave a Lightning Talk, "Rethink LLM Inference Abstractions: New Trends and Challenges in LLM Serving" β on the combinatorial explosion across models, hardware, and workloads, and why serving at scale is increasingly a distributed systems problem.
And of course β congrats to everyone who played 20 Questions with vLLM at our booth π―
Thanks to the MLSys organizers for putting on such a great week. If we missed you in Bellevue, our DMs are open β always happy to talk inference, vLLM, and what we're building.
On to the next one. π οΈ
1
4
47
2,966
Great cohosting this luncheon with @a16z and Mirendil at MLSys 2026 yesterday! π
We brought together top researchers and AI systems engineers for an afternoon of rich conversations on @vllm_project, the frontier of inference, and where AI systems are headed next.
Huge thanks to everyone who joined β the energy in the room was something else. This is exactly the kind of cross-pollination between labs, infra teams, and industry that pushes the whole stack forward.
More to come. π
#MLSys2026 #vLLM
2
8
26
7,084
π Command A is ready to serve on vLLM: day-0.
Frontier open-source, production-ready. Huge congrats to the Cohere and vLLM teams!
Read more π
cohere.com/blog/command-a-plβ¦
Introducing: Cohere Command A
Weβve created our most powerful LLM yet, optimized it to run on as little hardware as possible, and released it open-source for all.
3
18
2,035
Shoutout to our co-founder @KaichaoYou for making this fix and writing up the full story.
From a 2024 hackathon bug β in-tree workarounds in vLLM β PyTorch Foundation TAC β fix landed in PyTorch 2.11.0.
This kind of unglamorous, multi-org debugging makes the whole stack better. π
vLLM and PyTorch worked together to fix a long-standing aarch64 install headache β as of PyTorch 2.11.0, pip install torch on GB200 / GB300 / GH200 just works. π
What changed: PyTorch 2.11.0 now publishes CUDA-enabled aarch64 wheels to the default PyPI index. No more custom --index-url flags. No more transitive dependencies silently swapping your GPU build for the CPU wheel. New users on Grace Hopper and Grace Blackwell systems can follow the standard install instructions and have vLLM work the first time.
In our latest blog, @KaichaoYou (co-founder @inferact, Lead Maintainer @vllm_project) shares the full story: π A 2024 hackathon bug bringing up vLLM on GH200 π§ vLLM's in-tree workarounds (use_existing_torch.py and [tool.uv] build-isolation passthrough) π€ From GitHub issue to PyTorch Foundation TAC discussion π The fix landing in PyTorch 2.11.0, driven by NVIDIA and PyTorch core.
A great example of cross-project collaboration under the PyTorch Foundation umbrella β and a reminder that boring infrastructure wins compound.
Read the full story: pytorch.org/blog/vllm-and-pyβ¦
βοΈ : Piotr Bialecki (@nvidia) β @ptrblck_de, Alban Desmaison (@Meta), Andrey Talman (@Meta), Nikita Shulga (@Meta)
1
5
43
3,592
Weβre at MLSys 2026 in Bellevue this week! β΄οΈ
Come find the Inferact team at Booth #2 in the Evergreen Ballroom.
Talks:
β’ @rogerw0108 (co-founder at Inferact) β βRethinking Open Source Contribution in the Age of AI Agentsβ, Mon 5/18, 11:36 AM
β’ @yifandotqiao (vLLM core contributor) β YPS Sponsor Lightning Talk β Mon 5/18, 11:36 AM
At the booth:
β’ 20 Questions with vLLM β a game with vLLM running on DGX Spark, with prizes π―
β’ vLLM Inferact swag π§’
β’ Inferact team members! happy to talk inference and vLLM
If youβre attending, come say hi, chat about inference, or learn what weβre building!
1
3
28
2,219
We're onto Inferact's second office this year! Yesterday, we finally broke it in with an office warming.
It's amazing to see how far we've come. The vLLM ecosystem has been growing at lightning pace, and we've been lucky to scale alongside it: helping teams serve inference faster, cheaper, and at scale.
Thank you to everyone who made it out yesterday β customers, partners, friends, and the whole Inferact team. It meant a lot to celebrate this milestone together.
We're hiring across all teams. If you want to join one of the fastest-growing AI infra companies and power the next generation of AI, check out our careers page or DM us.
Excited for many more office warmings to come!
11
11
117
16,810
Proud of what the team has shipped here!
And prouder that all this work is in vLLM main or heading upstream π
May 11
vLLM tops the Artificial Analysis leaderboard π
vLLM tops @ArtificialAnlys on DeepSeek V3.2 and ranks among the top deployments of MiniMax-M2.5 and Qwen 3.5 397B.
The leading deployments of these models are now open source.
How each result was built:
πΉ DeepSeek V3.2 β Aggressive op fusion across the attention path collapsed ~33 per-layer kernels down toward ~10.
πΉ MiniMax-M2.5 β Custom EAGLE3 draft trained against the target's own token distribution via TorchSpec, plus a custom QK-norm fusion for MiniMax's TP-aware attention.
πΉ Qwen 3.5 397B β Targeted fusions plus a QK-norm fix for Qwen's linear-attention path.
Every optimization is in vLLM main or on its way upstream.
Huge thank you to @inferact, @digitalocean, @nvidia, @RedHat_AI, and the vLLM community π
Full breakdown π
vllm.ai/blog/vllm-tops-artifβ¦
1
15
2,007
Inferact retweeted
May 11
vLLM tops the Artificial Analysis leaderboard π
vLLM tops @ArtificialAnlys on DeepSeek V3.2 and ranks among the top deployments of MiniMax-M2.5 and Qwen 3.5 397B.
The leading deployments of these models are now open source.
How each result was built:
πΉ DeepSeek V3.2 β Aggressive op fusion across the attention path collapsed ~33 per-layer kernels down toward ~10.
πΉ MiniMax-M2.5 β Custom EAGLE3 draft trained against the target's own token distribution via TorchSpec, plus a custom QK-norm fusion for MiniMax's TP-aware attention.
πΉ Qwen 3.5 397B β Targeted fusions plus a QK-norm fix for Qwen's linear-attention path.
Every optimization is in vLLM main or on its way upstream.
Huge thank you to @inferact, @digitalocean, @nvidia, @RedHat_AI, and the vLLM community π
Full breakdown π
vllm.ai/blog/vllm-tops-artifβ¦
2
30
152
23,001