serial startup founder • early adopter • open source AI, local LLMs, personified autonomous agents

Joined October 2008
67 Photos and videos
Pinned Tweet
Replying to @garrytan
Hot take: I can't see any startup building their critical core operations on Claude Managed Agents or any proprietary harness as investable. The past weeks have shown why it's critical to build on top of an open, neutral framework: ✅ Model diversity (cross-review / critique, less agent groupthink) ✅ Provider-agnostic (outages, random policy changes and suspensions) ✅ Local or fine-tuned LLMs for specialized tasks ✅ Private / E2EE cloud LLMs for tasks needing critical privacy Otherwise your startup will always be resting on an "unstable tectonic plate." All of your IP is in the harness and you truly need full control. And as OSS LLMs improve, you will have (and need) full control over the intelligence layer as well.
10
16
98
30,574
This is the way: a panel of models working together can outperform even the most powerful frontier model (Claude Fable), at lower cost! Model & provider diversity FTW.
Introducing the Fusion API, the smartest compound model in the market. Fusion achieves Fable-level intelligence at half the price. How it works 👇
1
75
Details of vllm 0.23.0 release here: github.com/vllm-project/vllm… KV cache offloading has been working in my testing this AM. Add these settings: --kv-offloading-backend native --kv-offloading-size 32 to offload up to 32gb of KV cache to CPU RAM. Of course this is only going to drive up demand and prices for more CPU RAM. 😂
1
1
84
Big news for local AI users with vllm 0.23.0: KV cache offloading is finally supported for hybrid mamba attention (HMA) models like Qwen 3.6, Gemma 4, DeepSeek V4-Flash, etc. Why does this matter? On 2x3090 (48gb), I can fit ~600K fp8 context on GPU for Qwen 3.6 27B 4-bit, but now I can add 32gb of CPU RAM to double that. Having enough kv cache is one of the biggest bottlenecks for local AI when it comes to concurrency. With 600K KV cache, you can maybe run 3-4 agents simultaneously at full context and without having to constantly evict agents from the KV cache and prefill them all over again as agents make requests. But with additional KV cache capacity that can be retrieved from RAM, you can support more agents. Prefilling 100K context over again when you're out of GPU KV cache space can take a minute or more whereas retrieving offloaded context from CPU RAM is a fraction of that. Congrats @vllm_project!
1
1
9
320
Anthropic has become the definition of an "unstable tectonic plate" for any company (startup or otherwise). And even the US govt. saw this months ago... Too risky to rely on for anything critical long-term.
Jun 10
As a biotech investor it's hard not to think that @AnthropicAI is now an operational risk for small biotech startups. So much trust lost so quickly with the Fable roll-out. @dagarfield @nc_frey
1
35
Good news: a new distillation target for open models, and we will probably see local models at this level within a year. But Anthropic is infuriating. They keep complaining AI will take everyone's jobs and needs to slow down, then THEY are the ones to release the first next-gen model? This will compel all the frontier labs to release their latest in response. Note I don't advocate AI slowing down, but I do advocate authenticity in your culture and messaging.
Introducing Claude Fable 5: a Mythos-class model that we’ve made safe for general use. Its capabilities exceed those of any model we’ve ever made generally available.
1
230
"Show me the incentives and I'll show you the outcome." - Charlie Munger "My job is simple. Create the technology, create the market." - Jensen Huang No doubt in my mind Nvidia will ultimately become the most important contributor to open source AI. Why? Wide availability & use of open models generates demand for inference, which generates demand for hardware. And that's how Nvidia really makes money - not ads, not subs, not seats, not tokens. Anyone making money these other ways has an inherent conflict of incentives when it comes to open source long-term. That includes all the Chinese labs - they're selling subs and tokens. OSS has been a great GTM strategy for them, but we are starting to see signs of pulling back already. Furthermore, increased use of open models will actually generate more demand for CLOSED models as well (they are complementary) - meaning even more inference, training, fine-tuning - all leading back to Nvidia hardware.
1
343
Incredible that one open model can do all this. And possible to train a LoRA for the 16B on a single RTX PRO 6000 or DGX Spark. "Cosmos 3 is the first omni-model for physical AI. It can understand and generate across: language · images · video · audio · action sequences"
Introducing NVIDIA Cosmos 3 We released NVIDIA Cosmos 3 last night. And today, seeing it take the top spots across 8 open model leaderboards feels surreal. We spent months working towards this moment. Here’s the breakdown: The Leaderboard Wins World Reasoning 🏆 #1 open model on VANTAGE-Bench for vision AI 🏆 #1 overall on Traffic Anomaly Reasoning (TAR) World Generation 🏆 #1 open model on Artificial Analysis Image-to-Video leaderboard 🏆 #1 open model on Artificial Analysis Text-to-Image leaderboard 🏆 #1 open model on PAI-Bench for physical AI synthetic data generation 🏆 #1 open model on Physics-IQ, which measures accuracy on physical laws 🏆 #1 open model on R-Bench for world generation quality World Action 🏆 #1 on RoboArena for specialized policy 🏆 #1 on RoboLab for action generation But the leaderboards are only part of the story. The real story is why we built Cosmos 3 in the first place. The Problem Training robots and autonomous systems in the real world is painfully hard. Robots need to try the same thing numerous times before they succeed reliably. Self-driving cars need rare edge cases that may never happen naturally. Smart machines need to understand physics, motion, contact, failure, and surprise. And real-world data is slow, expensive, and sometimes dangerous to collect. At some point, the answer cannot just be “collect more data.” You can’t collect your way out of an infinite physical world. You have to generate it. That… was the question behind Cosmos: Can one model understand the physical world deeply enough to reason about it, simulate it, and generate actions inside it? What We Built Cosmos 3 is the first omni-model for physical AI. It can understand and generate across: language · images · video · audio · action sequences It is not just a VLM. Not just a video generator. Not just a robot policy model. It is all of them, in one single model. That matters because physical AI has been fragmented for a long time. Cosmos 3 is our attempt to collapse that fragmentation. Depending on how you configure the inputs and outputs, the same model can act as a vision-language model, a video/world generator, a world simulator, or a world-action model. No separate architecture required. The Architecture Under the hood, Cosmos 3 uses a dual-tower Mixture-of-Transformers architecture. One tower is autoregressive for reasoning. It handles next-token prediction for language and discrete understanding. The other tower is diffusion-based- for generation. It denoises images, video, audio, and action trajectories. Two towers. Dual-stream joint attention. One shared world representation. Each modality gets its own tools: visual encoders, video VAEs, audio VAEs, and action projectors that can map different embodiments into a unified action space. Action is a first-class modality in Cosmos 3. That’s what makes it more than a video model. It doesn’t just predict and generate what the world might look like. It can connect reasoning and world modeling to physically grounded action. Why This Matters One of the most interesting findings from the ablation work is that training action domains together creates positive transfer. That means adding more embodiments does not just add more use cases. It can actually make the model better. This is the heart of why omnimodal training matters. A shared world representation is not just convenient. It can make each individual task stronger. That’s the part that feels like the beginning of something much bigger. The part I’m most excited about is that Cosmos 3 is fully open. Developers get the models, scripts, optimization, inference endpoints, post-training recipes, datasets, and benchmarks. Everything is available under the Linux Foundation’s OpenMDW 1.1 License. You can use Cosmos 3 out of the box. You can use the VLM, world model, or world-action pieces separately. You can post-train it for your own domain, embodiment, or accuracy target. That’s what makes this feel different. Cosmos 3 is not just a model release. It is the foundation for building intelligence for autonomous machines. For me, Cosmos 3 feels like a step toward a world where physical AI development becomes much more scalable and accessible - to a new age of developers and agents. That’s what we built Cosmos 3 for. I cannot wait to see what you build with it. Download Models on Hugging Face huggingface.co/collections/n… Customize Models on GitHub github.com/NVIDIA/cosmos Read the Tech Blog to Learn More developer.nvidia.com/blog/de…
116
"Unmetered intelligence" - like privacy - should be a fundamental human right.
Our goal is to deliver unmetered intelligence to every home and every desk with Windows. NVIDIA RTX Spark marks a real breakthrough toward that vision. Looking forward to sharing more with Jensen, who will be joining us live from Taiwan, at Build this week! blogs.windows.com/windowsexp…
43
Wow, MiniMax M3 is showcasing head-to-head comparisons vs. the top frontier models: Opus 4.7, GPT 5.5, Gemini Pro 3.1 - and looks strong. Open weights coming in days...
Introducing MiniMax M3: The First Open-Weights Model to Combine Three Frontier Capabilities - Coding & Agentic Frontier: 59.0% SWE-Bench Pro, 66.0% Terminal Bench 2.1, 34.8% SWE-fficiency, 28.8% KernelBench Hard, 74.2% MCP Atlas - MiniMax Sparse Attention scales context to 1M - Natively Multimodal from Step Zero API: platform.minimax.io Token Plan: platform.minimax.io/subscrib… 🚀New! MiniMax Code: code.minimax.io Weights & Tech Report in ~10 Days
2
216
The question is how many parameters, how much VRAM? 🤔
Replying to @MiniMax_AI
@MiniMax_AI M3 has arrived! 😍 "Latest M-series language model for agentic reasoning, tool use, coding, and long-context tasks" Context Window: 1,000,000
1
2
191
Wow! Just tried Step 3.7 Flash. It's the first local model I've tried where I might seriously consider it over Qwen 3.6 27B, at least for a couple key agents. It is FAST, thanks to MoE. It seems a bit sharper on agentic tasks but also has the superior writing / creativity from Gemma 4 31B. Problem is it ties up both an RTX PRO 6000 RTX 5090...
⚡️ Step 3.7 Flash is here: The new frontier is agent efficiency. #1 ClawEval-1.1 (67.1), #1 SimpleVQA Search (79.2), #2 SWE-PRO (56.3), 95.3 on V* Python. Open weights under Apache 2.0. Built for agentic, coding, search, and multimodal workflows — balancing speed, cost, and reliable execution. - 400 TPS. 198B sparse MoE, ~11B active. 256K context, 3 reasoning levels. - Understands UIs, charts, docs, images — then writes code or calls tools to act on what it sees. - Web visual search reaches further: more sources, deeper follow-up. - Reliable tool use — less drift, fewer broken toolcalls. 98% on τ²-bench across all difficulty levels. - Works with Claude Code, KiloCode, Hermes Agent, OpenClaw, and protocols like MCP. - Runs locally on Mac Studio M4 Max, DGX Spark, AMD AI Max 395. GitHub: github.com/stepfun-ai/Step-3… HuggingFace: huggingface.co/stepfun-ai/St… GGUF: huggingface.co/stepfun-ai/St… ModelScope: modelscope.cn/models/stepfun… API: platform.stepfun.ai Blog: static.stepfun.com/blog/step…
2
2
283
Local models I've tried but failed to displace Qwen 3.6 27B for agents: - DeepSeek V4 Flash - Minimax M2.7 - Nemotron 3 Nano Omni (not really for agents) Definitely curious for open-source Qwen 3.7 releases and Minimax M3!
1
366
🤣 Funniest thing I've read in a while. Claude Opus 4.7 in a nutshell.
PICARD: Data, shields up DATA: Brilliant! Shields can reduce damage we sustain. Not immunity. Not hubris. Just prudence. It's not precaution—it's strategy. [camera shakes] WORF: HULL BREACHES ON NINE DECKS DATA: Here's what happened: you told me to raise shields, and I didn't
55
0) the ability to obtain its own electricity and internet access? Solar-powered agents on Starlink soon?
An autonomous agent will need four things to function as a real economic actor: 1) the ability to own (assets, accounts, credentials, intellectual property); 2) the ability to contract (to bind itself and be bound, in a form a counterparty can rely on); 3) the ability to litigate (to sue, be sued, and have judgments enforced); and 4) the ability to persist (to outlast any individual human's involvement, the way Apple outlasts Tim Cook).
1
51
RTX PRO 6000 GPUs now $9,999 up from $8,699 yesterday. 🤯 Same exact price jump at Central Computer also, so it's not just Micro Center.
Unfortunately knew it was just a matter of time the rtx 6000 pro just jumped at Microcenter from $8699 to $9999. $9999 is the highest they've ever had it listed at.
108
vLLM 0.21.0 is out! Biggest updates for local AI folks IMO are related to increasing kv_cache for Qwen 3.6 / Gemma 4: ✅ turboquant kv_cache: try `--kv-cache-dtype turboquant4bit_nc` (or turboquant_k8v4) ✅ nvfp4 kv_cache: 4-bit with hw acceleration if you have Blackwell GPU `--kv-cache-dtype nvfp4` ✅ kv_cache offload to CPU RAM: esp. helpful for single-GPU owners, try `--kv-offloading-size 16` (These are not new vLLM features, but they didn't work for Mamba hybrid attention models like Qwen 3.6 / Gemma 4 until now) And be sure to turn on MTP speculative decoding! For Qwen 3.6: --speculative-config '{"method": "mtp", "num_speculative_tokens": 3}' github.com/vllm-project/vllm…
1
4
256
SF Local AI Meetup today was fantastic! Great meeting so many folks interested in local AI, GPUs, agents and more. Ever since early days in 2022/23 with 2x3090s running Llama 1 7B on first revs of llama.cpp, I've been looking forward to when private, local AI you own & run 24/7 could become truly useful. That day is now here, and sooner than I thought!
The first Local AI Get-Together was a massive success This pic is missing quite a few people who left before we hit the 4-hour mark, but thank you to everyone who stopped by 💙 Local AI is very real, very alive, and apparently willing to talk GPUs, open weights, inference engines, agents, and homelabs for hours We should do this again soon
1
2
196

Replying to @TheAhmadOsman
Thanks a lot for putting this together @TheAhmadOsman. Great meeting everyone!
1
52
Wow, OpenAI really has reclaimed the moral high ground over you know who. Bravo!
Apr 29
Replying to @b_nnett
Not affiliated with Codex. But we do love OSS and congrats. Keep it up and let me know when you hit 1k users and will send you something special!
195
Qwen 3.5 27B API prices are $0.325/M in, $3.25/M out. Same range as GLM-5, Kimi K2.5, Qwen 3.6 Plus. ➡️ Serving locally via 3090 / Mac is a no-brainer! 🧠 The math: 500M input tokens / mo 50M output tokens / mo = $3,900/year in API costs So you are easily paying back your hardware investment within one year. And of course your hardware will not go to zero value in a year (in fact it may be even worth more given the rate prices are rising). The above numbers are well within the potential local generation throughput: maybe 8 hours / day. Math looks even better if you are running tasks 24/7! There are electricity costs, but still a fraction of the token value. And either API providers are pricing based on model capability or it's an expensive model to serve (probably both).
4
1,197