UCSF Geriatrics

UCSF Geriatrics

EBRAINS

Premakumar Yanda

NASEMCapacityDevelopment

NASEMCapacityDevelopment

NASEMCapacityDevelopment

TheValueist

Andreou Lab @ JHU

TheValueist

TheValueist

@TheValueist

Jan 31

$NVDA $MU $SNDK $LITE EXECUTIVE SUMMARY The SemiAnalysis post accurately identifies a real architectural shift: NVIDIA has described BlueField Astra (Advanced Secure Trusted Resource Architecture) as extending DPU-based trust and control from the north-south (front-end) domain into the east-west (GPU fabric) domain by creating dedicated, out-of-band control connectivity between BlueField-4 and ConnectX-9 SuperNICs, with policy enforcement occurring in SuperNIC hardware rather than via host software. (NVIDIA Developer) The post’s most specific quantitative claim, “8 CX-9 backend NICs,” is not corroborated in NVIDIA’s published Rubin platform description, which states that each Vera Rubin NVL72 compute tray contains 4 ConnectX-9 SuperNIC boards. (NVIDIA Developer) The “8” figure is directionally plausible as a count of PCIe-facing NIC endpoints under SocketDirect/Multi-Host style bifurcation (1 SuperNIC platform potentially presenting as 2 PCIe x16 devices), but that interpretation differs materially from “8 ConnectX-9” physical NICs and is not explicitly confirmed for the NVL72 tray in NVIDIA’s public materials. (NVIDIA Developer) BlueField-4’s inclusion of 128GB of LPDDR5X memory and a 64-core Grace CPU is corroborated by NVIDIA’s technical disclosures, supporting the claim that the DPU is materially more compute- and memory-heavy than prior generations. (NVIDIA Developer) Assertions regarding BlueField-4 being “priced at 75% margins,” limited adoption to 1 customer, and the BlueField software team being effectively single-customer focused are not supported by public evidence; multiple ecosystem participants and at least some service providers have publicly described deployments or integrations with BlueField DPUs, and NVIDIA has publicly communicated broader intended adoption. (VAST Data) WHAT NVIDIA ANNOUNCED AND WHAT IS NEW NVIDIA’s core claim is that BlueField Astra creates a unified, trusted control architecture across 2 networking domains inside the Vera Rubin NVL72 compute tray by establishing dedicated connectivity between the BlueField-4 DPU and ConnectX-9 SuperNICs, routing provisioning and policy through an out-of-band DPU control path, and isolating the SuperNIC control plane from the tenant-controlled host OS. (NVIDIA Developer) In NVIDIA’s description, this is positioned as enabling secure bare-metal multi-tenancy where tenants can use the SuperNIC data path for AI collective traffic but cannot access or modify management functions, with policies programmed via BlueField and enforced directly by SuperNIC hardware. (NVIDIA Developer) BlueField-4 is described as materially more capable than BlueField-3, integrating a 64-core Grace CPU (Arm Neoverse V2), ConnectX-9 networking up to 800 Gb/s, and higher-bandwidth LPDDR5X with 128GB capacity. (NVIDIA Developer) NVIDIA frames this as shifting infrastructure services (networking, storage, telemetry, security) off-host to improve determinism and utilization, rather than as a narrow NIC offload. (NVIDIA Developer) ConnectX-9 is positioned as the endpoint adapter for large AI fabrics with emphasis on programmable congestion control, fairness/isolation, and security features at the NIC to handle correlated burst patterns typical in large-scale AI collective operations and Mixture-of-Experts style traffic. (NVIDIA Developer) Within NVIDIA’s Vera Rubin NVL72 description, each compute tray is stated to contain 4 ConnectX-9 SuperNIC boards delivering 1.6 Tb/s of network bandwidth per Rubin GPU (as presented by NVIDIA, implying a multi-rail or aggregate arrangement at the GPU level). (NVIDIA Developer) CLAIM-BY-CLAIM FACT CHECK AND VALIDATION Claim: “NVIDIA just announced the ability for BlueField-4 DPU to control … backend NICs allowing for unified management of … north-south … & … east-west networks.” This is broadly accurate at the architectural level. NVIDIA explicitly states that BlueField Astra, running on BlueField-4, extends manageability/provisioning/policy enforcement into the east-west fabric via dedicated connections to ConnectX-9, with out-of-band control and unified control of north-south and east-west under a trusted control point. (NVIDIA Developer) The mechanism described is control-plane unification and isolation, not data-plane forwarding of east-west traffic by the DPU. (NVIDIA Developer) Claim: “BlueField-4 DPU … control 8 CX-9 backend NICs.” This specific count is not supported in NVIDIA’s published Rubin platform description, which states 4 ConnectX-9 SuperNIC boards per compute tray. (NVIDIA Developer) The diagram associated with BlueField Astra also visually depicts 4 ConnectX-9 SuperNIC elements rather than 8. (NVIDIA Developer) A plausible reconciliation is that “8 NICs” refers to PCIe-visible NIC endpoints rather than 8 physical ConnectX-9 units. The ConnectX-9 SuperNIC documentation describes SocketDirect/Multi-Host configurations where a 32-lane PCIe interface can be split into 2 separate x16 buses, and “SuperNICs that support Socket Direct can function as separate x16 PCIe cards,” which could make 4 physical SuperNIC boards present as 8 PCIe-attached NIC functions. This interpretation remains inferential for the NVL72 compute tray because NVIDIA’s Rubin platform blog does not explicitly state that the NVL72 tray uses SocketDirect to create 8 discrete NIC endpoints. (NVIDIA Developer) Net assessment: the “8” figure is not validated as “8 ConnectX-9,” and the most defensible reading from public NVIDIA materials is 4 ConnectX-9 boards per compute tray. (NVIDIA Developer) Claim: “Each BlueField-4 has 128GB of LPDDR5X.” This is corroborated by NVIDIA’s technical disclosure describing BlueField-4 memory capacity as 128GB and identifying high-bandwidth LPDDR5X as part of the integrated design. (NVIDIA Developer) Claim: “The whole point of a DPU is to save CPU cores.” This is an incomplete framing relative to NVIDIA’s stated positioning for BlueField-4 in Rubin-era AI factories. NVIDIA explicitly frames BlueField-4 as a control/security/data-movement/orchestration layer intended to improve determinism, isolation, and utilization by moving infrastructure services off-host, including multi-tenant security and control-plane trust properties that are not reducible to CPU-core savings. (NVIDIA Developer) CPU offload is a component of the value proposition, but the Astra design emphasizes trusted control in bare-metal multi-tenant settings and isolation of the SuperNIC control plane from tenant OS. (NVIDIA Developer) Claim: “DPUs are mainly seen as a cost center; BlueField-4 is expensive.” This is a qualitative judgment that cannot be objectively confirmed or refuted from public sources. The hardware composition of BlueField-4 (64-core CPU, 800 Gb/s-class networking, 128GB LPDDR5X) strongly implies a higher bill-of-materials and power/thermal envelope than prior DPUs, consistent with a higher price point in typical enterprise channel structures, but public list pricing is not generally disclosed for leading-edge DPUs, and system-level commercial terms (bundling within rack platforms, discounts, software attachment) dominate realized pricing. (NVIDIA Developer) Claim: “BlueField-4 is priced at 75% margins (4x cost of goods), so cost advantages fade away.” The arithmetic is correct: a 75% gross margin implies COGS equals 25% of revenue and revenue equals 4x COGS. This does not validate the premise that BlueField-4 is priced at 75% gross margin on a product basis. NVIDIA’s consolidated GAAP gross margin has been reported at 75.0% for FY2025, indicating that 75% is a real corporate-level datapoint in the relevant era, but product-level gross margin for BlueField-4 is not publicly disclosed and would be expected to vary materially by product mix and deal structure. (NVIDIA Newsroom) Additionally, for GPU-centric AI systems, the dominant ROI lever for infrastructure offload and control-plane security is often GPU utilization, reliability, and multi-tenant operability rather than CPU BOM reduction. NVIDIA explicitly argues that off-host infrastructure services improve determinism and utilization. (NVIDIA Developer) The “CPU-cores saved vs DPU cost” framing is therefore an incomplete economic model for Rubin-era architectures intended to enable secure bare-metal multi-tenant offerings. (NVIDIA Developer) Claim: “Largest hyperscalers develop their own cheap DPU (AWS Nitro, Google IPU, Azure SmartNIC).” The build-vs-buy observation is directionally accurate. Amazon Web Services describes the Nitro System as offloading virtualization and I/O functions to dedicated hardware (Nitro Cards) with a locked-down administrative model, consistent with an in-house DPU/SmartNIC strategy. (Amazon Web Services, Inc.) Google Cloud describes an Infrastructure Processing Unit (IPU) co-designed with Intel as an on-host offload SoC to improve isolation and performance, and positions Titanium as a tiered offload architecture spanning on-host and scale-out offloads. (Google Cloud) Microsoft has publicly documented Azure Accelerated Networking (AccelNet) as offloading host networking to custom FPGA-based SmartNICs, including experience operating at large host counts, consistent with a long-running internal SmartNIC program. The adjective “cheap” is not objectively verifiable; hyperscaler in-house silicon can be lower unit cost at scale, but frequently reflects significant engineering and ecosystem costs that do not map cleanly to a simple “cheaper NIC” narrative. (Amazon Web Services, Inc.) Claim: “Only 1 customer has adopted BlueField DPUs; BlueField software team only works on that customer’s feature requests.” This is not supported by public evidence. Public materials indicate multiple enterprise and platform integrations with BlueField DPUs, including VMware support of NVIDIA BlueField DPUs in vSphere Distributed Services Engine, and multiple storage/security ecosystem integrations. (Dell Technologies Info Hub) VAST Data has publicly stated that a BlueField DPU-based architecture was designed with NVIDIA and is being deployed at CoreWeave, directly contradicting an implication that BlueField has only a single real-world customer footprint. (VAST Data) Claims about internal allocation of NVIDIA’s BlueField software engineering resources are not publicly verifiable and should be treated as unsubstantiated anecdote absent direct evidence. TECHNICAL SIGNIFICANCE OF “FRONTEND DPU CONTROLS BACKEND SUPERNICS” The meaningful technical novelty is the explicit separation of tenant-controlled host software from the control plane of the east-west NICs that drive GPU collective performance, paired with an out-of-band trusted control path anchored on the DPU. NVIDIA describes the control plane as completely isolated from the host OS so that bare-metal tenants cannot tamper with provisioning or policy, with policies routed via the DPU and enforced in SuperNIC hardware. (NVIDIA Developer) This is structurally aligned with how hyperscalers have historically treated SmartNIC/DPUs: the offload device is not merely a throughput accelerator but a hard trust boundary to reduce the blast radius of tenant compromise and to make compliance/audit and multi-tenant isolation feasible at scale. (Amazon Web Services, Inc.) From a performance engineering perspective, the design attempts to avoid a common pitfall: pushing high-rate east-west data-plane traffic through a general-purpose DPU can create bottlenecks and latency variance. NVIDIA’s description instead places the SuperNIC as the data-plane endpoint optimized for congestion control and fairness, while the DPU provides management and policy. (NVIDIA Developer) This division is consistent with the stated goal of keeping GPU collective operations from being bottlenecked at the network edge. (NVIDIA Developer) The “8 backend NICs” phrasing is best interpreted cautiously. Public NVIDIA text indicates 4 ConnectX-9 SuperNIC boards per compute tray. (NVIDIA Developer) If the platform uses SocketDirect-style bifurcation, the number of PCIe-facing NIC devices can exceed the number of physical SuperNIC boards, and that could be the source of an “8” count in a slide narrative. However, that remains interpretive; the precise internal wiring, PCIe topology, and how many PCIe endpoints are exposed to the BlueField control plane is not explicitly enumerated in the Rubin platform blog. (NVIDIA Developer) ECONOMIC AND ADOPTION CONTEXT The SemiAnalysis post frames the DPU economic case primarily as “CPU cores saved vs DPU price,” implying that high gross margins negate cost advantages. That model is incomplete for GPU-dominant AI systems where the highest-cost resource is GPU time, and the highest-value outcomes are typically higher GPU utilization, more deterministic job completion, less operational overhead, and secure multi-tenant enablement. NVIDIA explicitly positions BlueField-4 as moving infrastructure services off-host to improve determinism and utilization, and as enabling secure bare-metal multi-tenant operation via Astra. (NVIDIA Developer) Under that framing, the relevant counterfactual is not merely “fewer CPU cores purchased,” but “higher effective tokens/sec, fewer tail-latency or collective stalls, reduced job interference, and lower risk/cost of tenant isolation failure,” each of which can dominate DPU cost in high-end AI deployments even at high hardware gross margins. (NVIDIA Developer) Cost and adoption sensitivity should be segmented: Hyperscalers with mature in-house SmartNIC/DPU stacks (Nitro, Titanium/IPU, Azure SmartNIC) have strong incentives to keep control-plane software, telemetry, and isolation models consistent across fleets, making third-party DPUs structurally harder to adopt outside of turnkey vendor-managed rack offerings. (Amazon Web Services, Inc.) CSPs and GPU cloud providers without hyperscaler-scale silicon programs, and enterprises deploying vendor-integrated racks, may rationally prefer an “appliance-like” platform where security and east-west control are delivered as part of the rack-scale reference architecture rather than being engineered in-house. NVIDIA’s public description of Astra is explicitly targeted at CSPs offering bare-metal GPU nodes securely in multi-tenant environments. (NVIDIA Developer) Ecosystem adoption signals extend beyond hyperscalers. Public disclosures show deployments/integrations of BlueField DPUs with storage/networking stacks, including VAST Data deployments at CoreWeave and VMware vSphere DSE support. (VAST Data) This evidence conflicts with any categorical claim of “1 customer” adoption, though it does not quantify share, revenue scale, or breadth across top-4 hyperscalers. KEY UNCERTAINTIES AND MONITORABLES Physical topology clarity: Whether “8 backend NICs” reflects 8 physical SuperNICs, 8 ports, or 8 PCIe-exposed NIC endpoints under SocketDirect/Multi-Host. Public NVIDIA documentation supports 4 SuperNIC boards per compute tray, while ConnectX-9 documentation supports architectures that can increase the apparent NIC endpoint count. (NVIDIA Developer) Commercialization pathway: BlueField-4 is described as debuting with Vera Rubin rack-scale platforms and potentially becoming available for other server platforms, which implies adoption may be tightly coupled to Rubin NVL72 shipments in early phases. (CRN) Pricing and margin reality: Corporate gross margin has been reported around mid-70% in the period referenced, but product-level gross margins and the extent of platform bundling are not disclosed. (NVIDIA Newsroom) Evidence of hyperscaler penetration vs “turnkey rack” penetration: Public disclosures support multi-party ecosystem use, but the degree of adoption inside top hyperscaler fleets (outside of buying complete NVIDIA racks) remains an empirical question. (VAST Data) BOTTOM LINE ASSESSMENT The SemiAnalysis post is directionally correct that NVIDIA has articulated a new DPU-centered control-plane architecture (BlueField Astra) intended to unify management and security across north-south and east-west networking domains for bare-metal multi-tenant AI infrastructure. (NVIDIA Developer) The post’s specific “8 CX-9 backend NICs” phrasing is not validated as stated; NVIDIA’s own Rubin platform description indicates 4 ConnectX-9 SuperNIC boards per compute tray, and “8” is more plausibly interpreted as an endpoint-count artifact (PCIe bifurcation/multi-host) rather than 8 physical ConnectX-9 units. (NVIDIA Developer) The claim that BlueField-4 contains 128GB LPDDR5X is corroborated. (NVIDIA Developer) Assertions about BlueField being a single-customer product and about product-level 75% margins are not substantiated by public evidence; corporate gross margin around 75% is real, but product-level margins and customer concentration are not disclosed, and publicly visible integrations point to a broader ecosystem footprint than “1 customer.” (NVIDIA Newsroom)

SemiAnalysis

@SemiAnalysis_

Jan 30

IMPORTANT: NVIDIA just announced the ability for their new BlueField-4 DPU to control 8 CX-9 backend NICs allowing for unified management of both frontend north-south network & backend high speed east-west networks. The only issue with this is that DPUs are mainly seen as an cost center and BlueField-4 is fucking expensive especially considering that each BlueField-4 has 128GB of LPDDR5X. Remember, the whole point of an DPU is to save CPU cores, but when the DPU is priced at 75% margins (i.e. 4x cost of goods), the cost advantages of BlueField quickly fade away. This is why many of the largest hyperscalers decide to develop their own cheap DPU (AWS Nitro, Google IPU, Azure SmartNIC, etc) instead of buying pricy NVIDIA DPUs. So far we have only seen adoption & real use of BlueField DPUs from 1 customer & basically the entire BlueField software team only works on that customers feature requests.

2,083

AccelNet Summer school on Human Data Management and Modelling/AI

Johns Hopkins Data Science and AI Institute

12: Making the ‘Net’ Work with Oss & Amu

Multilayer network science: theory, methods, and applications

Announcing General Availability for AccelNet on Windows Server 2025 | Microsoft Community Hub