Hot Chips 2026: High Bandwidth Flash promises massive bandwidth and capacity, but its usability is extremely limited - Tom's Hardware

At Hot Chips 2026, GPU IP company OXMIQ Labs revealed the limitations of High Bandwidth Flash (HBF). While HBF offers significantly higher memory capacity than traditional High Bandwidth Memory (HBM),... Read original →

While High Bandwidth Flash (HBF) presents a notable leap in memory capacity, its practical utility remains severely constrained. Technologically, HBF leverages 3D NAND to achieve up to 3.072 TB/s bandwidth, yet limitations in block and page sizes hinder throughput performance, rendering it unsuitable for most AI inference tasks. This restricts HBF’s role to cold data storage—such as MoE expert weights and KV caches—while hot data continues to rely on HBM. OXMIQ Labs’ modeling indicates a 14x capacity increase in 72-GPU racks, but at the expense of aggregate bandwidth. Software compatibility is another hurdle, requiring major updates to frameworks like vLLM for memory management and endurance monitoring. From a competitive standpoint, Nvidia and AMD are unlikely to shift focus from HBM, positioning HBF as a niche solution for specialized applications. In the next 12 months, unless standardized interfaces and toolchains mature, widespread adoption remains unlikely. This development underscores a critical divergence in AI hardware evolution: bandwidth remains paramount, while capacity expansion must be achieved through heterogeneous integration.