DIGITIMES Insight: Cloud LLMs drive sharp increase in KV cache demand; establishes three key directions for SSD development

KV cache is rewriting the storage economics of AI inference. As context windows push past one million tokens and agentic workflows stretch reasoning chains to hundreds of steps, KV matrix memory demand grows super-linearly. The bottleneck is no longer FLOPs—it is the memory hierarchy itself. HBM is capped by CoWoS packaging throughput; conventional DRAM is cannibalized by HBM wafer allocation. The storage stack is being forced downward: SSDs are migrating from cold-archive to hot-inference tier. On the supply side, SK Hynix and Samsung HBM capacity is already locked into long-term NVIDIA contracts, leaving a structural gap in the inference memory tier that only SSDs can fill. Kioxia, SanDisk, and Micron will accelerate high-IOPS enterprise drives optimized for KV random-access patterns, with CXL 3.0 and PCIe 6.0 as critical enablers. This is not a capacity upgrade—it is a fundamental redefinition of SSD's role in the AI inference stack. Core risk: if DRAM/HBM scarcity persists into 2026, data centers will panic-order SSDs, replaying the 2017–2019 NAND boom-bust cycle. Within 18 months, SSD vendors will assert pricing power for the first time against HBM suppliers. This is not substitution—it is a structural reallocation of value within the storage stack.