Characterization of GPU-based Inference for Reasoning-Centric LLMs (Micron, Argonne) - Semiconductor Engineering

Researchers from Micron Technology and Argonne National Laboratory have published a system characterization study on inference scaling for large language models (LLMs), focusing on the shift from trad... Read original →

Micron and Argonne’s study exposes a foundational shift in AI hardware demands: reasoning-centric LLMs are turning inference from compute-bound to memory-capacity-bound. This accelerates adoption of high-density HBM and optical interconnects, benefiting advanced packaging ecosystems like CoWoS. Geopolitically, U.S. export controls are expanding from training to inference chips, pressuring firms reliant on NVIDIA to diversify—spurring ASIC development in Taiwan, China; South Korea; and mainland China. Competitively, AMD and Groq may target low-latency inference niches, while NVIDIA fortifies its moat via software lock-in (e.g., TensorRT-LLM). Within 18 months, the industry will prioritize memory bandwidth over raw FLOPS, making the 'memory wall'—not the compute wall—the critical bottleneck, thereby commercializing near-memory and in-memory computing architectures.