AI inference demand surges as training slowdown talk fades

The inference pivot is quietly dismantling the training oligopoly that defined the 2023–2024 GPU supercycle. The value anchor has shifted from 'who trains the largest model' to 'who serves a billion concurrent requests at sub-50 ms latency.' This is not a demand contraction—it is a structural reallocation of the hardware stack. HBM density stops being the sole bottleneck; INT8/FP8 datapath efficiency, UALink interconnect maturity, and on-chip cache-to-bandwidth ratios become the new competitive axes. Distributed inference deployment scatters compute demand from hyperscale data centers into regional edge nodes, multiplying data-sovereignty compliance costs. The EU AI Act's deployment-transparency provisions make 'where you infer' as consequential as 'what you infer on.' NVIDIA's CUDA moat loses grip on inference workloads. Google's TPU v5p, Amazon's Trainium2, and a wave of fabless ASICs are carving out 30 % or more of the inference market. Huawei's Ascend line, unburdened by training-scale parity requirements, is gaining APAC penetration its training chips never achieved. Over the next 18 months, inference will consume 70 %+ of AI compute spend and custom silicon will breach 40 % share. The pyramid market flattens: total demand expands while single-point dependency shrinks—a structural tailwind for the entire semiconductor value chain.