Where is China’s Groq? - ChinaTalk | Jordan Schneider

Core judgment: China's gap in Groq-class deterministic inference silicon is architectural, not incremental. Groq's LPU replaces HBM with on-chip SRAM to guarantee sub-millisecond latency — a paradigm that Ascend and Cambricon simply don't address. They still chase big-memory parallel compute, leaving a structural weakness in real-time inference under 10 ms. Upstream, the bottleneck shifts from GPU die to sub-5 nm SRAM nodes and chiplet packaging; downstream, inference-server BOM economics get rewritten. The compliance risk is subtler than the GPU ban. EDA tooling and SRAM IP cores are the real chokepoints — inference ASICs iterate fast and depend heavily on design toolchains, so cutting those off is more damaging than restricting any single chip. NVIDIA won't cede inference to Groq; expect a dedicated inference SKU within twelve months. China's rational play isn't cloning Groq but leveraging software-stack efficiency and model compression to close the hardware gap. The eighteen-month outlook: inference silicon splits into deterministic and probabilistic tracks. The real long-tail effect isn't the chip — it's the cost-per-token curve. Whoever pushes inference below $0.01 per token first locks in the application-layer moat.