Inference is giving AI chip startups a second chance to make their mark

The AI industry is shifting focus from training to inference serving, creating market opportunities for specialized chip startups. Major platforms—Nvidia, AWS, and Intel—are adopting disaggregated inference architectures combining different accelerators optimized for prefill and decode operations. Startups like Cerebras, SambaNova, Lumai, and Tenstorrent are positioning novel architectures (wafer-scale, RDUs, optical, RISC-V) to capture segments of this expanding market.