Groq Reportedly Weighs 3D-Stacked DRAM Over SRAM for Next AI Chip

The current hardware shows the gap. According to NVIDIA, each Groq 3 LPU holds 500MB of SRAM with 150TB/s of on-chip bandwidth, and a Groq 3 LPX rack combines 256 LPUs for 128GB of SRAM and 40PB/s of aggregate SRAM bandwidth. Each rack already adds 12TB of DDR5 alongside that SRAM, TrendForce reports.

For data-center operators, the memory choice shapes how many racks a model needs and how much power each token costs. More capacity per chip could cut the number of LPUs needed to hold a large model, but slower DRAM-class bandwidth would trade against the latency that SRAM gives inference.

Manufacturing is part of the picture. TrendForce, citing Seoul Economic Daily, says Samsung Foundry makes the Groq 3 LPU on its 4nm process at Pyeongtaek, and cites Economic Daily News for TSMC chairman C.C. Wei saying in April that TSMC was working with customers on next-generation LPU development. Cerebras, another SRAM-heavy designer, says its future CS-6 system will combine stacked DRAM with wafer-scale SRAM.

Still unconfirmed: the memory vendor, the stack height, the bandwidth, and any launch date. The report gives none of them.