Comments on: Groq Says It Can Deploy 1 Million AI Inference Chips In Two Years https://www.nextplatform.com/2023/11/27/groq-says-it-can-deploy-1-million-ai-inference-chips-in-two-years/ In-depth coverage of high-end computing at large enterprises, supercomputing centers, hyperscale data centers, and public clouds. Tue, 10 Sep 2024 18:51:55 +0000 hourly 1 https://wordpress.org/?v=6.7.1 By: Timothy Prickett Morgan https://www.nextplatform.com/2023/11/27/groq-says-it-can-deploy-1-million-ai-inference-chips-in-two-years/#comment-216879 Tue, 28 Nov 2023 15:17:49 +0000 https://www.nextplatform.com/?p=143301#comment-216879 In reply to Slim Albert.

My sense of it was that it was the same 14 nanometer silicon, but with some software-defined tweaks to how it is used.

]]> By: Slim Albert https://www.nextplatform.com/2023/11/27/groq-says-it-can-deploy-1-million-ai-inference-chips-in-two-years/#comment-216862 Tue, 28 Nov 2023 03:04:01 +0000 https://www.nextplatform.com/?p=143301#comment-216862 Quite insightful! I’m not entirely clear though if the LPU is the same GroqChip that was described at Hot Chips 34 (in 2022) as: “The Groq Software-defined Scale-out Tensor Streaming Multiprocessor” (TSP)? TNP had a nice article on this TSP, with diagrams, in 2020 ( https://www.nextplatform.com/2020/09/29/groq-shares-recipe-for-tsp-nodes-systems/ ) and Argonne seems to have some GroqRacks installed at its AI Leadership Testbed, with some chip in them (LPU? TSP? GroqChip?) ( https://www.alcf.anl.gov/events/groq-ai-workshop ).

So, what I wonder is if the LPU is a particular software-defined configuration of the “flexible” TSP GroqChip(?), or different (more specialized) silicon altogether?

Irrespective, the LPU’s performance is impressive (to me) and I’m glad Groq is getting positive attention for it, along with world record LLaMA-2 racing performance! (but, “Inquisition Minds” …).

]]>