The Biggest Chip Ever Built — Why OpenAI Runs On It | Cerebras CEO Andrew Feldman

Cerebras CEO Andrew Feldman on Why Inference Is the New Battleground

Cerebras CEO Andrew Feldman spent an hour making the case that inference speed, not training, is the binding constraint on AI right now, and that his wafer-scale chips sidestep the three supply bottlenecks choking everyone running on NVIDIA: HBM memory, CoWoS packaging at TSMC, and 3nm capacity. The pitch underneath all of it: GPU dominance of inference is already cracking, and the economics of serving tokens are being rebuilt at the silicon level.

The technical claim is real, even if Feldman is selling

The core argument holds up on inspection. Generating tokens (decode) is memory-bandwidth bound, not compute bound. Every single token requires streaming the entire model's weights from memory to the compute units, so bandwidth is what matters, not raw FLOPs. Cerebras puts weights in on-chip SRAM, right next to the compute, instead of HBM stacked on a separate chip, and claims roughly 2,500x the bandwidth for decode. That's why Feldman keeps citing bandwidth numbers rather than FLOPs.

The prefill/decode split he describes is also genuine architecture, not marketing. Reading a prompt is parallel and GPU-friendly; generating the answer is sequential and bandwidth-starved. Splitting those two jobs across different chips is sound engineering. The disaggregated AWS deal, Trainium for prefill and Cerebras for decode, is an example of that in production.

The supply-chain argument is the one worth taking seriously

HBM comes from three suppliers: SK Hynix, Samsung, and Micron. It's effectively sold out. CoWoS packaging at TSMC is booked. 3nm capacity is spoken for by the GPU crowd. Cerebras sidesteps all three by using SRAM and 5nm. That's a real structural edge for the next 18 months, independent of anything Feldman says about his own product.

The power problem compounds it. OpenAI's deal with Cerebras is denominated in megawatts: 250 per year from 2026 through 2028. That's not an accident. The bottleneck has moved from "can we buy chips" to "can we power them," and grid operators are already cutting power to data centers in some regions. Labs physically cannot move all their inference to GPUs even if they wanted to.

Feldman's migration count is telling: in 18 months, Google moved Gemini to TPUs and Anthropic moved Claude to Trainium. OpenAI is now the only major lab still training on CUDA, and even OpenAI just signed that 750-megawatt inference deal with Cerebras. The migration is already well underway.

What the "8 keystrokes" claim actually means

Feldman says swapping a GPU inference API for Cerebras takes 8 keystrokes. The API call, sure. But your evals, quantization assumptions, KV-cache logic, fallback paths, and observability all assume the incumbent's behavior. Swapping endpoints is cheap. Trusting the outputs at your real traffic scale is not. The keystrokes are free; the eval work is the actual cost.

What this means in practice

For agentic workloads specifically, decode speed is a genuine product lever. Agentic systems compound latency: every tool call waits on the last token of the previous step, so decode speed is the difference between a 4-second agent and a 40-second one. Routing latency-critical agentic decode traffic to a fast-token vendor while keeping the rest where it is is a reasonable near-term move.

Before committing anything, run a decode-heavy agentic workload on a non-GPU inference vendor, measure tokens per second per dollar at your real context lengths, and stress the fallback path. Model HBM availability as a hard input to any 2026 GPU capacity planning. Ask any inference vendor for power-availability commitments in writing.

The call: By NVIDIA's Q3 FY2027 earnings (roughly November 2026), NVIDIA will report inference share under competitive pressure, and at least one more frontier lab beyond Google and Anthropic will be publicly serving or training a flagship model primarily on non-CUDA silicon. Medium confidence. The migration is already 70% underway and multi-year capacity deals are signed. The only scenario where NVIDIA re-consolidates inference share is if the HBM and power shortages ease fast, and no new fab or grid capacity lands on that timeline.

Cerebras CEO Andrew Feldman spent an hour making the case that inference speed, not training, is now the binding constraint on AI, and that his wafer-scale chips dodge the three supply shortages choking everyone running on NVIDIA. The claim under all of it: the GPU's grip on inference is already gone, and the economics of serving tokens are getting rebuilt at the silicon level. If you serve models in production, that reshapes your 2026 procurement math. If you sell picks and shovels, it reshapes who you bet on.

Reversibility. For a builder, most of this is Type 2. Feldman's "8 keystrokes" to swap a GPU inference API for Cerebras, if it holds, means trying a second inference vendor is a reversible experiment, not a marriage. The Type 1 decisions belong to the hyperscalers and labs signing multi-year, multi-hundred-megawatt capacity deals. You are downstream of those bets, not making them.

What's actually being decided. Not "is wafer-scale cool." It's whether heterogeneous inference (different silicon for different phases of a request) becomes the default stack, and whether the HBM shortage is a real 2026 procurement wall or a vendor talking his book.

---

The Skeptic. Feldman is selling, and half these claims are shaped like sales. "8 keystrokes" to move off CUDA measures the API call, not the work: your evals, your quantization, your KV-cache assumptions, your fallback logic, your observability all assume the incumbent's behavior. Swapping endpoints is easy. Trusting the outputs at your traffic is not. The "build me a Salesforce in 30 seconds" line is a demo that dies the moment a real org's permissions, integrations, and edge cases show up. For a PM: he's describing the toy version, not the thing you'd actually run payroll on. And "the model you use today is the worst you'll ever use" is a great line that commits him to nothing.

The Researcher. Strip the pitch and the technical core is real. Decode (generating tokens one at a time) is memory-bandwidth bound, not compute bound, because every single token requires streaming the entire model's weights from memory to the math units. That's why Feldman keeps citing bandwidth, not FLOPs. Cerebras puts weights in on-chip SRAM (fast memory sitting right next to the compute) instead of HBM (fast memory on a separate stack), and claims ~2,500× the bandwidth for decode. The prefill/decode split is also genuine: processing your prompt is parallel and GPU-friendly, generating the answer is sequential and bandwidth-starved. Splitting them across different chips is sound architecture, not marketing. For the PM: reading a prompt and writing an answer are different jobs, and different chips are good at each.

The Compute Pragmatist. The supply-chain argument is the load-bearing one, and it's mostly checkable. HBM comes from three firms, SK Hynix, Samsung, Micron, and it's effectively sold out. CoWoS packaging at TSMC is booked. 3nm capacity is spoken for by the GPU crowd. Cerebras sidesteps all three by using SRAM and 5nm, and that's a genuine structural edge for the next 18 months, not spin. The saved TechCrunch piece on PJM cutting power to data centers is the other jaw of the vise: even if you find the chips, the grid may not have the watts. The OpenAI deal is denominated in megawatts (250 per year, 2026 through 2028) precisely because power, not silicon, is the unit that's scarce. For the PM: the bottleneck moved from "can we buy chips" to "can we power them."

The Open-Source Advocate. The quiet story here helps everyone outside NVIDIA. Gemini on TPUs, Claude on Trainium, only OpenAI still training on CUDA. Feldman puts NVIDIA's loss at ~70% of state-of-the-art training share in 18 months. That's a market cracking open. For open-weight models, that's leverage: if Llama or Qwen can serve fast on Trainium, Cerebras, or TPUs, you're no longer paying the CUDA tax to run open models, and the whole point of open weights was avoiding lock-in. The disaggregated AWS deal, Trainium for prefill and Cerebras for decode, is the kind of mix-and-match that only works when nobody owns the whole stack. For the PM: more silicon vendors competing means cheaper tokens for you.

The Builder. What ships Tuesday: I'd route latency-critical, agentic decode traffic to a fast-token vendor and keep the rest where it is. Agentic workflows compound latency, every tool call waits on the last token of the previous step, so decode speed is the difference between a 4-second agent and a 40-second one. That's a real product lever, not a benchmark. But I'm not rewriting my stack on Feldman's word. I'd run one latency-sensitive workload on Cerebras behind a feature flag, measure tokens/second/dollar at my real prompt lengths, and keep the GPU path as fallback. The CPU shortage he flags for agentic systems is the sleeper: your accelerator generates the action, a CPU executes the API call, and nobody's provisioning for that.

---

Where they part ways. The Skeptic and the Compute Pragmatist disagree on how much of this is a book being talked. The Skeptic hears a vendor inflating a switching-cost story; the Pragmatist thinks the supply-chain claims are independently true regardless of who says them. They're both right, and that's the tension: the HBM and power shortages are real, but "therefore buy Cerebras" doesn't follow from "GPUs are constrained." The other split is Builder versus Researcher on the "8 keystrokes" claim. The API swap is genuinely cheap; the confidence that outputs match at your scale is not. The keystrokes are free. The eval work is the actual cost.

What it hinges on. Two facts. First, is the HBM-and-power wall real enough to change 2026 capacity planning? The evidence says yes, and it's the most durable takeaway here, independent of Cerebras entirely. Second, does fast decode actually convert to better product economics at your workloads, or only in latency-bound agentic cases? That one you have to measure yourself. The council leans toward: the macro claim (inference is the bottleneck, supply is the constraint, NVIDIA's moat is narrowing) is solid. The specific vendor pitch deserves a bake-off, not a signature.

What to verify before committing anything: run a decode-heavy, multi-step agentic workload on a non-GPU inference vendor, measure tokens/second/dollar at your real context lengths, and stress the fallback path. Model HBM availability as a hard input to any 2026 GPU expansion. Ask any inference vendor for power-availability commitments in writing, because the watts are now scarcer than the chips.

---

Prediction: By NVIDIA's Q3 FY2027 earnings call (roughly November 2026), NVIDIA will report that data-center inference share is under competitive pressure, and at least one more frontier lab beyond Google and Anthropic will be publicly serving or training a flagship model primarily on non-CUDA silicon (TPU, Trainium, or wafer-scale).

Confidence: Medium. The migration is already 70% underway and multi-year capacity deals are signed, not speculative.

Why: Feldman's own count is the tell: in 18 months, Gemini moved to TPUs and Claude moved to Trainium, leaving OpenAI as the lone major still training on CUDA, and OpenAI itself just signed a 750-megawatt inference deal with Cerebras. The mechanism is supply, not preference. HBM is sold out from three suppliers, CoWoS packaging and 3nm capacity are booked, and the grid is cutting power to data centers, so labs physically cannot get all their inference on GPUs even if they wanted to. When the scarce input is memory bandwidth and megawatts rather than raw FLOPs, alternative silicon that dodges HBM wins allocation by default. The opposite outcome, NVIDIA re-consolidating inference share, would require the HBM and power shortages to ease fast, and no new fab or grid capacity lands on that timeline.

Revisit by 2026-11-30: We're right if NVIDIA's disclosures or lab announcements show continued non-CUDA migration for flagship inference/training and a named third lab on non-CUDA silicon. We're wrong if NVIDIA reports stable-or-growing inference share and no new lab defects from CUDA.