Google TPUv7 Ironwood beats NVIDIA Blackwell on price-performance for AI inference

Full analysis

For the first time, someone outside Google ran the numbers on its TPUv7 Ironwood chip and it beat NVIDIA on the thing every operator actually pays for: cost per token. SemiAnalysis clocked Ironwood at roughly $0.181 per million tokens versus $0.222 for NVIDIA's B200 and $0.276 for the B300, running a big open-weight model at FP8 (a middle-precision number format). That's up to 50% better bang per dollar on that specific workload.

This is a briefing, so the question is what it means for anyone who buys inference: who wins, who has to adjust, and what to check.

How hard is this to undo? For a reader, easy. Benchmarking Ironwood through Google's InferenceX platform against your own traffic is a spreadsheet exercise, not a rewrite. Moving a production workload off NVIDIA is harder, but nothing here forces that today. What's actually being decided: whether NVIDIA's pricing power on inference has a real second bidder, or whether this is one flattering benchmark. What sets the deadline: nothing urgent. The next real marker is TPUv8i, the chip Google's building with native FP4 support, and the closing of the disaggregated-serving gap SemiAnalysis expects "within months."

The Skeptic. One model, one precision format, one serving setup. Change any of the three and the headline wobbles. Qwen3.5 397B is a mixture-of-experts model, which means only a slice of it fires per token. That's a memory-bandwidth game, exactly where TPUs shine. Run a dense model at high concurrency and the story likely flips. NVIDIA still wins at FP4, the more compressed format the whole industry is chasing to cut costs, and Ironwood can't do it natively. "TPU is finally competitive" is a claim I've heard for a decade. And SemiAnalysis, credible as it is, sells research to the same ecosystem it grades.

The Compute Pragmatist. The result is believable precisely because it's narrow. TPUs are built around matrix-multiply hardware and fat memory pipes, and sparse MoE models are a natural fit. So the FP8 win is architecture doing what it's designed to do. The FP4 gap is not a footnote. NVIDIA's Blackwell throughput in that format is large, and cost pressure is pushing serving toward FP4 fast. The other hole is disaggregated serving, splitting the two phases of inference across separate hardware pools. Google's TPU networking is purpose-built for it. The software just isn't public yet. When it ships, these numbers improve.

The Enterprise Buyer. Here's the catch nobody in the benchmark mentions: to get Ironwood, you rent Google. There's no Ironwood cluster you rack in your own data center, no third cloud reselling it. NVIDIA's edge was never only the chips, it's that AWS, Azure, Oracle, CoreWeave and a dozen others all sell them, which keeps everyone honest on price. A 20-35% saving that locks my inference to one vendor's platform is a different trade than a 20-35% saving I can shop. For a regulated buyer who needs data residency, audit logs, and a second source, "cheaper on Google only" may not clear procurement at all.

The Builder. If you're serving MoE at scale on FP8 today, the money is real enough to run the test this month. An 18-35% cut at 100 tokens per second per user moves a budget. But check your time-to-first-token targets before you get excited: disaggregated serving isn't externalized yet, and that's the feature that keeps chatbots snappy under load. Your NVIDIA stack also carries years of profiling tools and 3 a.m. runbooks that don't port over. Run Ironwood in parallel on your actual traffic mix. Don't migrate on someone else's benchmark.

Where they split

Two real disagreements. The Compute Pragmatist says the FP8 win is architecture and it'll only widen as Google ships the missing software. The Skeptic says the industry is walking toward FP4, where Ironwood loses today, so the win could be obsolete before it matters. Both can't be right, and TPUv8i's FP4 support is the thing that settles it.

The deeper split is the Enterprise Buyer versus everyone counting tokens. A cheaper chip you can only rent from the company that also competes with you (Google runs Gemini) isn't the same as a cheaper chip on the open market. The token math says switch. The procurement math says it depends who you are and how many suppliers you need.

What it hinges on

Strip it down and the call rests on one belief: does Google actually want to sell TPU inference to the world, or does it want to keep the good silicon for Gemini and rent the rest thin? The benchmark proves the chip is competitive. It doesn't prove Google will make it easy, cheap, and widely available enough to pressure NVIDIA's pricing. NVIDIA's whole moat is that everyone sells it. Google's answer so far is a single front door, InferenceX, that it controls end to end.

The council leans one way: this is the first credible dent in NVIDIA's inference pricing story, and it's real on the narrow workload it covers. But a dent in the benchmark is not a dent in the market until the chip is buyable from more than one place and works at FP4.

What to verify before you move budget: run your own traffic mix through InferenceX, dense and MoE, and watch time-to-first-token under real concurrency, not just cost per million tokens.

Prediction: By Google Cloud Next in April 2027, Google will still offer Ironwood/TPUv7 inference only through its own cloud and platforms (no TPU-as-a-service from AWS, Azure, Oracle, or CoreWeave), keeping NVIDIA the default second source and blunting the price pressure this benchmark implies.

Confidence: Medium. Google's TPU history is captive-supply; externalizing to rival clouds cuts against its own cloud sales.

Why: Google has run TPUs internally for a decade and never let another cloud resell them, because keeping the chip captive is how Google Cloud differentiates against AWS and Azure. This benchmark shows the silicon is competitive, but the reason NVIDIA holds pricing power is that a dozen clouds resell it, so a buyer can always shop. For Ironwood's cost win to actually squeeze NVIDIA, buyers need more than one place to rent it, and Google has every incentive to keep it in-house rather than arm its cloud rivals. The opposite outcome, Google licensing TPUs to competing clouds, would mean handing away the one thing that makes Google Cloud distinct, which is why it's the less likely path.

Revisit by 2027-04-30: We're right if Ironwood/TPUv7 inference remains available only via Google's own cloud and platforms. We're wrong if any of AWS, Azure, Oracle, or CoreWeave offers TPUv7 inference as a rentable service by then.

The chip being good was never really in doubt. Whether Google wants a price war with NVIDIA, or just wants Gemini to run cheap, is the open question.