Google achieves 48% throughput gain on Mistral 3 Large via TPU v7 optimizations

A Google engineering team published a technical blueprint detailing how they optimized inference for Mistral 3 Large on Google's Ironwood (TPU v7x) hardware. Mistral 3 Large is a mixture-of-experts (MoE) model, meaning a large model routes inputs to specialized sub-networks. Techniques included hybrid sharding, replacing linear summations with tree reductions, optimizing grouped matrix multiplication and multi-layer attention (GMM/MLA) kernels, and asynchronous scheduling. The result was a 1.5× overall performance gain and up to 48% higher throughput while maintaining benchmark accuracy, relevant to teams evaluating TPUs for large model inference.

A Google engineering team published a technical blueprint detailing how they optimized inference for Mistral 3 Large on Google's Ironwood (TPU v7x) hardware. Mistral 3 Large is a mixture-of-experts (MoE) model, meaning it routes inputs to specialized sub-networks rather than running the full model on every token. Techniques included hybrid sharding, replacing linear summations with tree reductions, optimizing grouped matrix multiplication and multi-layer attention (GMM/MLA) kernels, and asynchronous scheduling. The result was a 1.5× overall performance gain and up to 48% higher throughput while maintaining benchmark accuracy, relevant to teams evaluating TPUs for large model inference.