Hawkeye Enables AI Agents to Write Hardware-Optimized GPU Kernels

Hawkeye is an open-source framework out of Harvard, Stanford, Together AI, and Caltech that lets AI coding agents write hardware-optimized GPU kernels. A kernel is the low-level routine that determines how fast a chip actually executes matrix math. Writing a good one has always required a scarce, expensive human. Hawkeye claims to match or beat vendor-tuned libraries across NVIDIA Ampere, Hopper, and Blackwell, plus AMD's MI350, and posts an 18.9x speedup over expert-written code on emerging attention variants.

The 18.9x number is real but narrow. It lands on exotic attention variants, linear attention and state-space model scans, where torch.compile can't fuse the pattern and hand-optimization was the only option. Most production inference volume still runs plain softmax attention, where FlashAttention-3, cuBLAS, and cuDNN are already close to the hardware floor. The benchmark suite was also curated by the same team that built the framework. Not fraud, but selection pressure. The win looks spectacular exactly where hand-tuning was hardest, and unremarkable where the volume actually lives.

The AMD angle is what makes this strategically interesting anyway. AMD's MI350 has real FLOPs and a chronic software gap. That gap is the primary reason NVIDIA commands the pricing it does. A framework that writes competitive AMD kernels attacks that gap at precisely the moment AMD is trying to take inference share. If the MI350 results hold outside the benchmark suite, the per-token cost advantage that well-staffed inference teams hold over everyone else starts compressing, and that flows straight into pricing at Together AI, Fireworks, and anyone currently underselling OpenAI and Google.

The less obvious contribution is the taxonomy of unit tests. The framework pairs human-written example kernels with hardware-specific profiling metrics, compressing expert intuition into a machine-readable scaffold that a model can grade its own work against. That is a portable idea, and it is the mechanism behind the speedup.

Before leaning on this, run one eval: take a non-standard attention kernel you already ship, generate the Hawkeye version, and diff outputs across your full input distribution including ragged batch edges. The speedup is easy to measure. Correctness is what ends careers, and kernel bugs are silent until live traffic finds them.

The call: By the March 2027 MLPerf Inference round, at least one inference provider or open-source serving stack (Together AI, Fireworks, vLLM, or SGLang) will ship or publicly demo Hawkeye-generated kernels for a non-standard attention variant in a production or near-production path. Together AI co-authored the framework and runs a commercial inference business where margin is per-token compute cost. It has both the means and the motive. The failure mode is silent correctness regressions forcing a quiet rollback, not lack of interest.

Hawkeye is an open-source framework, out of Harvard, Stanford, Together AI, and Caltech, that lets AI coding agents write hardware-optimized GPU kernels. The kernel is the low-level routine that decides how fast a chip actually runs your matrix math, and writing a good one has always needed a scarce, expensive human. Hawkeye claims to match or beat vendor-tuned libraries across NVIDIA Ampere, Hopper, Blackwell, and AMD's MI350, and posts an 18.9× speedup over expert-written code on emerging attention variants.

For the technical AI leader this is a Type 2 question, easy to reverse. You are not signing a multi-year contract. You clone the repo, point it at one non-standard kernel you already hand-tune, and see if it beats what your engineer wrote. What is actually being decided is whether kernel engineering stays a moat held by a few well-staffed inference teams, or becomes something a mid-size shop can rent from an agent. There's no deprecation clock forcing your hand, so the question is when to test, not whether.

The Skeptic. 18.9× against the Flash Linear Attention Triton kernels is a real number on a narrow target. Ask what fraction of your inference volume actually runs those exotic scans and gates. Most of it is still plain softmax attention, where FlashAttention-3, cuBLAS, and cuDNN are already ground down close to the metal and the win shrinks toward zero. "Matched or exceeded on established workloads" is soft. Matched is not beaten, and vendor libraries ship updates every quarter. The benchmark suite was curated by the same team selling the framework. Not fraud, but selection pressure. For a PM: this looks amazing exactly where hand-tuning was hardest, and unremarkable everywhere the volume actually lives.

The Compute Pragmatist. The MI350 coverage is the interesting part. AMD's hardware has real FLOPs and a chronic software gap, which is the whole reason NVIDIA charges what it charges. An agent framework that writes competitive AMD kernels attacks that gap at exactly the moment AMD is trying to take inference share. If it generalizes, the per-token cost advantage that well-staffed teams hold over everyone else compresses, and that flows straight into pricing at Together AI, Fireworks, and anyone underselling OpenAI and Google. For a PM: the thing keeping AMD chips from being a cheaper NVIDIA is missing software, and this is a machine that writes some of that software.

The Researcher. The agent is not the contribution. The taxonomy of unit tests is. Pairing human-written example kernels with hardware-specific profiling metrics compresses expert intuition into a machine-readable, evaluable scaffold instead of baking it into weights. That is a clean, portable idea. The 18.9× lands where torch.compile cannot fuse the pattern, which is precisely the regime where hand-optimization was the only option. The claim that deserves follow-on stress-testing is the cross-architecture reach: one framework spanning Ampere through Blackwell and MI350 is either a genuine generalization or four benchmark suites in a trenchcoat. For a PM: they found a way to write down what a kernel expert knows so a machine can grade its own work against it.

The Safety Lens. The paper's bigger claim is the one to watch: small amounts of well-curated human knowledge let AI beat the best human baseline. Kernels are a benign place to prove it. Efficient compute for everyone is mostly good. But if that recipe generalizes past kernels, curated expert taxonomies become a lever that a lot of actors can pull without recruiting the experts, including ones outside safety-conscious orgs. The diffuse effect matters more than any one artifact. For a PM: the scary part is not fast kernels. It's the proof that you can hand a machine a small, focused study guide and have it exceed the people who wrote it.

Where they disagree

The Skeptic and the Compute Pragmatist are looking at the same 18.9× and reaching opposite conclusions. The Skeptic says it's a corner case that barely touches production volume. The Pragmatist says the corner cases are exactly where the moat lives, and closing the AMD software gap reprices GPUs regardless of what softmax attention does. Both can be right: the number is narrow AND the narrow thing is strategically loaded.

The Researcher and the Skeptic split on the generalization claim. The Researcher sees a portable method, expert knowledge as a gradeable scaffold. The Skeptic sees one paper, self-curated benchmarks, and a story too clean to trust before adversarial evaluation. That tension is the whole decision.

What it hinges on

Three beliefs. One, does the win survive off the self-curated benchmark, on a kernel Hawkeye's authors did not pick. Two, does the cross-architecture claim hold, or does MI350 quietly lag once you leave the suite. Three, does an agent-written kernel stay correct at batch edge cases, because kernel bugs are silent and catastrophic and won't show up until live traffic hits them.

Before anyone leans on this, run one eval: take a non-standard attention kernel you already ship, generate the Hawkeye version, and diff outputs across your full input distribution including the ragged batch edges and the happy path. The speedup is easy to measure. The correctness is the thing that ends careers.

The council leans toward this being real but narrow, and strategically bigger than the workload it wins on. The AMD angle is why.

Prediction: By the March 2027 MLPerf Inference round, at least one inference provider or open-source serving stack (Together AI, Fireworks, vLLM, or SGLang) will ship or publicly demo Hawkeye-generated or Hawkeye-derived kernels for a non-standard attention variant in a production or near-production serving path.

Confidence: Medium — Together AI's authorship gives it a direct path to its own serving stack.

Why: Together AI co-wrote the framework and runs a commercial inference business whose margin is per-token compute cost, so it has both the means and the motive to fold Hawkeye kernels into the workloads it already serves. The 18.9× win lands specifically on non-standard attention (linear attention, SSMs, hybrid scans) that torch.compile can't fuse, which is exactly the fast-growing architecture class serving stacks are scrambling to support well. The opposite outcome, that this stays a paper with no production adoption, is less likely precisely because one of the four authoring institutions sells inference for a living and open-sources its stack in public. The failure mode isn't lack of interest, it's silent correctness regressions forcing a quiet rollback.

Revisit by 2027-03-31: We're right if a named inference provider or serving framework ships, demos, or publicly commits Hawkeye-derived kernels for a non-standard attention path by the March 2027 MLPerf round. We're wrong if Hawkeye stays confined to the paper and repo with no production serving adoption from Together AI, Fireworks, vLLM, or SGLang.

The AMD story is the one to watch underneath this. If the MI350 kernels hold up outside the benchmark, the interesting consequence isn't faster attention, it's that the software gap keeping AMD cheaper-but-slower starts closing on autopilot.