The Future of Frontier Model Architectures with Walter Goodwin, Fractile Founder and CEO

Full analysis

Walter Goodwin, founder and CEO of a chip startup called Fractile, went on Sarah Guo's No Priors podcast and made one claim worth sitting up for: the thing holding back fast AI inference isn't raw compute. It's memory bandwidth, meaning how fast a chip can shovel the model's weights in and out of memory. He says his chip, ramping in the second half of 2026, delivers about 25 times more memory bandwidth per chip than the high-bandwidth memory NVIDIA uses, while sticking with cheaper, higher-capacity DRAM. If true, that changes what a long-running AI agent costs to run.

This is briefing mode, so the question isn't "should you buy a Fractile chip." You can't, yet. The question is what this conversation tells you about where inference costs are heading and who gets squeezed. Nothing here is hard to undo for the reader. There's no deadline. The value is in the map.

The Skeptic

A chip that ramps in "the second half of 2026" and does 25x anything, from a 150-person startup, with no independent benchmark in the episode. I've seen this movie. Groq and Cerebras both pitched blazing-fast inference, both hit a wall on memory capacity, and both now tell you to fall back to GPUs for anything with real context length. Goodwin admits Fractile started on that same SRAM path and pivoted. Good for him. But "we pivoted and solved the hard part" is exactly what you'd say whether or not you solved it. The 25x is a vendor number on a workload the vendor chose. Until someone runs a trillion-parameter model at thousands of tokens per second in front of a customer who didn't get a discount, this is a thesis, not a product.

The Researcher

Strip the sales pitch and the underlying diagnosis is real and useful. Compute per chip has grown about a million-fold over 20 years. Memory bandwidth, how fast weights move, only about 40x. That gap is why a model that only fires a few of its "experts" per word, what people call a sparse mixture-of-experts, runs badly on today's GPUs: the chip spends its time waiting for weights, not calculating. Goodwin's point that model shapes are partly dictated by hardware, not just by what's smartest, is the most defensible thing he says. DeepSeek, Mixtral, and Llama's expert layouts are compromises with the memory wall. He also says he thinks fully automated chip design is a few years out for prototyping. A big semiconductor CEO told him ten years; Goodwin puts it closer. He's probably right that the design step speeds up and the fab step, 3 to 5 months of physical manufacturing, does not.

The Compute Pragmatist

Here's the part that matters for your bill. Goodwin says the hyperscaler custom chips, Google's TPU, Meta's MTIA, Microsoft's Maya, OpenAI's Jalapeno, are near-identical under the hood. Same high-bandwidth memory, same matrix-multiply circuits, same TSMC packaging, often the same Broadcom as the back-end builder. His read: those chips exist mainly to beat down the price you pay NVIDIA, not to do anything NVIDIA can't. That rings true, and it's the useful takeaway. If you're renting inference from a cloud, don't expect their in-house silicon to give you a capability jump. Expect it to give them negotiating leverage, some of which may reach you as lower prices. The capability step-change, if it comes, comes from a different memory design. That's the bet Fractile and peers like Etched and d-Matrix are making.

The Builder

What would I do with this on a Tuesday? Nothing, because I can't buy it. But it changes how I plan agent workloads now. The reason long-running agents get expensive and slow is the KV cache, the growing pile of attention state that every extra token of context drags along. Groq and Cerebras can't hold a big one, so you fall back to GPUs. That's why "fast inference" marketing falls apart the moment your agent has to remember 200,000 tokens of a codebase. If a DRAM-based chip genuinely holds a big cache at high speed, long-context agents get cheaper and the "use a small model and summarize aggressively" workarounds matter less. That's a real design change. Note the word "if."

The Open-Source Advocate

The interesting second-order effect lands on open-weight models. Goodwin is saying the architectures people download today, the specific sparsity ratios, the attention tricks, are shaped by what runs well on HBM GPUs. Change the memory constraint and the open ecosystem could build much sparser models that are cheaper per answer at the same intelligence. That's a win for anyone fine-tuning Llama or running DeepSeek on their own hardware. But it cuts the other way too: if the best architectures start assuming a specific new chip, open models could fragment, with the fastest variants locked to silicon most people can't rent. Goodwin himself names that risk for the frontier labs, a competitor's 5x breakthrough that only works on one chip could kill you during the nine months it takes to switch. Same trap applies downstream.

Where they disagree

The Researcher buys the diagnosis; the Skeptic doesn't buy the product. Those can both be right. Memory bandwidth really is the binding constraint, and Fractile can still miss its ramp or deliver 5x instead of 25x. The second split: the Compute Pragmatist sees hyperscaler chips as pure price leverage, which is good for your bill today. The Open-Source Advocate sees a future where capability leaps come from exotic memory, which is bad for your portability tomorrow. The decision, for anyone deploying agents, lives in that gap between cheaper-same and faster-but-locked-in.

What it hinges on

One fact settles most of this: does any DRAM-based inference chip actually hold a large KV cache at near-SRAM bandwidth in a customer's hands, not a deck. If yes, long-context agent costs fall and model architectures loosen. If the 25x shrinks to single digits under real workloads, this is another fast-chatbot chip that still punts to GPUs on the hard part. Don't re-architect anything around it now. Do pressure your inference vendors on long-context pricing, because that's where the squeeze is and where competition is about to arrive.

Prediction: Fractile will not publish an independent, third-party benchmark showing its chip running a model above one trillion parameters at 1,000+ tokens per second on long-context workloads before NVIDIA's GTC 2027 conference in March 2027.

Confidence: Medium — H2 2026 ramp claims from 150-person chip startups routinely slip, and vendor demos precede audited numbers by many months.

Why: Goodwin pitched a chip "ramping in the second half of 2026" but offered no independent benchmark in the episode, only his own 25x bandwidth figure on a workload he chose. Chip startups in this exact niche, Groq and Cerebras, both shipped and then spent a year or more before anyone outside the company ran the hard long-context case that Goodwin says is the whole point. A silicon ramp means first samples, not volume or verified performance, and the gap between "ramping" and "a neutral party measured it on a trillion-parameter model" is historically a year-plus. The opposite outcome, a clean third-party long-context benchmark inside roughly six months of ramp, would require the fab, the packaging, the software stack, and an independent tester all to line up faster than any comparable startup has managed.

Revisit by 2027-03-31: We're right if no independent benchmark (MLPerf, an academic lab, or a named customer with no equity or discount) has shown Fractile running a 1T+ parameter model at 1,000+ tokens/second on long-context work by NVIDIA's GTC 2027. We're wrong if such a benchmark is published and verifiable by then.

Two more things for the reader. The hyperscaler-chips-are-price-leverage read is the part you can act on today: lean on your cloud vendor at renewal, their in-house silicon is your bargaining chip too. And watch long-context pricing specifically, because that's the line item a new memory design would move first.