Refacto AI

Podcast episode

How the 4 New AI Models Change How You Work

ai-in-adtech cost-compression engineering

Four frontier AI models dropped in a single week — from xAI, OpenAI, Cognition, and Cursor — and the real story isn't which one scored highest on a benchmark. The architecture shift underneath all the release notes is this: an expensive "orchestrator" model makes the routing decisions, while cheaper, faster "doer" models handle the actual execution. That split is what makes the economics interesting.

The seductive number is xAI's claim that Grok 4.5 matches Anthropic's Opus 4.8 on coding tasks at 31 cents versus $1.80 per task — roughly 6× cheaper at the execution tier, where token volume lives. But that figure comes from xAI's own benchmark, and your real bill includes retries when the cheap model fails the hard 15% and the expensive one has to clean up. Meanwhile, two of the week's most capable new models — SWE-1.7 and Composer 2.5 — are built on Kimi K2.7, a Chinese open-weights base. The proprietary layer everyone's charging for is the interaction data on top, not the model underneath.

Build the model router before you pick the models. The savings are real only if your abstraction layer is clean enough to swap providers without a rewrite.

Full analysis

Four frontier models dropped in one week, and the story underneath the release-note noise is an architecture shift: a expensive "thinker" model orchestrates cheaper, faster "doer" models that do the actual work. For anyone building AI into an ad-tech stack — bidding logic, creative generation, campaign agents — the question isn't "which model is smartest" anymore. It's "which model do I put in which slot, and what does the whole pipeline cost per task at production volume?"

Reversibility: Mostly Type 2. Swapping an implementation model behind an orchestrator is a config change, not a rewrite — if your abstraction layer is clean. The Type 1 risk is committing your agent architecture to a single vendor's orchestration pattern before the pattern stabilizes.

Forcing function: Real but soft. Grok 4.5 shipped in pydantic-ai v2.7.0 and GPT-5.6 is GA on Azure Databricks today — the tooling is live, so this isn't vaporware you can defer. But no contract is expiring this week.


The Skeptic — The benchmark that matters here isn't SWE-bench, it's AutomationBench — and it's xAI's own benchmark, where Grok 4.5 conveniently wins at 51.4%. Read that number the way you'd read a vendor's own case study. The "31 cents per task" figure is the seductive part, but per-task cost on a curated eval set tells you nothing about your task distribution, where a cheap model fails the hard 15% and your orchestrator burns tokens retrying. The real bill is retries plus the expensive model cleaning up. For a PM: the cheap model is cheap until it's wrong, and wrong is expensive in a bidding pipeline.

The Researcher — The genuinely new claim is Silas Alberti's: RL post-training on an already-saturated base (Kimi K2.7) produced gains "the ceiling isn't obviously hit yet." If that reproduces, it means small teams can keep squeezing capability out of open bases without a new pre-train — which is the whole reason SWE-1.7 and Composer 2.5 exist. Everything else is incremental. GPT Live's full-duplex voice (listens and speaks at once instead of waiting for silence) is an interaction upgrade, not a reasoning one — the heavy thinking is offloaded to GPT-5.5/5.6 in the background. For a PM: the voice model isn't smarter, it just talks more naturally while a bigger model does the thinking behind it.

The Open-Source Advocate — Note what the two most interesting new models are built on: Chinese open weights. SWE-1.7 fine-tunes Kimi K2.7; Composer 2.5 also rides Kimi. The proprietary layer isn't the base model — it's the interaction data Cognition and Cursor own. That's the repeatable playbook for ad-tech: your first-party auction logs, creative-performance data, and bid outcomes are the moat, not the model. The episode frames Grok 4.5 as the "Western alternative" to Kimi/GLM for data-sovereignty reasons — but that framing exists precisely because the open Chinese models are good enough to threaten. For a PM: the cheap models everyone's excited about are mostly Chinese open weights with a Western wrapper.

The Compute Pragmatist — The orchestrator/sub-agent split is really an inference-economics story. If Grok 4.5 matches Opus 4.8 on coding at 31¢ vs $1.80, that's ~6× cheaper for the execution tier — and execution is where the token volume lives. A campaign agent that runs a million task-steps a day pays for the expensive orchestrator only on the routing decisions, not every step. That's the number to model. But watch the second-order cost: full-duplex voice and always-listening agents mean continuous inference, not request-response bursts. Your bill stops being per-query and starts being per-minute-of-attention. For a PM: the pricing model is quietly shifting from "pay per question" to "pay to keep the AI awake."

The Builder — What actually ships Tuesday: your model router. The whole thesis collapses if you've hard-coded one provider's SDK. Ken's team already has the shape of this right — Claude with its own Jira identity posting to Slack, per-session cost analysis via merge request, a configurable "make it more adversarial" instruction knob. That's the orchestration substrate. Dropping Grok 4.5 or SWE-1.7 into the implementation slot behind that is a swap, not a project — pydantic-ai already supports Grok 4.5 as of v2.7.0. The trap: none of these models are drop-in equivalent on your evals. You need a task-level harness that scores each on your real workflows (bid optimization, brand-safety classification, creative copy) before you route production traffic. For a PM: build the switchboard first, pick the models second.


Where they disagree:

  1. The Skeptic vs. The Compute Pragmatist on the 6× cost win. The Pragmatist sees a real repricing opportunity in the execution tier. The Skeptic says the headline per-task cost hides retry cascades and orchestrator cleanup, and that xAI's home-field benchmark inflates the case. This is the crux: the savings are real only if the cheap model's failure rate on your task mix is low enough that retries don't eat the margin.

  2. The Open-Source Advocate vs. the "Western alternative" framing. The episode sells Grok 4.5 as the compliance-safe escape from Chinese open weights. But the Advocate points out the actual innovation (SWE-1.7, Composer 2.5) is built on those same Chinese weights — so the moat was never the base model. If your data is the moat, the sovereignty argument for paying a premium weakens considerably.

  3. The Researcher vs. everyone on durability. If Alberti's "RL ceiling isn't hit" holds, the implementation tier gets cheaper and better every few months, and any model commitment you make now is a six-month asset. If it doesn't reproduce, today's price gaps are more stable and worth building around.


What this hinges on: Not "which model wins." It's three beliefs. (a) Does a cheap implementation model's failure rate on your task distribution stay low enough that the 6× cost gap survives retries? (b) Is your data — auction logs, creative performance — the real moat, making base-model choice a swappable commodity? (c) Does the RL-on-saturated-base result reproduce, meaning the cheap tier keeps improving?

The council leans toward: build the router, treat models as commodities, own your eval harness. The direct ad-tech impact of these specific releases is low — nobody's launching a new ad product because Grok 4.5 exists. The structural impact is real: the orchestrator/sub-agent stack is now the default shape, and it maps cleanly onto agentic-bidding architectures like the ones being positioned in the QuantumPath discussions. What to verify before repricing anything: run your top three real workflows through Grok 4.5 and SWE-1.7 as implementation models behind your existing orchestrator, measure end-to-end cost including retries and orchestrator escalation — not the vendor's per-task number.


Prediction: By OpenAI's next flagship release (GPT-6, which the episode's leakers place within ~4 weeks — so by 2026-09-30), the "orchestrator + cheap sub-agent" stack will be the explicitly documented default in at least two of the major agent frameworks (LangChain, pydantic-ai, OpenAI Agents SDK, or CrewAI), with built-in model-routing/tiering as a first-class feature rather than something you hand-roll.

Confidence: Medium — The tooling is already moving this direction; pydantic-ai and Agents SDK shipped multi-model support this week.

Why: The saved-reading releases show framework maintainers adding Grok 4.5 and GPT-5.6 support within days of launch, and the entire episode describes the tiered stack as an established pattern practitioners already run. Framework authors follow real usage, and multi-model routing is the obvious next abstraction once every team is manually splitting orchestrator and implementation roles.

Revisit by 2026-09-30: We're right if at least two major agent frameworks document first-class model-tiering/routing (orchestrator vs. implementation) as a supported feature. We're wrong if routing remains something developers assemble manually with no framework-native support.

Comments