Refacto AI

Podcast episode

How to Get the Most Out of Fable 5 and GPT-5.6 Sol

cost-compression inference model-pricing orchestration

OpenAI's GPT-5.6 "Sol" and Anthropic's Claude "Fable 5" are both out. This episode, hosted by Nathaniel Whittemore alongside Eric Provinchar, Christine Zhu, Tariq, and Matt Schumer, is entirely about how to use them, not how they benchmark. Worth your time if you're paying frontier prices.

The most interesting claims: Tariq says Sol is the "first model where quality is bottlenecked by my ability to clarify unknowns," meaning the model is now smarter than your spec. Zhu argues Claude handles hard strategy work better and faster than you do. And OpenAI is coaching customers to spend less: audit legacy prompts for bloat, match model size to task, default to lower "effort" settings. That tells you something about margins. The practical pattern underneath all of it: replace single-shot prompting with evaluator-checked loops, where a second model grades the output until it meets your criteria.

The vendor claims are real but vendor-sourced. Zhu's "Claude owns strategy" call dies when there's no clean way to check it. Run the prompt audit; hold off on unsupervised loops until you've built the guardrails.

Full analysis

Your draft

Two frontier labs shipped new top-tier models. OpenAI's GPT-5.6 "Sol" and Anthropic's Claude "Fable 5" are both out, and this episode is entirely about how to use them, not what they benchmark at. Underneath the how-to, the labs are telling you your old prompts are now a liability, that inference cost is your problem to manage, and that single-shot prompting is being replaced by evaluator-checked loops. That's a workflow shift, and it's a Type 2 (easily reversible) decision for most teams. You can audit prompts and pilot loop patterns on one workflow without betting the company. The forcing function is soft: no deprecation deadline, just the standard "your context library rots a little with every model generation" tax.

The Skeptic. The most quotable claims here are the least verifiable. "Removing repeated instructions raised scores 10-15% and cut tokens up to 66%" comes from OpenAI's internal testing. No benchmark named, no task distribution, no baseline. "Up to 66%" is a ceiling, not a median, and vendors quote ceilings. Christine Zhu's "Claude can do the hard work better and faster than you" on strategy and product judgment is exactly the claim that dies in production: high-stakes narrative work has no clean eval, which is precisely why people keep humans on it. For the PM: the labs are telling you to trust the model with harder work right as they stop giving you a way to check it. The tenacity warning cuts the other way too. A model that takes autonomous action unless you say "draft, don't send" is a regression sold as a feature.

The Researcher. Strip the marketing and there's one genuinely interesting technical claim: the bottleneck moved. Tariq's "first model where quality is bottlenecked by my ability to clarify unknowns" is a real, falsifiable statement about capability. It says model competence now exceeds human specification quality on agentic coding. If true, that reframes evals: you stop measuring the model in isolation and start measuring the human-plus-model loop. The four-quadrant "unknown unknowns" framing and the four loop types (turn/goal/time/proactive) are useful taxonomy, not new science. Evaluator-model-checks-generator-model is the same LLM-as-judge pattern the literature has documented for two years. For the PM: the new development here is the model grading its own work against your success criteria until it passes. That is what the loop patterns actually do. Watch that the judge model doesn't share the generator's blind spots. Correlated errors make the loop confidently wrong.

The Open-Source Advocate. Everything described here is a prompting-and-orchestration discipline, not a proprietary moat. That's the good news for anyone not paying frontier prices. Prompt audits, tier-matching, effort dials, evaluator loops: all of it ports to Llama, Qwen, Mistral, DeepSeek. The "match model size to task" advice (Sol to Tariff to Luna) is just a routing pattern you can build over open weights, sending the cheap 80% of traffic to a 7B-70B local model and reserving the frontier API for the hard 20%. The loop taxonomy is already implementable with open frameworks. For the PM: none of this requires OpenAI or Anthropic specifically. The technique is free even if their models aren't. The one thing you can't cheaply replicate is Sol/Fable-tier reasoning on genuine unknown-unknowns work, but Zhu's own framework says that's a minority of tasks.

The Compute Pragmatist. The buried headline is economic: OpenAI is now telling customers to spend less. "Start one effort level lower," "dial effort down by default," "audit legacy prompts for token bloat." Labs don't coach you to cut consumption unless margins or capacity are tight. The 66% token reduction claim, even halved in reality, is a direct opex lever: at a million queries a day, trimming legacy instruction bloat is real money before you touch model choice. Tier-matching is the same lever. Running Sol on everyday tasks is like renting a B200 to serve a chatbot FAQ. For the PM: the vendor's own advice is that your inference bill is probably 30-50% waste from prompts written for last year's model. The loop patterns cut the other way, though. Evaluator-checked re-queuing can multiply token spend per task, so goal-based loops need hard iteration caps or they burn budget silently.

The Builder. What ships Tuesday: a prompt-audit pass across your library and a routing layer for tier-matching. Both concrete, both low-risk, both probably net-positive on cost. What doesn't ship Tuesday: production loop orchestration. "The model never self-declares done; an evaluator re-queues work" is a beautiful pattern and an on-call nightmare. Where's the max-iteration cap? The cost ceiling per session? The circuit breaker when generator and judge disagree forever? The Sol tenacity issue is the real near-term hazard. Any workflow with tool access needs "draft, don't send" guardrails audited now, because a more autonomous model plus a stale system prompt is how you get an agent emailing a customer at 3 AM. For the PM: the cheap win is cleaning up prompts; the expensive trap is shipping self-looping agents without a kill switch.

Where they disagree

  • Did the bottleneck actually move, or did the labs just reprice the same capability? The Researcher takes Tariq's "human is now the constraint" claim as a real capability step. The Skeptic reads it as unfalsifiable flattery. Of course the vendor says the model's ceiling is your prompting. This is the whole episode's hinge: if the human really is the binding constraint on impact work, the entire "loops and clarifying unknowns" discipline is worth building; if not, it's ceremony around the same GPT-4-class judgment gap.
  • Do the loop patterns save money or burn it? The Compute Pragmatist sees token savings from tier-matching and prompt hygiene. The Builder sees evaluator loops silently multiplying spend per task. Both are right. The savings are on the input side, the risk is on the iteration side, and a team that adopts loops without iteration caps could easily erase its prompt-audit savings.
  • Is any of this frontier-specific? The Open-Source Advocate says the technique is free and portable; the labs' own advice inadvertently commoditizes their moat down to "reasoning on genuine unknown-unknowns," a minority of tasks. The Skeptic would add: which is exactly the slice you can't evaluate, so you can't prove the frontier premium is worth it there either.

What it hinges on

Two beliefs. First: is the 10-15% quality lift / 66% token cut from prompt audits real on your task mix, or an OpenAI ceiling? That's directly testable. Run your existing prompts and a stripped version through your own eval harness this week; the answer costs a day. Second: can evaluator-checked loops stay inside a cost and iteration budget in production? Test that with a hard cap and a spend ceiling on one low-stakes goal-based workflow before it touches anything revenue-facing. The council leans clearly toward doing the cheap, reversible work now (prompt audit, tier routing, tenacity guardrails) and staying skeptical of the high-stakes-impact and self-looping claims until your own numbers back them.

The call

The episode's boldest claim is the one worth betting against in its strongest form: that Fable 5 does high-stakes "impact work" (strategy, product judgment, narrative) "better and faster than you." That's the claim with no eval behind it.

Prediction: By Anthropic's next Claude model release (the generation after Fable 5), the dominant production use of Claude will still be "optics and execution" tasks: drafting, summarizing, coding assists. Not the autonomous "impact work" (strategy, high-stakes judgment) this episode says Fable 5 unlocks. No independent benchmark or credible enterprise case study will show Claude reliably owning unsupervised strategic decisions.

Confidence: Medium. The impact-work claim has no eval and no verification path.

Why: The single claim doing all the work here, that the model rather than the human is now the constraint on high-leverage work, comes entirely from lab-affiliated voices (an Intuit UX PM and Anthropic's own Claude Code team) with no external benchmark, because high-stakes judgment tasks have no clean success metric to grade against. The pattern across every prior frontier release is that adoption concentrates in exactly the "execution" tasks this episode dismisses as underuse, precisely because those tasks are checkable and reversible, while unsupervised strategy work stays human-owned for liability and trust reasons that a smarter model doesn't erase. For the opposite to happen, enterprises would have to hand off decisions they can't audit to a model whose vendor simultaneously warns it takes unintended autonomous actions unless explicitly fenced. That's an unlikely combination in the same product cycle.

Revisit by 2026-12-31: We're right if usage data, case studies, or surveys still show Claude concentrated in drafting/coding/summarizing with humans retaining strategic sign-off. We're wrong if a credible independent source documents enterprises running Fable 5 (or its successor) on unsupervised high-stakes strategic or product-judgment decisions at scale.

The safe, non-obvious money is on the workflow advice, not the capability hype. The prompt-audit and tier-routing guidance is real, portable, and worth acting on now. The "the model can now do your hard thinking" framing is

Comments