Refacto AI

Podcast episode

Computer-Use Agents and the Future of the Agentic Internet

agents coding-agents open-weights reliability tool-use

Practical AI hosts Demetrios Brinkmann and Chris Benson on where computer-use agents actually stand today: what they can do, where they break, and what happens to online shopping when software does the clicking instead of you. No lab announcements. Just a working practitioner's read on what's shipping and what's breaking.

Brinkmann is bullish on screen-driving agents (software that controls a computer visually, like a human would) for unlocking platforms that will never give you a clean connection: government portals, LinkedIn, legacy SaaS. But the more useful lesson is the Claude Code regression. Anthropic baked its coding tool's behavior into the model weights, so when the model updated, the whole thing got jittery and verbose. Benson floated agent-to-agent shopping arriving before year-end, though neither host cited real purchase volume running through agents today.

The Claude Code story is the warning worth taking seriously. Standardize on one lab's model-plus-tool bundle, and their next training run is your next outage. Keep a thin layer of your own between the model and the task, and test every version before you upgrade.

Full analysis

Your draft

A Practical AI episode with Demetrios Brinkmann talking through where computer-use agents actually stand today, why Claude Code got worse after its last update, and what happens to online shopping when your agent, not you, is the one clicking "buy." No lab announcements. Just a working practitioner's read on what's shipping and what's breaking.

This is easy to undo. Nothing here asks you to sign a contract or rip out a stack. What's actually being decided, for anyone running AI in production: do you keep pouring effort into custom tool connections, or do you let a screen-driving agent do the job the ugly way? And do you tie yourself to one lab's coding tool when that tool can degrade on the next model update? No deadline forces the call. But the Claude Code regression is a live warning, so it's worth thinking through now.

The Skeptic

Brinkmann says there are "very few things" OpenAI's computer-use agent can't do. That's the demo talking. Booking a flight and filling a German DMV form are great party tricks, and they're also low-stakes, single-user tasks where a wrong click costs him five minutes. Run that same agent against a thousand customer checkouts and the error rate stops being charming. Nobody in this episode cited a success rate, a cost per task, or a failure count. "Very few things it can't do" with no denominator is exactly the claim that dies in production. And a screen-driving agent breaks the instant a site changes its layout, which sites do constantly.

The Builder

The useful, boring insight here: computer-use agents let you skip building formal connections to services that will never give you a clean one. Government portals, LinkedIn, crusty old SaaS. If your automation target is a locked platform, a screen-driving agent gets you to value faster than waiting for that platform to expose a proper hook. But the Claude Code story is the real lesson for anyone shipping agents. Anthropic trained its model against its own tool scaffolding, and when the model changed, the whole thing got jittery and verbose. If you standardize on one lab's model-plus-tool combo, their next update is your next outage. Build your own thin layer between model and task, and test every version before you upgrade. The on-call engineer at 3 AM will thank you.

The Open-Source Advocate

The most useful technical point in the whole hour, and it's buried. Open-weight models like Hermes and the Chinese releases may behave better inside your own tooling precisely because they were never trained against a proprietary tool wrapper. Anthropic baked Claude Code's behavior into the model weights, so the model and the harness are now entangled. An open model you run yourself has no such baggage. It does what your scaffolding tells it. For anyone burned by the Claude Code regression, that's a real reason to keep an open model in the mix. Not for raw capability, but for predictability across versions. You control the update schedule. Nobody surprises you.

The Compute Pragmatist

Everyone's excited about screen-driving agents. Nobody priced them. Reading a screen, deciding, clicking, waiting for the page, reading again is slow and expensive per task compared to a direct connection that just returns data. For one person booking one flight, who cares. At scale, a screen-driving agent doing what a proper connection could do in one call is paying a heavy tax in time and tokens. This is fine for overnight batch work. It's a bad fit for anything a customer waits on. The right read: computer-use is the bridge you use when no proper connection exists, and you rip it out the moment one does.

The Skeptic (on agentic commerce)

The e-commerce disruption thesis is a thought experiment wearing a prediction's clothes. Chris Benson floated agent-to-agent shopping happening "before this year is out." There is no evidence in this episode that any real volume of purchases runs through agents today. And Brinkmann himself names the reason it stalls: nobody trusts a shopping agent run by the same company that profits from the merchants. That trust problem isn't a detail. It's the whole thing. If OpenAI plays both sides, buyers route around it, and the marketplace never forms.

Where they disagree

The real split is Builder versus Compute Pragmatist on how much computer-use matters. The Builder sees a fast path to automating locked platforms. The Pragmatist sees an expensive, slow stopgap you abandon the second a proper connection appears. Both are right, and the resolution is the same: use it where nothing else works, don't build your business on it.

The second split is Open-Source Advocate versus the Claude Code fans. Brinkmann predicts Anthropic's next model "is gonna be incredible" and fixes the regression. The Advocate says the deeper problem is that tying your workflow to one lab's model-plus-tool combo makes you hostage to their training choices, and no single fix removes that. One good next model doesn't undo the structural risk.

What this actually hinges on

Two beliefs. First, whether computer-use agents hold up on tasks that repeat thousands of times, not one-off personal errands. Nothing in this episode tests that. Before you deploy one against real customer volume, run it a few hundred times against your actual target and count the failures. If it's clicking checkout, count them very carefully. Second, whether the Claude Code problem is a one-off or a pattern. The council leans toward pattern. When a lab bakes its own tooling into the model weights, every model update risks changing behavior you depend on. The fix isn't hope. It's keeping your scaffolding model-agnostic and testing across versions.

Prediction: Anthropic's next major Claude model release will ship alongside an explicit change to how Claude Code's tool scaffolding is decoupled from the model weights, and Anthropic will publicly acknowledge the verbosity or behavior regression that followed the Opus/Fable update.

Confidence: Medium. The fix is public and the pressure is visible, but timing and framing are Anthropic's to control.

Why: Brinkmann, an experienced practitioner who called Claude Code clearly superior six months ago, has "pretty much stopped using it" after the last update, and the episode names the cause precisely: the model got trained against its own tool wrapper and the wrapper's instructions got baked into the weights, producing jittery, over-explained output. When a named power user churns and the mechanism is that specific, a lab whose paid coding tool depends on those users does not let the regression stand quietly through another release. The less likely path is Anthropic shipping the next model with the same coupling and no acknowledgment, because that would keep bleeding exactly the developers Claude Code was built to hold.

Revisit by 2027-03-16: We're right if Anthropic's next flagship model release comes with documented changes separating Claude Code's scaffolding from model behavior or an on-record acknowledgment of the post-Opus regression. We're wrong if the next model ships with no such change and no acknowledgment, or if Anthropic stays silent on the Claude Code behavior entirely.

Comments