Bot and AI agent traffic now over 50% of web requests, poisoning ad retargeting

Full analysis

Your draft

Cloudflare says more than half of all web requests now come from bots and AI agents, not people. Two things follow from that, and they point in opposite directions. Some brands love the new traffic. A direct-to-consumer pillow company says referrals from AI assistants convert several times better than its normal site visitors, and UK retailer John Lewis watched AI-driven search visits climb from 0.3% to 2.5% in a year. But retargeting desks, the teams that re-show ads to people who already visited a site, are drowning. One agency's e-commerce clients saw bot traffic jump 80% year on year. The bots fill carts, sign up for newsletters, and trip the same signals a human would, which corrupts the audience lists that power targeting. Cost-per-action started climbing in Q4 2024 and hasn't stopped.

What's actually being decided here isn't "should we block bots." It's where the budget goes when open-web signal stops being trustworthy. That's hard to undo once it moves. Budget that shifts to Meta, Amazon, and Google this planning cycle doesn't come back next year just because someone ships a better filter.

The Market Analyst. Follow the money and this is a three-way split. The walled gardens (Meta, Amazon, Google, the closed platforms that measure their own results) win by absorbing dollars that would have tested into the open exchange. No new multiple, just more volume at existing rates. Retail media networks like Walmart Connect win harder and with less fanfare, because their targeting runs on actual purchase data, which a bot can't fake. The squeezed middle is the independent open-web stack. Verification vendors DoubleVerify and IAS have a real opening: if AI-agent traffic needs a genuinely new detection layer, that's a product they can sell, not a feature they give away. But that's a window, not a birthright. For a non-specialist: when the open web's numbers stop being believable, money runs to the places that grade their own homework.

The Skeptic. Steelman the doubt, because this story fits what everyone already wanted to believe. The 50% figure is Cloudflare counting raw web requests, every page load, script, and scrape. It is not a count of ad impressions on measured, bid-on inventory. Conflating the two is sloppy, and invalid-traffic vendors have filtered a lot of this for years. The pillow-brand anecdote runs the opposite direction of the panic and gets waved past. What has to be true for this to be the crisis it's sold as: pre-bid filters failed quietly, fraud scoring lagged AI-agent behavior by multiple quarters, and agencies hadn't already baked inflated impression counts into their baseline numbers. Often they had. And budget flight to walled gardens predates any of this. It's about measurement being easier over there, not signal being purer.

The Operator. Tuesday morning, this is already a fire. Q3 renewals force the conversation nobody wants: why has CPA drifted up since last October? The first thing that breaks is the seed list. Bots filling carts and signing up poison the sample that feeds lookalike models, so teams spend real money chasing audiences that don't exist. The second break is frequency caps. Bots eat the caps, so real humans see the ad fewer times while the invoice stays flat. The tactical response is retreat: narrower URL-level inclusion lists, harder caps on open-exchange buys, more pre-bid filtering stacked on contextual signals. That's the ~20% CPM bump, the cost of buying a smaller, cleaner pool. The quiet casualty is the long-tail publisher, who loses the budget before getting a chance to prove the traffic was real.

The Customer / End User. Split the advertiser in two, because they're not asking for the same thing. The performance e-commerce buyer wants clean retargeting signal and is furious it's gone. The brand experimenting with agentic commerce, where an AI agent shops on a person's behalf, wants exactly the traffic the first buyer is blocking, because those visits convert. Here's the trap: the narrow inclusion lists and hard filters the first buyer demands will also catch the good AI referrals the second buyer wants. John Lewis's 2.5% of visits and the pillow brand's high-converting referrals are the same category of traffic the fraud filters are being tuned to reject. Nobody has clean tooling to tell a shopping agent from a scraping bot yet. So the near-term fix suppresses tomorrow's best channel.

The CFO. The real cost isn't the CPM increase. It's the audience models that stay quietly broken and don't show up on any rate card. You can see a 20% CPM bump on an invoice and defend it. You cannot easily see that your lookalike model has been training on ghosts for three quarters, which means every downstream buy off that model is mispriced and you don't know by how much. Moving budget to retail media and walled gardens buys measurement you can defend to the board, and that defensibility has real value in a renewal conversation. But it's a one-way door. The take-rate compression hits the independents now; the audience-model corruption is the bill that arrives later, unlabeled.

Where they part ways. Two real disagreements. First, is this a crisis or a convenient story? The Skeptic says budget was already leaving the open web for reasons that have nothing to do with bots, and this is a tidy excuse pinned on one variable. The Market Analyst and Strategist say the excuse doesn't matter, because the dollars move either way and the moats harden regardless. Second, and more useful: the Customer split. The same filters that fix the retargeting problem break the agentic-commerce opportunity. One buyer's poison is another's best-converting channel, and the industry has no tool to separate them.

What this hinges on. Two beliefs. One, whether AI-agent traffic is genuinely new enough to defeat existing invalid-traffic filters, or whether the vendors catch up in a quarter or two and this normalizes. Two, whether agencies can build the thing that tells a helpful shopping agent from a harmful scraper before the blunt filters throw out the good traffic with the bad. The council leans toward the budget shift being real and durable, because it's the easy, defensible move under renewal pressure, and defensible beats optimal when a client is asking why CPA is up. What to verify before committing budget wholesale: pull your own served-impression IVT rate on measured programmatic rather than the Cloudflare raw-request number, and check whether your CPA drift tracks bot volume or just tracks the open web's ordinary decline. If it's the latter, you're solving the wrong problem with a one-way budget move.

Prediction: By the Q4 2026 holiday planning cycles finalizing in October and November 2026, at least one major verification vendor (DoubleVerify or IAS) will publicly launch or heavily market a dedicated "AI agent" or "agentic traffic" detection product distinct from its existing invalid-traffic filtering.

Confidence: Medium. Clear demand signal plus an obvious product gap, but timing could slip a quarter.

Why: The story shows agencies actively raising CPMs ~20% and narrowing inclusion lists because their current filters can't separate human-like AI-agent behavior (adding to carts, signing up) from real users. That's a named, unsolved problem that DoubleVerify and IAS are the only players positioned to sell against at scale, and both have been stuck under a "mature category, no new product" story that pressures them to find a fresh cycle. When buyers are visibly paying more to route around a problem, the vendors whose whole business is detecting bad traffic will package a named solution fast, because the alternative is watching budget move to walled gardens that need no verification layer at all. The less likely outcome is silence, and that only happens if the existing filters quietly already handle agent traffic, in which case the vendors lose a marketing hook they have every reason to grab.

Revisit by 2026-12-15: We're right if DoubleVerify or IAS announces or actively markets a named AI-agent / agentic-traffic detection offering separate from standard IVT filtering by then. We're wrong if neither does and both continue treating AI-agent traffic as covered by existing invalid-traffic products.