Why Better Brand Lift Data Makes Advertising Smarter with Alistair Hill of OnDevice

Full analysis

Brand lift measurement is dirty, and the guy telling you so sells brand lift measurement. Alistair Hill, CEO and co-founder of OnDevice, went on AdTechGod's podcast and made a set of claims that are almost certainly true and almost entirely self-serving at the same time. The interesting part is not the vendor pitch. It's what the pitch reveals about how loose the whole category still is.

What's being decided: Nothing, for the market. This is a vendor showcase, not news. For the operator, the question is narrower and real: do you tighten how you vet brand lift vendors, and do you start demanding log-level data (raw, impression-by-impression records) in your measurement RFPs? That's an easy call to reverse. You can add a line to an RFP tomorrow and pull it next quarter. Low stakes, so move fast.

No deadline. Nothing here has a clock on it.

The Skeptic

Every claim Hill makes happens to indict everyone but OnDevice. The 0.02% figure comes from OnDevice's own norms database, which nobody outside OnDevice can check. The 25-point-lift horror story is unnamed and unverifiable. "The survey industry is absolutely exposed to fraud" is true, and also exactly what you'd say if you were selling the fix. None of this is dishonest. It's just that a man paid to appear on a podcast, selling a problem, will describe the problem as severe and widespread. The directional claim holds up: double-digit brand lift is genuinely almost never real. But treat the specific numbers as marketing until an outside party confirms them. For a general reader: the plumber telling you your pipes are rotten also sells pipe.

The Operator

The useful part is the mechanics, and Hill actually gives them. The average campaign moves consideration about one percentage point. In a 200-person survey sample, five bad responses can swamp that entirely. That's the real due-diligence question to put to any measurement partner: how do you match the exposed and control groups on brand relationship and purchase history? If both groups aren't balanced on, say, primary Walmart shoppers, the lift is noise. Ask that, and ask for log-level data. A vendor who won't hand over the raw rows is telling you something. This costs you nothing to adopt and it separates the serious vendors from the ones selling you the highest number they can print.

The Customer / End User

The advertiser is the customer here, and the advertiser has been the problem. Hill says buyers pick the cheapest vendor that reports the highest lift. That's the whole rotten incentive in one sentence. The measurement company that tells you the truth (one point of lift) loses the RFP to the one that tells you 25. Until brand marketers stop rewarding the flattering number, no amount of facial-recognition survey validation fixes the category, because the demand is for a good story, not a true one. The vendors are responding rationally to what buyers pay for.

The Market Analyst

For the market, this episode is close to zero. No M&A, no earnings signal, no regulatory move, no numbers a public measurement name has to answer for. But the structural drift is real and worth naming across the ecosystem. Log-level data as a standard RFP ask puts pressure on every black-box measurement vendor, not just survey shops. Comscore, Nielsen, DoubleVerify, IAS, VideoAmp, iSpot all get asked the same question eventually: show me the rows. The vendors built on summary outputs and proprietary panels have the most to lose from a transparency norm. The ones already handing over raw data have a wedge. In plain terms: the industry is slowly agreeing that "trust me" is no longer an acceptable answer from a measurement company.

The CFO

Follow the money on the AI claim, because that's the only forward-looking idea here. Hill wants brand lift data to stop being a post-campaign report and start being the input to the next media buy, linking outcomes to impression-level variables like CTV genre, daypart, frequency, and creative. Sounds great. It also requires log-level data at volume, clean group matching, and enough campaign history to train a model that generalizes. Most advertisers don't run enough comparable campaigns to build that. The value is real for the biggest spenders and thin for everyone else. The one-point average lift also caps the upside: if the whole effect you're optimizing toward is a single point of consideration, the model has very little signal to chase.

Where the council splits

Two real disagreements. First, the Skeptic and the Operator part ways on the numbers. The Skeptic says treat 0.02% and one-point-average as marketing. The Operator says the mechanism behind them (small effects, tiny samples, unmatched groups) is sound and usable even if the exact figures aren't checkable. Both are right: use the method, ignore the stat.

Second, the CFO and the Market Analyst disagree on the AI loop. The Analyst sees a transparency shift that favors data-open vendors. The CFO sees a capability that only pays for the handful of advertisers with enough campaign volume to train on. The transparency norm is broad. The AI payoff is narrow.

What it hinges on

Whether brand marketers change what they reward. The demand for the flattering number is the disease; every fraud-detection feature Hill describes is treatment for a symptom. If buyers keep paying for the highest reported lift, honest vendors keep losing, and log-level transparency becomes a checkbox nobody actually reads. The council leans toward: adopt the log-level ask and the group-matching question now, because they cost nothing, and stay cold on the AI planning loop unless you're a top-tier spender.

No high-conviction prediction this week.

This is a vendor showcase with no dated, checkable market consequence. The transparency drift is real but slow and has no anchor to grade against. Naming a date would be manufacturing conviction the episode doesn't support.