Episode 183: Ask Olivia Kory Whether AppLovin Ads Work & She Explains Attribution vs Incrementality

Full analysis

Olivia Kory, Haus's Chief Marketing Strategy Officer, went on Marketecture and told Ari Paparo and Eric Franchi something most measurement vendors won't say out loud: AppLovin's e-commerce ads actually work. The bigger tell in the conversation is which channels she says don't stand up to a holdout test, and why that gap is quietly capping how much money advertisers will pour into them.

Reversibility: Type 2 for any single operator. Measurement is a dial you can turn. But the industry-wide shift toward causal testing is Type 1, and it's already moving.

What's actually being decided: Whether the channels that refuse rigorous incrementality testing (affiliate, retail media networks, Amazon search) can keep growing on faith, or whether the buy side finally makes proof of causation a condition of the budget.

Timeline: No hard forcing function. This plays out over the next few budget cycles as CFOs get louder about ROI.

The Market Analyst. AppLovin gets validated here, and it matters that the validation is independent. Kory confirmed AppLovin opened its e-commerce pilot to third-party incrementality testing from day one, late 2024. That's a company that wants to be measured because it knows it passes. Compare that to the channels Kory grouped as measurement-shy: affiliate, RMNs, Amazon search. For an informed outsider: the ad channels willing to be graded are pulling ahead of the ones that hide the test. AppLovin's edge is speed. Kory said its effects are short-term and immediate, unlike a CTV test she won't even run under eight weeks. Fast, provable ROI is exactly what wins budget in a tight year.

The Skeptic. Kory sells incrementality testing. Her thesis that affiliate, RMNs, and Amazon search are "limiting their advertising TAM" by not offering holdouts is, conveniently, an argument for buying more Haus. The fact-check flags it: no data behind "advertisers are holding back dollars." And her affiliate finding of little incremental lift came from a handful of tests she herself called a small sample. For a non-specialist: the person telling you these channels are unproven makes her living proving things. That doesn't make her wrong. It means her central assumption, that skepticism is suppressing spend rather than the channels just being cheap and easy, is untested.

The Operator. The practical takeaway is CAPI, and it's not what most think. Conversion APIs, the server-side pipes that send purchase data back to ad platforms, are winning over browser pixels. But Kory frames them as optimization fuel, not measurement. Feed the platform your conversions and its machine-learning model gets better at finding buyers. That's the trap. If you send your purchase signal into AppLovin or Meta to improve their targeting, then use their reported conversions to judge them, you're grading the platform with data you handed it. The clever move Kory mentioned: some advertisers send only predicted-incremental conversions through CAPI. Optimize on causation, not correlation. Most teams aren't set up to do that Tuesday morning.

The Customer / End User. The advertiser here is the growth marketer who needs to show a number this quarter. That's why AppLovin and Meta win: fast, visible ROI. It's also why RMNs get a pass. Retail media is easy to buy, sits next to the transaction, and reports a tidy ROAS. Nobody wants to run a geo-holdout that might reveal half of it was going to happen anyway. Kory's honest point cuts both ways. Advertisers aren't demanding incrementality from RMNs because the current numbers look good enough and testing is a hassle. The skepticism she describes is real but passive.

The CFO. Attribution measures who bought after seeing the ad. Incrementality measures who bought because of it. The gap between those two is where ad budgets go to die. Kory's affiliate finding, little incremental lift, is the nightmare case: paying a commission on sales that would have happened anyway. If that holds at scale, affiliate and a chunk of RMN spend is a rebate on existing demand dressed as new growth. The reason nobody acts on this is that the proof is expensive and the channels resist the test. A CFO who forces one clean holdout on the biggest "unmeasured" line item will learn more than a year of ROAS dashboards.

The tensions.

Skeptic versus Market Analyst: is the measurement gap actually suppressing spend, or is that a vendor's sales pitch? Kory asserts advertisers hold back dollars from unmeasured channels. But RMN spend keeps climbing anyway. If skepticism really capped budgets, retail media wouldn't be the fastest-growing line in the deck.

Operator versus Customer: CAPI is sold as measurement progress, but it mostly makes the platforms better at optimizing themselves. The advertiser thinks they're getting cleaner data. They're mostly feeding the machine that grades its own homework.

What this hinges on. One belief: that advertisers will eventually demand causal proof from the channels that currently avoid it. The council leans skeptical that this happens fast. AppLovin proves the upside of volunteering for the test, but RMNs and affiliate have every incentive to keep the lights dim, and buyers have shown they'll spend anyway. Before acting: run one geo-holdout on your largest unmeasured channel. Not a vendor's model. Your own test. If the lift is there, keep spending. If it's not, you just found your budget for next year.

Prediction: No major retail media network (Walmart Connect, Amazon Ads, Target Roundel, Kroger) will open its inventory to standardized third-party geo-holdout incrementality testing on AppLovin's terms before the 2026 holiday budget commitments lock in Q3 2026.

Confidence: Medium. RMNs have every incentive to avoid a test they might fail.

Why: AppLovin volunteered for third-party testing because its fast, short-term effects pass, and Kory confirmed that made validation easy. Retail media networks are in the opposite position: their reported ROAS sits right next to the transaction, so a clean holdout risks showing a big share of those sales would have happened anyway, exactly what Kory found with affiliate. A channel growing fast on flattering attribution numbers has no reason to invite a test that could shrink its budget, and buyers keep spending without demanding one. The opposite outcome, an RMN opening its books to standardized causal testing, only happens under pressure that doesn't yet exist.

Revisit by 2026-10-15: We're right if no top-four RMN has publicly committed to standardized third-party geo-holdout testing by then. We're wrong if any of Walmart Connect, Amazon Ads, Target Roundel, or Kroger announces one.