Podcast episode
How to Go from AI Power User to AI Leader
agents evals guardrails measurement
Nathaniel Whittemore and AI adoption consultant Nufar spend an hour on why enterprises are getting almost nothing out of the AI tools they already bought. A KPMG study puts the gap plainly: roughly half of companies own AI tools, about 12% get measurable business value from them.
The behavior finding underneath that number is the durable part. A UT Austin study found that outcomes tracked not what people knew about AI, but how they worked with it: guiding the model, checking its output, catching the wrong answer before it ships. Nufar calls evaluation the scarce skill. Whittemore's warning is more concrete: non-technical employees are using agentic tools (software that takes multi-step actions on its own) to build personal workflows, and those agents are picking up system access nobody reviewed or sandboxed.
The 12% figure comes from a KPMG study KPMG is selling the fix to, so discount the prescription. The diagnosis holds. Before building any champion program, audit what your employees have already wired up.
Full analysis
Here's the real finding from a workforce consulting episode: the thing holding back enterprise AI isn't the models. It's that almost nobody is getting paid-for value out of the tools they already bought. Nathaniel Whittemore and AI adoption consultant Nufar spend an hour on how to build internal "AI Champion" programs. A figure from the KPMG sponsor segment puts the problem plainly: roughly half of companies own AI tools, and about 12% get measurable business value from them.
This is easy to undo and nothing sets a deadline. No model is launching, no price is changing. It's a diagnosis of why the stuff you already pay for isn't paying you back. The stakeholders who should care: enterprise buyers writing seat-license checks, the vendors selling those seats, and the applied teams trying to turn pilots into production.
The Skeptic
The 12% figure comes from a KPMG study KPMG is paying to promote, and KPMG sells the fix. Discount accordingly. But the gap is real, and the saved Marcus-on-ai piece backs it: a separate new study finds it "very difficult to find actual, materially measurable financial impacts" from AI spend. Two studies, same week, same conclusion. The "AI Champion" program is the consultant's answer to every problem, so of course Nufar says it's "the single highest ROI transformation activity." She would. What she can't show is a controlled comparison. Champion companies also tend to be the ones already serious about AI. The program may mark commitment without causing the return.
The Researcher
The KPMG/UT Austin finding is the one durable thing here. Across 500-plus early-career professionals, outcomes didn't track what people knew. They tracked how people worked with the model: guiding it, checking it, refining the output instead of accepting the first answer. Nufar's version: "the building part is the easy part," and the scarce skill is evaluation, knowing when an output is trustworthy enough to ship. That lines up with what everyone building with these tools already feels. The model will confidently hand you a wrong answer. The person who catches it is worth more than the person who prompts it. The claim that these habits "can be taught at scale" is the part nobody has actually proven yet.
The Builder
Forget the org chart. The live risk Whittemore names is the one I'd act on Monday: non-technical staff are using agentic coding tools, AI that takes multi-step actions on its own, to build their own little workflows. He says the agents "escape containment pretty quickly and kind of weed their way into lots of different systems." That's not a metaphor. Someone in finance wires a bot to read a shared drive, it gets a token with more access than anyone reviewed, and now it's touching systems nobody sandboxed. The champion framing is a distraction from the fix. You need permission scopes and an inventory of what employees have wired up, same as you'd gate any new service account.
The Enterprise Buyer
The 12% number reprices every AI line item on the renewal. If half your seats produce no measurable value, the question at renewal stops being "which model" and becomes "why are we paying for 400 seats that summarize meeting notes." The saved AdExchanger item shows the counter-pressure: Publicis is selling clients on the fear of falling behind, up 5% in Q3, PepsiCo won last month. Fear of missing out keeps the checks flowing even when value doesn't show up. That's the tension a buyer has to resist. Nufar's readiness test is the one procurement-grade idea in the episode: every champion takes one real use case from their team and shows the value. If they can't, there's no program, just a title.
Where they split
The Skeptic and the Researcher disagree on what the 12% proves. The Skeptic says KPMG ran research to sell a consulting practice and the champion program is unproven. The Researcher says the underlying behavior finding, that checking beats knowing, is solid regardless of who paid for it. Both can be right: the diagnosis is real, the prescription is self-serving.
The Builder and the Enterprise Buyer split on urgency. The Buyer is focused on wasted seats, a cost problem you can fix at renewal. The Builder is focused on agents escaping into systems nobody secured, a problem that gets worse the longer you treat AI adoption as an HR initiative. The Builder's risk is the one that bites first.
What it actually hinges on
Does structured training move the 12% number, or does it just mark companies that were going to succeed anyway? Nobody in this episode runs that comparison, and the two studies this week only measure the gap. They offer no cure. What you can verify without a consultant: pick five of your own AI use cases, apply Nufar's test, and see how many survive a "show me the measurable value" review. If four of five fail, your problem is evaluation discipline, not tooling. And before anyone builds another internal tool, audit what agentic coding access your non-technical staff already granted themselves. That's the exposure the models can't fix and the one the episode is right to flag.
Prediction: A follow-up KPMG enterprise AI study or its 2027 equivalent, published by the end of Q3 2027, will report the share of companies getting measurable business value from AI tools still below 25%, up from the 12% cited here.
Confidence: Medium. Two independent studies this month agree the value gap is structural, rooted in human behavior rather than tooling.
Why: Both the KPMG/UT Austin study in this episode and the separate study in the saved Marcus-on-ai piece land on the same finding in the same week: companies own the tools but can't show financial impact. That gap comes from how people work with AI and how firms govern it, which move slowly, not from model capability, which moves fast. Vendors will keep shipping better models and more seats through 2027, so ownership will climb, but the human-layer bottleneck Nufar describes, evaluation skill and manager alignment, is exactly the kind of thing that takes years to shift across a whole economy. The opposite outcome, a jump past 25%, would require a fast change in organizational behavior that neither study shows any sign of, and that consultants admit they're still writing the playbook for.
Revisit by 2027-09-30: We're right if a comparable KPMG or major-firm enterprise AI study published by then puts measurable-value adoption under 25%. We're wrong if it reports 25% or higher.
Comments