Programmatic Advertising Platform

Everyone assumed the model was the problem. The model was the one thing working.

In programmatic advertising, an engine decides where the budget goes — which channels, which audiences, which hour. This one outperformed the buyers, and visibly so. The scoreboard wasn't subtle about it. Adoption sat near zero anyway.

I got this wrong first. My opening read matched the room's: retrain the model, make it smarter, and surely the traders would come around. I spent two weeks chasing that before I admitted it was the wrong suspect. The model didn't need to be more right. It needed to stop handing down verdicts without arguments.

The traders weren't being stubborn. They were being rational — the model never gave them anything to bet on. A bare number asks for faith. Traders deal in collateral, not faith, so they ignored it.

A campaign took two weeks, and a winning model nobody used was worth nothing.

Planning a campaign the old way meant about two weeks of media-buyers reconciling spreadsheets against their gut. The whole point of the engine was to collapse that — and it could have, except the people whose job it was to act on it didn't.

That's the quiet failure mode of a good algorithm: it isn't wrong, it's unused. The business had paid for an edge that sat on a screen no one bet on. Chasing more accuracy would have spent the budget on the half that already worked, while the half that was broken — trust — stayed broken.

So I stopped touching the model and rebuilt the screen between it and the trader.

I built three things. A confidence score that resolved to one of three verbs — act, review, or ignore — and never reached the screen as a naked 87%. A reasoning panel that named the exact signals behind each call. And an override that logged the correction and fed it into next week's model.

The override was the one decision. The safe build would have stopped at a prettier score with a tooltip — explainable, demoable, and still ignored. Giving traders a button to overrule the algorithm felt like undermining it. It was the opposite. The first time a buyer watched their own pushback sharpen the following week's calls, the relationship flipped from fighting the model to coaching it. Trust wasn't something I could design into the score; it was something they earned by being allowed to argue back.

The whole case fits in one card. Switch the confidence and watch the verb change — then open the reasoning, then override it:

The receipt.

Same model, before and after the trust layer

What it took to admit: the smartest thing on the screen wasn't the model.

The hard part wasn't the design. It was walking back my own first call — that a model this good should sell itself — after I'd already argued for it. It didn't sell itself. Campaign planning dropped from about two weeks to three hours, and not one of those hours came from a better algorithm.

The model never changed. The humans finally bet on it. A correct answer no one acts on is worth exactly nothing.

The cost of the right diagnosis was admitting the wrong one was more flattering. "Make the model smarter" is the answer that protects the data-science team and the budget. "They don't trust it" points the finger at the interface — which was the actual ceiling.

The principle it left me with.

When a model is right and nobody acts on it, the problem is almost never accuracy. It's that the model asks for faith and gives no collateral. The trust layer — a verb instead of a number, the reasons named, and a way to argue back that the model actually listens to — is what converts a correct answer into an acted-on one.

It reset how I open every AI project since. Before anyone asks to retrain, I ask whether the humans were ever given anything to bet on. Most of the time the model was already right. That was the problem.

Want to know more about this work?

Details are confidential; I'll walk through the artifacts and numbers on a call under mutual NDA.

Design Patterns Used in This Case

This project is where the Act / Review / Ignore rule was forged — and it directly informed three core patterns now used across multiple AI products:

• The Act / Review / Ignore Rule: every confidence score bound to one action — act, review, or ignore — with a reason for every review and an override for every act.
• Confidence Score Patterns: All confidence visualization techniques used here — numeric, color, language, gauges — became the foundation of the Confidence Score Patterns library.
• Human-in-Loop Patterns: The override and feedback mechanisms became core examples of keeping humans in control when AI makes recommendations.
• ML Explainability Patterns: The reasoning surfaces and feature importance visualizations that buyers used to understand algorithm decisions directly informed the explainability patterns library.