The easiest way to keep a human in the loop is to ask them to confirm everything. It is also the surest way to lose them: ask for a person’s judgement on every answer and you get it on none. Three of the AI products I’ve shipped spend the ask in exactly one place each, and where that place sits did more for adoption than any change to the model.
Death by “are you sure?”
A confirmation step looks like respect. On paper it keeps the human in control. But attention is a finite account, and every “are you sure?” is a withdrawal from it. Past a certain rate the prompts stop being questions and become wallpaper, and the muscle-memory answer to wallpaper is skip. The system is still technically asking. Nobody is still technically answering.
That’s the trap hiding inside good intentions. We add the confirmation step to respect the user’s autonomy. But ask for a person’s judgement on everything and you get their judgement on nothing — the prompts blur together, attention drains out, and people start clicking on reflex. The thing we added to keep the human in control was the exact thing pushing them out of it.
Spend the asks where they count
So the design question is never whether to ask. It is where. In the private-equity screening product, the ask sits on one event: when the evidence behind a claim runs thin, the model says “I’m not sure about this one” and hands the call to the analyst, whose override is logged rather than lost. Everywhere the evidence is solid it shows the claim beside its source and gets out of the way. In the due-diligence dossier the one ask is the partner’s sign-off, and it stays locked until every signal’s evidence has been opened — more friction, on purpose, at the single moment where the signer carries the downside. And in the out-of-home planning platform the trader was never asked to approve a recommendation at all: the plan arrived with its KPIs attached and one click to change it, so their judgement went into shaping the answer, not rubber-stamping it. Three products, three different moments, one rule — a tap that rare is a tap the system has earned. The patterns library names the underlying split the Act / Review / Ignore rule; it is a synthesis across the AI products, not the invention of any one of them.
Ask for a person’s judgement on everything and you get their judgement on nothing. The best human-in-the-loop systems spend the ask only where a human’s input would change the answer — a tap that rare is a tap the system has earned.
What I believe now
Human-in-the-loop was never about making people verify every decision a machine makes. It’s about choosing the one moment where their attention is worth more than the friction of asking for it — and protecting that moment by refusing to spend it anywhere else.
Want the playbook, not the story? The human-in-the-loop patterns I design from — confidence thresholds, undo-don’t-confirm, edit-not-approve — with the do’s, don’ts, and worked examples.
See the pattern reference: Human-in-the-Loop Patterns →Continue exploring