The recommender was 94% accurate. The students skipped it 60% of the time. Not because it was wrong — because we'd made them click "confirm" so often they'd stopped reading. We'd asked for their judgment so relentlessly that we'd trained them to switch it off.
Death by "are you sure?"
In 2019 I was looking at a learning platform that, on paper, was doing everything right. It recommended the next course. It asked the student to confirm. It kept the human in the loop. And it was quietly failing — the engine was 94% accurate, yet six out of ten recommendations got skipped. When I watched a few students go through it, the reason was almost embarrassing: they weren't disagreeing with the recommendations. They were exhausted by them. "Do you want to take this course?" appeared so often it had become wallpaper, and the muscle-memory answer to wallpaper is skip.
That's the trap hiding inside good intentions. We add the confirmation step to respect the user's autonomy. But ask for a person's judgment on everything and you get their judgment on nothing — the prompts blur together, attention drains out, and people start clicking on reflex. The thing we added to keep the human in control was the exact thing pushing them out of it.
Spend the asks where they count
The shift that fixed it was to stop treating every recommendation as equally worth interrupting someone for. A confirmation is a withdrawal from a finite account of attention. So I started spending those asks only where the model was genuinely unsure and a human's input would actually change the answer. Where the model was confident, it just acted, and the student could undo it if they cared to. Where the model was lost, it said so plainly and got out of the way. The asks that remained landed, because they'd become rare enough to mean something.
The best human-in-the-loop systems are nearly invisible. The user doesn't feel a machine asking permission at every turn. They feel it quietly handling what it's sure of — and tapping them on the shoulder, rarely, exactly when their judgment is the thing that matters.
What I believe now
Human-in-the-loop was never about making people verify every decision a machine makes. It's about choosing the one moment where their attention is worth more than the friction of asking for it — and protecting that moment by refusing to spend it anywhere else. Get the timing right and people override the system confidently, on the calls that deserve it, instead of reluctantly skipping all of them.
Want the playbook, not the story? The human-in-the-loop patterns I design from — confidence thresholds, undo-don't-confirm, edit-not-approve — with the do's, don'ts, and worked examples.
See the pattern reference: Human-in-the-Loop Patterns →Continue exploring