ML Explainability Patterns

A recommendation you can trace back to evidence is one you'll stand behind. One you can't is one you'll override — and then you rebuild your own process beside it. These patterns show a person why the machine decided. Not the full math, not the model internals. Just enough of the reasoning to act on it without a stats degree.

1. Feature Importance (What Mattered Most)

Name the top 3–5 features that drove the call, ranked. "This lead scored high because: (1) 85% match on job title [importance: 45%], (2) Company size in target range [importance: 30%], (3) Recent activity on site [importance: 15%]." The person reads three reasons, not a black box.

Strengths: Cheap to build. The user can check the work. Repeated exposure trains their intuition.

Weaknesses: It hides feature interactions. When features are correlated, the ranking lies.

2. Counterfactual Reasoning ("What If")

Show the one input that flips the call. "The system rated this lead high-value. Change [one thing] and it drops to medium-value." Or: "You're 3 steps away from the next tier. Here's what closes the gap."

A counterfactual lands because the person reads it as a causal story, not a verdict. The decision stops feeling fixed. They know exactly what lever to pull.

Choose it when the user can act: sales (close a deal), education (lift a score), recommendations (steer toward better picks).

3. Comparative Explanation ("How It Compares")

Set the pick against its rivals. "I recommended Option A. It beats Option B on [dimension 1] and loses to it on [dimension 2]."

A person reads a comparison faster than an absolute score. The trade-off is right there: Option A is riskier with more upside, Option B is the safe call.

4. Reasoning Path ("Step by Step")

Walk the chain in order. "To predict conversion, I checked industry (SaaS), then company size (50–200), then engagement (high), then landed on 73%."

It reads the way a human analyst would talk through the case. The person follows the branch, step by step.

Works for: tree-based models, decision trees, rule-based systems — anything with a real path to trace. Breaks for: neural networks. There's no path to show, and inventing one is hand-waving with a UI.

5. Confidence + Uncertainty Bands

Ship the prediction with its range. "Predicted revenue: $45K (±$15K). Past errors put the real number between $30K–$60K."

A band is an honest surface. It says: the model's best guess is $45K, it isn't 100% sure, and here's the spread it usually misses by. This one doesn't explain why — it tells the person how hard to lean.

Implementation Checklist

• Pick the explanation type from the domain — feature importance, counterfactual, or comparative.
• Compute it: tree models give you importance for free; neural networks need LIME/SHAP; rule-based systems trace the rules.
• Say it in plain words: "Feature X has importance 0.34" becomes "Job title was the strongest signal — 34% of the decision."
• Watch a real user read it: can they say why the model decided? Do they lean on it more after?
• Don't fake it: if you can't follow the model's logic, don't write a story that pretends you can.
• Progressive disclosure: top 3 features by default, the rest one click away.

Trade-offs: Accuracy vs. Understandability

*If tree-based.

See This Pattern In Action

• AI-Assisted Due Diligence: the surface that let analysts pull a score apart and see why the model ranked a deal.
• Programmatic Advertising Platform: naming the exact signals behind every media-buyer recommendation.

One pattern a month. The tradeoffs I paid for, plus code.