For expert skeptics, provenance is the product.

The stakes: an accurate score that added work instead of removing it.

Doubt is the job here, and these analysts are very good at it. A score they couldn’t cross-examine wasn’t a tool — it was a liability with a UI, and they treated it like one. It didn’t help that the number came out of a large language model reading messy deal documents: the same technology that, pushed too far, will invent a confident answer it can’t back up.

The LLM read board packs, statements and interview notes like a junior analyst, and scored deal risk accurately. But on screen, a ninety-percent-confident score looked identical to a fifty-percent guess, and nothing said whether a claim came from a real document or was quietly made up. So analysts built parallel scorecards by hand and demoted the AI to sanity check.

The accuracy is the ML team’s result to defend. Mine is everything that made an accurate score acted on: the evidence beside the number, the abstention state, the override logged instead of lost. The job was to make a correct score survive cross-examination by the most skeptical reader in the building — and to hold the launch until it could.

The test: if the principle holds, belief should move without touching accuracy.

An analyst won’t stake a recommendation on a number they can’t defend to the investment committee — so an unexplainable score, however accurate, becomes one more thing to audit by hand. The model added work instead of removing it.

The mechanism: references beside the number, and an honest “I’m not sure.”

The product served one analysis in two interpretation modes — because analysts read differently. Some want the full written argument to sit with and annotate; others want to interrogate. So the system generated a descriptive mode — a detailed written analysis of the deal, the scores explained in prose — and a conversational mode for questioning that same analysis. Neither was a free-form chatbot: underneath both runs an agentic architecture with strict operational boundaries — specialised components playing as an orchestra, not a single improvising soloist — so whichever mode an analyst chose, the answers stayed structured, auditable, evidence-based.

Above the single deal sat the scoring dashboard. The whole pipeline scored side-by-side — the multi-deal view where screening actually starts — and any deal drills down into the per-signal breakdown, the written analysis, and the conversation. One analysis, three surfaces, no forced workflow.

So I built an “explain this score” surface that grounds the model in its evidence. An analyst could pull any rating apart into the signals behind it, challenge the weighting, and watch the score answer back — and every signal came stapled to the exact document it was retrieved from, so no claim floated free of a source. Making the model survive that, before release rather than after, is the argument the whole project hung on — and the reason it took the weeks it did.

Three things were non-negotiable: a cited source behind every generated number; a visible “I’m not sure about this one” where the model abstains instead of bluffing — an LLM’s worst failure is a fluent wrong claim; and a logged override when the analyst disagreed. The override drew the longest fight: logging disagreement felt exposing. It was actually the moat. Every correction became structured data the system learns from, so the platform comes to reflect how this firm thinks. A rented model any competitor can rent too; a record of the firm’s own judgment, nobody can.

The abstention threshold was a design argument, not just a statistical one. Every claim carried its own confidence; run low, and it rendered as “I’m not sure about this one” — a signal promoted from the eval dashboard to a state the analyst can act on. The pitch is the whole difficulty: too eager and analysts learn to skim past the crying wolf; too shy and one fluent wrong claim burns the trust bank. The threshold fails toward humility — the cheaper mistake.

Trust also had to be cheap to enter. Deal flow lives in email — decks, spreadsheets, forwarded intros — and no analyst re-keys email into a system. So the platform meets them inside Outlook: tag a message, drag an attachment, get “Received, thanks” while agents stitch the pile into a deal profile — and the prequalification score surfaces back in the inbox, where prioritisation actually happens. The fuzziest judgment got the same discipline: management quality — pure gut feel, traditionally — became a scorecard of leadership tenure, employee sentiment and filed financials on one timeline, every input traceable.

Pull a score apart and challenge it below — the signals, their 3 cited sources, their weights, and the score answering back:

What holding the principle cost — and the objection it had to survive.

Building it first wasn’t free. The accurate LLM sat finished while I built the surface that could defend it — the retrieval that tied each claim to a source, the abstention state, the override — weeks the team could feel, against a model that already worked. Waiting was the product owner’s call as much as mine; what I own is the argument for it, because shipping a correct score nobody acted on would have been the more expensive mistake.

Accuracy gets you a correct number. It doesn’t get you a decision. The gap between the two is the whole job — and it’s made of trust, not math.

Falsifiable evidence — 60% faster, and the skeptics now open with the score.

Interrogation turned into reliance · every number with its window

Screening ran 60% faster, and analysts went from ignoring the model to leading their deal memos with it. The LLM didn’t get more accurate — they could finally trace every claim to its cited source and see where its confidence came from. What the NDA keeps off this page — the sample, the eval design, the baseline, who ran the count — I walk through on a call, artifacts open.

Where the principle breaks.

Provenance-first assumes the reader will actually open the references — true of analysts whose names go on the memo, untrue of users grazing a feed. It also taxes latency and retrieval cost on every claim; in a product where a wrong answer is cheap, that tax buys nothing. And citations cannot save a model that’s guessing: a source beside a wrong number just makes the error easier to find — which is the point, and which some rooms don’t want. If your users don’t doubt for a living, this case argues against copying it.