AI-Assisted Private Equity Investing

Accuracy was never the bottleneck. Trust was.

PE analysts are paid to be suspicious of confident numbers. Doubt is the job, and they're good at it. A score they couldn't cross-examine wasn't a tool — it was a liability with a UI, and they treated it like one. It didn't help that the number came out of a large language model reading messy deal documents: the same technology that, pushed too far, will invent a confident answer it can't back up.

The LLM scored deal risk accurately — reading board packs, statements and interview notes the way a junior analyst would. That was never the bottleneck. The bottleneck was that every score read the same on screen whether the model was ninety percent confident or fifty percent guessing, and nothing on screen told you whether a claim was pulled from a real document or quietly made up. With no way to check the reasoning, analysts built parallel scorecards and let the AI rank as a sanity check at best.

So the work was never to make the model more accurate. It was to make a correct score survive cross-examination by the most skeptical reader in the building — to tie every generated claim back to the document it came from — and to hold the launch until it could.

A score nobody trusts is a screen nobody opens.

Deal screening is where capital decisions begin, and the cost of a missed risk is measured in funds, not clicks. An analyst who can't defend a number to the investment committee won't stake a recommendation on it — so an unexplainable score, however accurate, just becomes one more thing to audit by hand. The model added work instead of removing it.

Shipping it that way would have looked impressive in a demo and moved nothing in practice: a confident verdict with nothing to cross-examine, quietly set aside by the people it was built for. What the analyst actually needed wasn't the score. It was the case the score could make for itself.

I held the launch until the score could explain itself.

So I built an "explain this score" surface that grounds the model in its evidence. An analyst could pull any rating apart into the signals behind it, challenge the weighting, and watch the score answer back — and every signal came stapled to the exact document it was retrieved from, so no claim floated free of a source. I held the release until the model could survive that — the call the whole project hung on, and the one that delayed it.

Three things were non-negotiable: a cited source behind each generated number, a visible "I'm not sure about this one" state where the model abstains instead of bluffing a confident answer it can't ground, and a logged override when the analyst disagreed. The abstention state mattered most — an LLM's worst failure is a fluent, wrong claim, and I'd rather it say nothing than hallucinate. The override drew the longest argument — logging disagreement felt exposing. What it actually built was the feedback that sharpened the system over time.

The design followed the evaluation layer directly: where the model's own confidence in a claim ran low, that reading became the on-screen "I'm not sure about this one" — a hidden eval score turned into a state the analyst could see and act on.

Pull a score apart and challenge it below — the signals, their cited sources, their weights, and the score answering back:

What holding the launch cost.

Holding the release wasn't free. The accurate LLM sat finished while I built the surface that could defend it — the retrieval that tied each claim to a source, the abstention state, the override — weeks the team could feel, against a model that already worked. The honest version is that I chose to ship later on purpose, because shipping a correct score nobody acted on would have been the more expensive mistake.

The override feature drew the loudest pushback inside the team — logging an analyst's disagreement felt like exposing them. I kept it anyway, and reframed it: not a record of being wrong, but the analyst teaching the model where it was.

Accuracy gets you a correct number. It doesn't get you a decision. The gap between the two is the whole job — and it's made of trust, not math.

The receipt.

Interrogation turned into reliance · every number with its window

Screening ran 60% faster across 42 deals in a 90-day window, and analysts went from ignoring the model to leading their deal memos with it. The LLM didn't get more accurate — they could finally trace every claim to its cited source and see where its confidence came from.

What this one taught me.

When an LLM product stalls, the instinct is to make the model more accurate. But accuracy was never this product's ceiling — belief was. A correct score no one can interrogate reads, to an expert, like a smooth pitch with no references attached — and experts are trained to walk past a smooth pitch, especially from a system that can state a wrong answer as fluently as a right one.

This became my first question on any LLM product. Before tuning the model, I ask whether every generated claim is grounded in a source the reader can open, and whether the model will abstain when it isn't sure — because the trust layer, not the accuracy layer, is usually what decides whether anyone acts on the number.

Want the full version of this work?

Details are confidential; I'll walk through the artifacts and numbers on a call under mutual NDA.

Design Patterns Used in This Case

This project produced two patterns I now reach for whenever AI meets domain experts:

• ML Explainability Patterns: How to ground each generated claim in the cited source it was retrieved from — and surface abstention when the model isn't sure — so a non-technical expert can read the why, not just the verdict.
• Human-in-Loop Patterns: The override-and-explain loop, the visible weighting on each risk factor, and disagreement handled with respect — the moves that keep the expert in command.