- Role
- Design Lead
- Surface
- Technical-DD platform · VC + PE
- Cycle
- 3 wks → 4 days
- Status
- Shipped · under NDA
A partner has four days to decide whether a startup's technology is real, and millions riding on the answer. An LLM could read the whole codebase in an afternoon. The diligence still took three weeks. The bottleneck was never the reading.
A diagnosis · symptom → wrong suspects → the real culprit
The symptom: a fast model, and a verdict nobody would sign.
VC and PE partners stake millions on a startup's technology on claims they will never personally check. They can't read the codebase. They can't reproduce the benchmark. So the deal stalls in a three-week diligence cycle while analysts assemble something a partner is willing to put their name to.
An LLM handled the hard technical work in an afternoon: read the unstructured repos, read the design docs, score four classes of risk — code quality, architecture risk, team velocity, founder credibility. The extraction was never the problem.
The problem was that a partner would read a clean machine verdict and still pick up the phone to re-check it by hand. The LLM produced an answer. It did not produce something anyone would stand on. That is what kept the cycle at three weeks.
The usual suspects: a smarter model, a higher score.
The obvious fixes all pointed at the engine. Make the model read more. Train it on more deals. Surface a single, confident risk score per company so the partner has one number to act on. Every one of those sharpens a tool the partner had already decided not to use.
Because a partner staking millions does not distrust a number for being imprecise. They distrust it for being unaccountable. A higher-confidence score the partner still can't trace back to evidence is just a more persuasive thing to be wrong about — and the first time it's wrong on a real deal, the whole platform goes back in a drawer.
The real culprit: the verdict had no chain of evidence.
The verdict was right. That was exactly why it was dangerous. A correct verdict the partner couldn't audit is indistinguishable, on the day of the deal, from a confident hallucination — the exact failure mode an LLM is prone to, and the one this design is built to guard against. The missing piece was never accuracy — it was provenance: grounding, a path from the generated verdict down to the cited source behind it, the repo scan, the infra audit, the interview note. Retrieval-grounded, in the language of the field: every claim traceable to the document it came from.
So the design work sat downstream of the extraction, in the trust it had to earn. Every one of the four signals carries its own confidence and, more importantly, its own cited source. Each score maps to a next step a partner can actually take — proceed, attach a condition, send an analyst back. The one decision that organized everything: no verdict is reviewable until its evidence is. A partner cannot sign off until they've opened all four signals and seen the trail underneath each one.
Try it — open each signal to see the evidence, then sign:
Dossier · Company K
Technical verdict: proceed with conditions
Core modules read clean; test coverage thins around the billing paths.
source: repo scan · static analysis pass 02
A single job queue becomes the chokepoint well before the growth plan needs it to hold.
source: design-doc review · infra audit 01
Merge cadence is steady, but two contributors carry most of the commit history.
source: repo scan · commit log 12 mo
Claims in the deck match the demo; one benchmark could not be reproduced.
source: repo scan · interview note 04
review all four signals first
That sign-off gate looks like a small interaction. It was the whole argument. It turns the LLM from an oracle you either believe or ignore into an evidence file a partner walks through, each verdict grounded in its cited source — which is the difference between a tool that sits in a drawer and one that compresses three weeks into four days. The analyst's overrides feed back to the model, so the trust runs both ways.
What it cost: I made the partner do more work, on purpose.
The honest line: the gate adds friction. A partner who just wants the number has to open all four signals before they can sign — I chose to slow the one moment everyone wanted to speed up. The first reaction was exactly what you'd expect: why am I clicking through evidence when the machine already decided?
The answer held because the friction is the product. A verdict a partner clicks through is a verdict they'll defend in the investment committee; a one-tap score is one they'll quietly re-check by hand, which is how you end up back at three weeks. I priced the extra clicks in on purpose. The thing that actually compressed the cycle wasn't a faster model — it was a partner who, for the first time, didn't feel the need to verify it twice.
The receipt.
What changed, and for whom
The rule I took away.
When a model already works and people still won't act on it, the missing piece is almost never accuracy. It's accountability — a path from the answer back to the evidence, sturdy enough that the person staking something on it will put their name to it.
That gap is where I start now when an expert and a model have to share a decision. The fastest way to make an expert trust a model is rarely a better model. It's giving them the receipts, and refusing to let them sign until they've read them.
The framework is the part worth walking through.
Details are confidential; I'll walk through the decisions, artifacts, and numbers on a call under mutual NDA.
Send me the roleDesign Patterns Demonstrated
- Confidence Score Patterns: Per-signal and per-document confidence, each one mapping to a next step a partner can take.
- ML Explainability Patterns: Provenance on every claim — a clean drill from the summary down to the source signal and code-quality measure behind it.
- Human-in-Loop Patterns: Analyst overrides fed back to the model; partner sign-off as real workflow, not a rubber stamp.
I write about this on The Trust Layer ↗