Clause

Where the project landed

RED for further standalone-LLM extraction iteration  →  YELLOW conditional go on the resulting hybrid design (ADR-026).

The production architecture pivots to a classical-primary, LLM-secondary hybrid: the TF-IDF+LogReg model decides whether a clause exists; the LLM only explains and cites clauses the classical model already flagged — a job a follow-up validation probe measured at 100% citation-verbatim accuracy and 93.8% explanation accuracy (with a stated upper-bound caveat: that probe used a frontier model, not the deployed local one).

What makes this worth reading

• Every comparison above ran on the identical, leakage-tested 77-contract test set (seed 42) — no metric shopping, no re-splitting.
• Decision criteria (the stop condition that triggered the pivot) were written down before the deciding experiment ran, not adjusted afterward.
• Three separate LLM interventions were tried and reported as failures against a simple classical baseline — and the project didn't stop there, it used that evidence to design the next architecture.
• A cheap 9-contract pilot's own ranking reversed at full scale — and that reversal is reported as a finding about pilot representativeness, not hidden.

Full documentation