Project Overview and Business Context
The client was a legaltech startup targeting UK commercial law firms. Their target customer was a 20-50 fee earner firm where associates spent two to four hours reviewing routine commercial agreements, a cost that was difficult to pass on in full to clients paying fixed fees. The value proposition was straightforward: AI-assisted first-pass review that produces a structured summary and risk report in under five minutes, leaving the associate to focus on the 20% of issues that require genuine legal judgment. The product needed to handle the most common commercial agreement types: NDAs, supplier agreements, SaaS subscription terms, employment contracts, and commercial leases. Each agreement type has a standard clause structure and a set of risk positions that a commercial solicitor would flag: unusual limitation of liability caps, one-sided termination rights, non-standard IP ownership clauses, problematic indemnity language. The AI review needed to extract these clauses reliably and compare them against a configurable playbook of acceptable versus flagged positions. SRA (Solicitors Regulation Authority) guidance on the use of AI in legal practice was a design consideration from the outset. The product positions AI output as a first-pass draft for solicitor review, not as legal advice. Every summary includes a prominent disclaimer that the output requires professional legal review. The law firm, not the AI, is the source of legal advice to their clients.
Technical Architecture and Stack Decisions
The application is built on Next.js with Supabase for authentication, data storage, and file management. Contract documents (PDF or DOCX) are uploaded to Supabase Storage, parsed server-side to extract text, chunked semantically by clause, and stored as vector embeddings in Pinecone with metadata tags for document ID, clause type, and page reference. The extraction pipeline uses a combination of regex-based structure detection (section headers, numbering patterns) and Claude classification to identify and label individual clauses by type. Once a contract is chunked and indexed, the review engine runs two passes. The first pass retrieves each clause type from Pinecone, passes it to Claude with a structured prompt asking for clause summary, risk assessment against the playbook, and a flagged/acceptable/notable classification. The second pass generates an overall contract summary: key commercial terms, total risk flag count by category, and a structured table of obligations, rights, and key dates. The Next.js frontend renders the review as a side-by-side interface: original contract text on the left, AI review annotations on the right, with colour-coded risk flags (red for high-risk clauses, amber for notable, green for standard). Solicitors can add their own annotations, accept or dismiss AI flags, and export the final review as a Word document or PDF. Vercel handles deployment with edge functions for the chunking pipeline to manage cold start latency.
Key AI and ML Components
Anthropic Claude handles all clause-level reasoning. The system prompt encodes the review playbook: for each clause type (limitation of liability, IP ownership, termination, data protection, dispute resolution), it specifies what a standard acceptable position looks like, what positions are typically flagged as high-risk, and what context a solicitor needs to make a judgment. Claude receives each clause text with its playbook context and returns a structured JSON object: clause summary (two to three sentences), risk classification (high/notable/standard), the specific risk reason if flagged, and a recommended solicitor action. The playbook is stored in Supabase and is configurable per law firm and per agreement type, so a firm specialising in technology agreements can tune the IP and data protection clause standards differently from a firm focused on property transactions. Pinecone handles the semantic search layer, which is used primarily for the obligation extraction pass: given the indexed contract, retrieve all clauses that contain time-bound obligations, payment terms, or renewal triggers, regardless of where they appear in the document structure. This retrieval-augmented approach handles contracts where obligations are scattered across sections rather than consolidated, which is common in practice.
Challenges Solved and How
PDF contract parsing is a persistent challenge. Contracts with scanned pages, multi-column layouts, footnotes, and tracked changes all produce messy text extraction. The parsing pipeline applies AWS Textract for image-heavy documents and a custom post-processing step that removes boilerplate headers, footers, and page numbers before clause segmentation. Clause boundary detection is imperfect for contracts without clear section numbering. The pipeline uses a combination of structural heuristics (numbered headings, defined term introduction patterns) and Claude classification to detect clause boundaries, with a fallback to fixed-length chunking for documents that resist structural parsing. Playbook calibration is the ongoing challenge after launch. The initial playbook is built from the client's own precedent preferences and reviewed by a senior solicitor. After deployment, the firm's solicitors can flag AI review errors, which are used to refine the playbook and system prompt in monthly calibration sessions. This feedback loop is built into the product as a structured annotation interface, not a manual email process.
Outcome and Measurable Results
The client ran a six-week beta with three commercial law firms, each reviewing 10-15 contracts per week through the tool. Average time from contract upload to completed AI review was 4.5 minutes for a standard 15-page agreement. Solicitors reported spending an average of 12 minutes on review and annotation following the AI first pass, compared to 90 minutes for a cold review of the same agreement type. High-risk clause detection accuracy, measured by solicitor agreement with AI flag classifications, was 83% for the NDA template and 79% for supplier agreements on the initial playbook. After three rounds of calibration based on solicitor feedback, accuracy reached 91% and 87% respectively. Two of the three pilot firms signed up as paying customers at the end of the beta, with one citing the playbook configurability as the deciding factor: they had specific liability cap positions relevant to their technology sector clients that a generic tool could not accommodate.
Lessons for Similar Projects
Build the playbook editor before pitching to law firms. Solicitors want to see their own risk positions reflected in the tool, not a generic playbook that does not match their practice area. Showing a configurable playbook in the demo converts significantly better than a fixed-template demo. Invest in the annotation interface. The value of the AI review is unlocked only if solicitors can efficiently review, accept, dismiss, and annotate AI outputs. A slow or clunky annotation interface undermines the time saving the AI creates. Calibration must be systematic. Build the feedback loop into the product from day one, with structured data capture on solicitor acceptance and rejection decisions. That data is how the playbook improves. Finally, address SRA guidance explicitly in your product documentation and sales materials. Law firms are cautious about AI adoption and want reassurance that the product positions AI as a tool for solicitors, not a replacement. Embedding that positioning in the product UI, the help documentation, and the sales pitch reduces the compliance conversation that would otherwise block adoption.