How do you review a contract against your own legal…

How do you review a contract against your own legal playbook with AI?

You do not ask a model to "review this contract". That produces a confident summary of a document nobody disputed.

You give it your playbook — the rules your legal team already applies — and make it argue every clause against those rules specifically. That turns a vague generation problem into a retrieval and routing problem, which is a problem software can be held to.

This is the whole pipeline for a system that does it: four services, three queues, and the one decision the architecture is built around. It is our own build, so every claim here traces to code rather than to a vendor's manual.

For the engineering lead deciding whether to build this, and the in-house counsel deciding whether to trust it. 8 minute read.

Four services, three queues

The system is not one program. It is four containerised services that never call each other directly — each reads a queue, does one job, and writes to the next.

Playbook processing — What it does: Turns a playbook document into structured legal terms · Cost: A few dozen model calls, and none at all for a spreadsheet
Contract review — What it does: Chunks the contract, embeds it, retrieves candidates per playbook term, reranks them · Cost: Hundreds of calls, tens of seconds
Scoring and analysis — What it does: Scores every chunk across twelve legal categories, routes it, analyses it · Cost: The bulk of the work
Word add-in — What it does: Highlights risky clauses in the document the lawyer already has open · Cost: —

End to end, a contract takes minutes rather than hours, and the model calls run into the thousands.

The queue boundaries are not decoration. They are what lets one stage fail, retry and resume without the others noticing — and they are why the second half of this article is mostly about failure.

The playbook is the product

The thing that makes this defensible to a lawyer is that it is not applying general legal knowledge. It is applying theirs.

A playbook arrives as a spreadsheet, a PDF or a Word document. It is parsed into structured terms, and those terms then drive everything downstream: which categories exist, which specialists run, what counts as high risk and why. Two companies can send the identical contract through and get different analyses, because their rules differ.

That also means the twelve legal categories below are a default, not a law. The schema is playbook-driven; a customer can define their own. Any article — including this one — that treats the twelve as a product constant is wrong.

And there is a cost decision hidden in the parsing. A spreadsheet is parsed deterministically, with no model calls at all. The expensive path exists only because PDFs force it. That is a design choice we would make again, and chapter 6 is about it.

Score, route, analyse — and the category with no specialist

Here is the part worth understanding, because it is where the economics live.

Every chunk is scored by twelve agents, one per legal category: Scope of Supply · Commercial Terms · Delivery & Acceptance · Warranty & Liability · Intellectual Property · Regulatory Compliance · Confidentiality & Data Protection · Insurance · Termination · Force Majeure & Resilience · Dispute Resolution · General Provisions.

Then the chunk is routed to specialist analysts — and there are eleven of them, not twelve.

That gap is the mechanism. General Provisions is a scoring category with no dedicated analyst. A high score there does not select a specialist; it fans the chunk out to all of them. Every other category routes to exactly one.

So the system spends twelve calls scoring in order to avoid up to eleven calls analysing, and it only pays because scoring is cheap and analysis is expensive. Roughly three-quarters of the analysis calls never happen. Chapter 1 takes the asymmetry apart properly.

One thing this is not: a novel idea. Conditional routing is a first-class primitive in every serious agent framework. LangGraph's Graph API documents add_conditional_edges for when you want to "optionally route to one or more edges", and the method "accepts the name of a node and a 'routing function' to call after that node is executed."

It also ships Send, returned from a conditional edge, for exactly the fan-out shape described above: "a first node may generate a list of objects, and you may want to apply some other node to all those objects."

The routing is free. The signal you route on is not. Twelve scorers are the cost of knowing where to send the work, and whether that trade pays is an arithmetic question, not an architectural one.

The system is built around partial failure, not around success

Anything that makes thousands of model calls per document will have some of them fail. The design question is whether that is an outage or a number in a config file.

Partial failure is a threshold. Scoring succeeds if at least a configured minimum of the twelve agents return. Analysis succeeds if at least a configured ratio of the selected analysts return. Neither is hard-coded, and chapter 5 argues that making it a constant is the actual mistake.

Retry happens in two places, and failure is declared only when both are exhausted. There is an in-process retry with exponential backoff, and beneath it the queue's own redelivery. Treating either one as the whole story is the common error — chapter 3.

A circuit breaker sits in front of the model. After a few consecutive failures it opens, fails fast for a couple of minutes, then tries again. Without it, a degraded model provider does not slow the system down; it fills the queue with work that is going to fail anyway — chapter 4.

And two workers must never claim the same chunk. That is enforced with an atomic conditional update in the database rather than a lock, because the workers are containers that can vanish mid-message.

Which raises the failure nobody plans for: your container is processing a message and the autoscaler decides it is idle.

AWS documents the fix and names this exact case in its first example — "a queue-processing asynchronous application such as a video transcoding job where some tasks need to run for hours even when cumulative service utilization is low." Chapter 7, and it has a sting in the tail that costs deployments.

What the lawyer actually sees

Not a dashboard. The contract they already had open in Word, with risky clauses highlighted in place, each carrying a risk level, a justification and a suggested revision.

That is a distribution decision as much as a technical one — it asks a lawyer to change nothing about where they work — and it forces a constraint most teams meet late: an add-in runs in an iframe, so authentication is token-based rather than session-based. Chapter 9.

And the output is filtered before it is shown. Low-risk findings are dropped, headings and table-of-contents fragments are excluded, and — the important one — the same clause found by several different analysts is collapsed into one finding.

That filter is not tidiness. Eleven parallel specialists reading the same paragraph will all report it, and a lawyer handed eleven findings for one clause stops reading. Chapter 2.

The questions this cluster answers

1 — Twelve scorers, eleven analysts — what happens to the category with no specialist?
2 — How do you stop eleven agents reporting the same clause eleven times?
3 — Where should retry live when your worker reads from a queue?
4 — What stops a degraded model from taking the whole queue with it?
5 — Is partial failure an outage, or a setting?
6 — Deterministic parsing or a model — when is the cheapest call the one you never make?
7 — Your container is mid-message and autoscaling just killed it
8 — Can you change an agent's prompt without a redeploy?
9 — Why does the contract reviewer live inside Word?
10 — What AI contract review will not decide for you

Two neighbouring pieces already cover ground this cluster deliberately does not repeat: the routing and retrieval mechanics in Smart Category Routing for Contract Review, and the queue topology in When to Mix SQS FIFO and Standard Queues in an Agent Pipeline.

Where it breaks, and what we are not claiming

The routing degrades when your categories overlap. If two categories describe substantially the same thing, chunks score highly on both, and the router does the safe thing — it sends the work to everyone. The saving quietly disappears and the bill goes back to what it would have been. That failure is not in the model; it is in the playbook's category design, which means it is a customer configuration problem wearing an engineering costume.

Nothing here decides whether to sign. The system finds clauses, assesses them against rules a human wrote, and proposes revisions. Ranking risks against a commercial relationship, deciding what to concede and knowing which counterparty always redlines clause eleven — none of that is in scope, and chapter 10 is entirely about the boundary.

Status, stated plainly. This is a system we built. It runs, it is demonstrable on a call, and it has zero delivered engagements — no law firm is running it today and no number in this article describes anyone's business but our own. We build these on request, against your documents and your playbook.

About Cognilium Cognilium is an AI engineering company — agent systems, retrieval, knowledge graphs, voice AI and the production plumbing that makes them survive contact with real workloads. https://cognilium.ai · https://www.linkedin.com/company/37180269/

Want to know whether this shape fits your documents? Book a 15-minute call — we will walk the pipeline and the failure modes with you, on your playbook if you bring one. No deck.

How do you review a contract against your own legal playbook with AI?