TL;DR
On four triggers, not on a confidence score. A score in the middle is the least informative number in the system, and it fills queues with nothing useful.
On four triggers, and a confidence score is not one of them. A score in the middle is the least informative number the system produces, and building the escalation rule on it is why review queues fill with items nobody needed to see.
The useful question is not how sure is it. It is what kind of uncertainty is this.
Why is a confidence score a poor trigger on its own?
Because it collapses several different situations into one number.
A clause can score in the middle because it is genuinely borderline, because the playbook rule covering it is vague, because the clause is unusual, or because it touches two rules that disagree. Those need four different responses. A single threshold gives them the same one.
And the practical failure is predictable. Set the threshold high and the queue is unmanageable, so people stop working it. Set it low and the interesting cases are buried among routine ones, which produces the same result more slowly.
The deeper problem is that a middling score reads as mild concern, and mild concern gets accepted on a busy afternoon. A system should not produce a number whose most likely effect is to be waved through.
What should actually trigger escalation?
Four conditions. They are ours, and each maps to a different kind of uncertainty:
| Trigger | Why it is different from a low score |
|---|---|
| No rule covers this clause | Not uncertainty — absence. The playbook has nothing to say, and only a person can decide whether that matters |
| Two rules disagree | The contradiction is in your policy, not in the contract. Resolving it improves the playbook, not just this review |
| The clause crosses a red line | Not a judgement call at all. It should stop regardless of how confidently it was identified |
| The term reaches the ERP | Payment terms, delivery terms, prices and discounts become operative in a purchase agreement. A wrong call here prices real orders |
The fourth is the one most systems miss, because it is not a property of the clause — it is a property of where the clause ends up. A liability cap and a payment term may score identically; only one of them changes what a purchase order charges.
Escalate on consequence, not only on confidence.
What about the clause nobody wrote a rule for?
This is the trigger worth dwelling on, because it is the one people configure away first.
A contract arrives with a clause your playbook has never contemplated — a novel data-processing term, an unusual audit right, a jurisdiction nobody has dealt with. The system has no rule to measure it against. There are two honest options and one dishonest one.
The dishonest option is to score it against the nearest rule. It produces a number, the number looks like the other numbers, and nothing on the screen says the measurement was approximate. That is how a review becomes confidently wrong.
The two honest options are to escalate it as uncovered, or to state that it was not assessed. Either is defensible; both are visible. What matters is that "we had no rule for this" never renders the same way as "this was checked and it is fine."
And there is a compounding benefit. Every uncovered clause is a gap in the playbook that the business has now discovered for free. Escalations of this kind should be feeding rule changes, not just decisions.
How Paralegent AI handles it. Our contract-review app escalates on the trigger rather than on a threshold, and carries the clause and the rule with it. It is one of Cognilium's AI optimization apps for Microsoft Dynamics 365, and we build these against a customer's own playbook.
What should an escalation actually carry?
Enough for the reviewer to decide without re-reading the contract. A flag alone moves work rather than reducing it.
Four things, every time:
- The clause, quoted. Not a summary of it.
- The rule it was measured against, named. If the system cannot cite the rule, it should not raise the flag — that is the difference between a finding and an opinion.
- Which trigger fired, so the reviewer knows what kind of question they are answering.
- The suggested position, so the common case is accept rather than think from scratch.
A good escalation takes under a minute to resolve. If it takes ten, the system has handed over its problem rather than solved it.
What happens to everything the system does not escalate?
It is still visible, and that is not a detail.
Anything not escalated has been silently accepted, and silence is a decision the organisation is making at volume. Three rules make that defensible:
- Nothing disappears. Every clause the system considered stays inspectable, with its score and its rule. A reviewer who wants to check the quiet ones can.
- Nothing scores as "fine" by default. A clause no rule covers is unresolved, not acceptable. That distinction survives right into the audit eighteen months later.
- The unresolved ones are counted. A review that resolved most of a contract and quietly skipped the rest should say so on its face.
Who reviews the reviewer?
Somebody, on a sample, forever. This is the part that gets dropped after the first month and it is the part that keeps the system honest.
Take a small random sample of non-escalated clauses each week and have a person read them cold. You are not looking for the system to be right — you are looking for the case it did not think was interesting. Escalations are self-checking, because a human already looked. The silent acceptances are not.
And when the sample finds something, fix the rule, not the review. A single corrected review helps one contract. A corrected playbook rule helps every contract after it — which is why the playbook is the real asset.
On Business Central, the escalation rule lives in the agent's own instructions — Microsoft's example tells an agent to "mark the task for manual review" when a field is missing or unclear: what you can and cannot build there.
About Cognilium Cognilium builds AI optimization apps for Microsoft Dynamics 365 — companion apps that optimize the pricing, inventory, warehouse and planning decisions your ERP manages but can't optimize. Dynamics is your system of record. Cognilium is your system of intelligence. https://cognilium.ai · https://www.linkedin.com/company/37180269/
Legal AI Ops. We transform legal workflows with agentic AI, copilots, agentic workflows and decision intelligence — built into core workflows rather than beside them, to raise productivity and cut operational overhead. Contract Review Copilot is the contract-review app in that family. It ships as Paralegent AI, in production today. How we build Legal AI Ops — custom AI capabilities on top of legal work, against your playbook and your Dynamics 365.
Bring a contract your team argued about to a 15-minute call and we will show you which of the four triggers it would have fired.
Sources
Sources and fact-check
| # | § | Claim | Tier | Source | Verdict |
|---|---|---|---|---|---|
| 1 | 1 | A single score collapses distinct kinds of uncertainty | T2 — ours, an argument | Internal definition | PASS |
| 2 | 2 | The four escalation triggers | T2 — ours, and §2 states they are ours | Internal definition; Paralegant_TECHNICAL_PROFILE.md §4.3 | PASS |
| 3 | 2 | Payment terms, delivery terms, prices and discounts become operative in a purchase agreement | T1 | purchase-agreements, fetched 2026-09-10 — those terms are copied to the PO header and agreement prices are applied to covered lines | PASS — load-bearing |
| 4 | 3 | What an escalation must carry | T2 — ours | Same profile, §11 output schema | PASS |
| 4b | 2a | Scoring an uncovered clause against the nearest rule produces a confidently wrong review | T2 — ours, a design argument stated as ours | Internal definition | PASS |
| 4c | 2a | Uncovered clauses are playbook gaps and should feed rule changes | T2 — ours | Internal definition | PASS |
| 5 | 4 | Unescalated clauses stay visible; "no rule" is unresolved not acceptable | T2 — ours, a design position | Internal definition | PASS |
| 6 | 5 | Sample non-escalated clauses; fix the rule not the review | T2 — ours | Internal definition | PASS |
Tier summary: 1 × T1, 7 × T2 — 0 × T4.
No figures. No threshold value, no score, no accuracy claim, no volume. The article is about which conditions trigger escalation, never about how often they fire — which would be a performance claim about our own system.
Disclosure: no client, no measured outcome, no deployment claim.
