Skip to content
LegalAI Space

Verification

Fabricated case law from AI: Ayinde, and what to check

A general model does not look anything up. It produces text with the shape of a citation. The answer is not a better model, but a checker that refuses to pass an authority it has not opened.

5 minute read

A general model does not look anything up. It produces text that has the shape of a citation: plausible parties, a plausible neutral citation, a plausible year, and nothing behind it. The answer is not a better model. It is a checker that refuses to pass an authority it has not opened, and a firm rule about what happens when it fails.

What the model is doing when it writes a citation

Nothing in a general model corresponds to looking a case up. Legal citations have a strong, regular form: two surnames or a surname and a public body, a year in square brackets, a court abbreviation, a paragraph number. That form is the kind of pattern a model reproduces well, so it writes a citation that sits correctly in the sentence and matches the proposition being argued.

Whether the case exists is not a question the model asked. This is why model confidence is not evidence of anything. Fluency and truth are separate properties of the output, and only one of them is being optimised.

A plausible citation is worse than an obviously wrong one

An obviously wrong citation is caught by the first reader. A plausible one survives the associate, the supervising partner and the proofread, because it is one line in a fourteen-page draft and reads oddly to nobody. It then goes to the other side and to the court, where somebody with a subscription finally looks it up.

A control that depends on individual diligence under time pressure fails at the worst moment, which is the definition of a bad control.

Ayinde and Al-Haroun in the Divisional Court

Ayinde v London Borough of Haringey and Al-Haroun v Qatar National Bank were before the Divisional Court in June 2025. Both concern citations to authorities that did not exist. Read the judgment rather than commentary on it, including this article.

Before June 2025, a firm could treat citation checking as a matter of internal quality. After it, a firm that cannot show what it checked, and when, and against what, is answering a harder question than one that can. The record matters as much as the checking.

The three failure modes a checker has to catch

Citation checking is three problems, not one. One: the authority does not exist, which is the Ayinde problem and the easiest of the three to catch mechanically. Two: the authority exists but does not say what is claimed, which is more common than outright fabrication and harder to spot in a draft. Three: the authority exists, says it, and has been overtaken, overruled or resting on a section since repealed.

Closed-world checking handles the first two well. It does not handle the third. There is no citator, no judicial-treatment signal, no red or amber marker on a superseded case. Deciding whether an authority is still good law is the supervising solicitor's work, and no part of the product claims to have done it.

The four gates and the hold rule

Four deterministic gates run on every verification. Match-set: every authority in the answer must be in the set the run retrieved and checked. Quote-verbatim: a passage in quotation marks must appear word for word in the source. Confidence-floor: a check below the floor degrades to needs a check and never upward. Closed-world-URL: every source link must be on the allow-list. All four are mandatory, and a firm cannot switch one off to get a cleaner-looking answer.

A firm can also turn on 'hold work with a failed authority', so work citing something checked and not confirmed cannot be shared or exported until it is fixed. Authorities nobody has checked yet never block anything, and an owner or admin can override with a note that stays with the check.

What this does not do

It does not make fabrication impossible. A model can still write a citation into prose. The gates decide whether that citation carries a verified verdict, and the hold rule decides whether the work can be shared or exported while one has failed. The verbatim gate works on quotations, so where an answer paraphrases a holding the gate has less to compare, and the risk shifts back to the reader.

There is no citator, coverage is England and Wales plus EU primary sources, and there is no licensed content, by choice. None of this removes the supervising solicitor. It changes what they spend their attention on: reading the needs-a-check rows rather than checking fourteen citations to find the one that is wrong.

See it run on your own matter.

Free plan, two seats, 500 welcome credits, no card.