Independent testing found leading legal research tools hallucinating in a meaningful share of responses — including the ones sold on the promise that their architecture prevented it.
Start a conversation with the AI Adoption Concierge, already scoped to legal research with AI. Choose the question closest to yours, or describe your situation directly.
Legal research is where generative AI delivers the most obvious value and carries the most measurable risk, and it is the one area where there is genuine independent evidence rather than vendor claims. Stanford researchers testing the leading commercial legal research tools found hallucination rates between roughly 17% and 33% depending on the product — meaning more than one query in six produced misleading or false information on the better-performing systems, and roughly one in three on the worst. The finding that matters most to a firm is not the ranking but the mechanism: both major vendors had marketed retrieval-augmented architectures as preventing hallucination, and the study concluded those claims were overstated. Retrieval reduces fabricated authority substantially. It does not eliminate it, and a workflow built on the assumption that it does will fail.
How these systems work, what testing has actually shown about them, and what a research process should look like given both.
What "grounded" means, what it fixes, and the failure modes it leaves entirely intact.
investigateIndependent testing, what it measured, and why the vendors’ own numbers differ.
investigateWhere AI belongs in the research process, and where the human step has to sit.
investigateHow the Institute approaches research tooling — evidence and process, never a product recommendation.
Less than the marketing implies and enough to be useful, which is an uncomfortable combination. The most-cited independent work is the Stanford study of leading commercial legal research tools, which reported hallucination rates in the range of 17% to 33% across the products tested — one system answering roughly 65% of queries accurately, another roughly 42% while hallucinating about twice as often. Vendors have reported improvement since; figures suggesting citation hallucination in low single digits have been published for at least one product. Both things can be true, and either way the number is not zero, which is the only figure that would let a firm skip verification.
No, and this is the single most consequential misunderstanding in the market. Retrieval-augmented systems answer from documents they have actually retrieved, which substantially reduces wholly invented authority — that part is real. But the Stanford work specifically examined vendor claims that these architectures prevent hallucination and found them overstated: retrieved-but-misapplied authority, mischaracterised holdings and unsupported synthesis all survive retrieval. A firm that hears "grounded" and concludes "no verification needed" has drawn exactly the wrong inference from a genuine improvement.
Generally yes for the search and synthesis stage, and the saving is smaller than the raw time-to-answer suggests. Finding candidate authority and getting an orientation on an unfamiliar area is dramatically faster. Confirming that the authority exists, says what the summary claims, and remains good law is real work that did not disappear. Firms that measure only time-to-first-answer conclude the saving is enormous; firms that measure the whole task to a usable result get a more modest and more honest number.
Most firms permit it with supervision, and the more interesting question is what it does to how juniors learn. Research has traditionally been how junior lawyers build the map of an area — the wrong turns and the reading around are where the understanding forms. A junior who receives a synthesised answer skips that, and may be unable to tell when the synthesis is wrong. Firms that have thought about this tend to require juniors to verify to primary sources rather than accept the summary, which preserves most of the learning while keeping the speed.
Describe how research is done at your firm today. The Institute will help you place the verification step.