What if your legal assistant invented a case, a clause, or an argument—out of thin air?
Imagine this: your legal assistant comes into your office with a draft pleading. It looks great—polished formatting, strong arguments, and even a few citations. But when you ask, “Where did this come from?” they shrug: “I made it up. I thought it sounded good.”
You’d be horrified. Rightfully so.
And yet, this is precisely what’s happening when lawyers use generative AI—particularly large language models (LLMs)—without guardrails. These systems are brilliant at mimicking the form of legal reasoning but have no understanding of the substance. Their so-called “hallucinations” aren’t occasional hiccups; they’re a structural risk. When ChatGPT or its cousins invent a precedent or draft a clause “out of whole cloth,” it’s not just a technical error—it’s professional peril.
What’s even more dangerous is that it looks good. In many cases, the generated text feels legitimate. The language is fluent, the tone authoritative, the citations plausible. Until they’re not.
Legal drafting is not just about generating words.
It’s about truthful, factual, and contextually accurate statements that carry legal weight. If a lawyer uses AI-generated content in a will, a trust, or a motion—without validating every detail—they’re inviting malpractice, sanctions, or worse.
This is not a theoretical concern. Judges are sanctioning lawyers who submit AI-written briefs that include fabricated cases. But what happens when the hallucinations are less obvious—when they involve subtly incorrect language in a contract, or a misplaced clause in a will?
The problem isn’t just using AI. It’s using it without understanding its limits. Treating it like a “super assistant” rather than a potentially unreliable narrator.
And that metaphor—legal assistant—may be exactly what we need. If your assistant made something up, you’d fire them. So what’s the standard for a machine that does the same?
Avoiding hallucinations is easy.
At TheFormTool, we built PRO and Doxserá to generate flawless, repeatable documents using rules, facts, and structure you control—not guesses. Offline, secure, and hallucination-free. Just the way legal work should be.
Our Security page.
