Last reviewed:
What is an AI hallucination? Definition and pitfall to avoid
A hallucination is a factually false statement but formulated with confidence by a generative AI model. It stems from the statistical mechanism of generation: the model predicts the most likely next word, with no native capacity to verify. It is the main operational and legal risk of an enterprise AI deployment.
Hallucination is not a bug, it is a structural property of LLMs. Four mechanisms produce it. Excessive generalisation: the model extrapolates a plausible response from patterns seen in training, even when the precise topic was not covered. Interpolation between contradictory sources: the model merges diverging information into a smooth but inaccurate statement. Pressure to respond: without guardrail, the model prefers to generate a confident response rather than acknowledge ignorance. Temporal mismatch: models trained before a given date are unaware of subsequent developments (knowledge cutoff). Three factors significantly reduce hallucinations: use of inference-time retrieval (RAG or web search), explicit instruction to the model to flag uncertainties, and systematic human validation on outputs with stakes. None eliminates the phenomenon entirely. Even when given the document, models still invent: on Vectara's leaderboard (September 2026), which measures document summaries, most OpenAI, Anthropic and Google models add information absent from the text in 5 to 15% of cases (6.5% for GPT-6 Sol, 10.4% for Gemini 3.1 Pro, 12% for Claude Opus 4.7).
Concrete example
Moffatt v. Air Canada case, February 2024. A customer queries Air Canada's AI chatbot on bereavement fare policy. The chatbot invents a retroactive refund procedure within 90 days, whereas the company's real policy requires booking at normal fare then refund request before travel. The customer follows the chatbot's instruction, pays the normal fare, and is refused the refund. He sues Air Canada before the Civil Resolution Tribunal of British Columbia, which condemns the company in February 2024 to honour the policy invented by its own chatbot. First major Anglo-Saxon legal precedent on the editorial responsibility of an AI chatbot.
A second case, in a professional context: Mata v. Avianca, June 2023, US federal court in New York. Two lawyers from Levidow, Levidow & Oberman filed a brief citing six court decisions fabricated by ChatGPT. When one of the lawyers asked the model whether the citations were real, it had "confirmed" they were. On 22 June 2023, Judge P. Kevin Castel sanctioned their bad faith: a $5,000 fine, owed jointly and severally by the two lawyers and their firm, and an obligation to write to their client and to each judge falsely named as the author of the fabricated decisions. The case has become the textbook example of unchecked generative AI use in court.
Comparison
| Technique | Effectiveness | Implementation cost | Typical use case |
|---|---|---|---|
| RAG (Retrieval-Augmented Generation) | High on facts grounded in a clean document base | Medium: vector infrastructure + continuous source curation | Assistant answering on a company knowledge base (support, HR, legal) |
| Chain-of-verification (self-check) | Medium: reduces errors but token-expensive | Low: prompt modification only | High-stakes content generation (summaries, reports) with no external data |
| Grounding + mandatory citation | High: the model is constrained to source every claim | Low to medium: prompt instruction + output post-processing | Analytical reports, document summaries, legal answers |
| Human-in-the-loop (systematic review) | Maximum: no unvalidated output reaches production | High: human bottleneck, doesn't scale beyond a certain volume | High-stakes decisions (legal, medical, financial, public communications) |
| Structured output (enforced schema) | High for structured data, low for free-form generation | Low: native in modern APIs (JSON mode, tool use) | Data extraction, classification, form filling |
FAQ
Why do AI models hallucinate?
Hallucination is structural, not a bug. An LLM predicts the statistically most likely next word, with no native ability to verify its claims. Four mechanisms drive it: over-generalisation, interpolation between conflicting sources, pressure to answer rather than admit ignorance, and time lag (knowledge cutoff).
How can I detect an AI hallucination?
Three signals: citations that don't appear anywhere on external search, oddly precise numbers with no traceable source, and confident claims on recent or niche topics. Any output with legal, financial or medical stakes needs independent human verification: even when summarising a document they were given, most major models add information absent from the text in 5 to 15% of cases (Vectara leaderboard, September 2026).
How do you reduce LLM hallucinations?
Three effective levers: RAG (Retrieval-Augmented Generation) to ground responses on source documents, explicit prompting to flag uncertainty, and systematic human review on high-stakes outputs. None eliminates the phenomenon; they cut its frequency and make errors detectable.
Who is legally liable for an AI hallucination?
The company deploying the chatbot or AI assistant remains liable for the statements produced. The Moffatt v. Air Canada ruling (February 2024) confirmed it: Air Canada was ordered to honour a refund policy invented by its own chatbot. The AI vendor is not party to the customer's claim.
Do all AI systems hallucinate?
All LLM-based generative models hallucinate, including the most recent. Deterministic systems (rule engines, full-text search) don't hallucinate but don't reason either. Hybrid architectures (RAG + LLM + verifiable citations) reduce the risk but don't remove it; hallucination is a property of the generative paradigm.
Is RAG enough to eliminate hallucinations?
No. RAG significantly reduces hallucinations by grounding answers on source documents, but the model can still rephrase, mix or distort retrieved passages. A sound setup requires explicit citation of source passages, careful chunking, and continuous evaluation of the grounding rate.
See also
Further reading
Moffatt v. Air Canada, Civil Resolution Tribunal of British Columbia, February 2024
Sources
- Moffatt v. Air Canada, Civil Resolution Tribunal of British Columbia, decision of February 14, 2024 (2024 BCCRT 149). https://www.canlii.org/en/bc/bccrt/doc/2024/2024bccrt149/2024bccrt149.html
- AI Incident Database, incident #563, Air Canada chatbot hallucination, 2024. https://incidentdatabase.ai/cite/563
- Mata v. Avianca, Inc., US District Court for the Southern District of New York, sanctions opinion of 22 June 2023 (FindLaw). https://caselaw.findlaw.com/court/us-dis-crt-sd-new-yor/2335142.html
- Hallucination Leaderboard (HHEM), Vectara, updated 22 September 2026 (hallucination rate when summarising a provided document). https://github.com/vectara/hallucination-leaderboard