An LLM hallucination is a model-generated output that appears plausible and fluent but is not supported by the input context, training data facts, or available evidence. In practice, hallucinations include fabricated citations, incorrect claims, invented URLs or API responses, and confident answers to questions where the model lacks reliable grounding.
What is Hallucination (LLM Hallucination)?
Large language models generate text by predicting likely next tokens given a prompt. This objective rewards fluent continuation, not truth. When the prompt is ambiguous, under-specified, or asks for information outside the model’s reliable knowledge, the model may still produce a coherent answer by pattern completion—resulting in invented details.
Hallucinations can be caused by multiple factors: missing or irrelevant retrieved context in RAG, overgeneralization from training patterns, prompt pressure (“answer even if unsure”), and distribution shift (the question differs from what the model saw during training). Tool-enabled agents can also hallucinate actions: a model may claim it “checked a database” without actually calling a tool, or it may fabricate tool outputs if tool results are not injected correctly.
Hallucination is best treated as a system-level reliability problem. Mitigation involves grounding (RAG, citations), calibration (encouraging uncertainty), verification (fact-check steps, external tools), and evaluation. Importantly, reducing hallucinations is not only about the base model; it depends on retrieval quality, prompting, and guardrails.
Where it’s used and why it matters
Hallucinations matter in any high-stakes use case: customer support, healthcare, legal, finance, or enterprise knowledge assistants. A hallucinated policy statement can cause compliance issues; a fabricated citation undermines trust; and hallucinated tool actions can trigger incorrect automation. Teams address this with retrieval evaluation, answer attribution checks, constrained outputs, and human review for sensitive flows.
Examples
- Fabricated citation: The model cites a document section that was never retrieved.
- Invented facts: It states a feature exists in a product version when it doesn’t.
- Tool-result hallucination: It claims an order is shipped without calling the shipping API.
FAQs
Are hallucinations the same as lying? Not exactly. The model is not intentionally deceptive; it is generating likely text without a truth objective.
Does RAG eliminate hallucinations? It reduces them when retrieval is strong and the model is instructed to use only provided evidence, but poor retrieval can still lead to incorrect answers.
How do you measure hallucinations? Use labeled factuality tests, citation faithfulness checks, and adversarial “unanswerable” queries that should trigger uncertainty or refusal.
What are common mitigations? Improve retrieval (hybrid search, reranking), require citations, add verification tool steps, and enforce policies like “say you don’t know” when evidence is missing.
