AI hallucination

An AI hallucination is the most-cited failure mode of language models: the model produces text that sounds authoritative but is factually wrong. A made-up book title, a fake court citation, a confident answer about an API that doesn't exist.

Why it happens: LLMs are trained to produce plausible next tokens, not true next tokens. The objective rewards fluency. When the model doesn't know something, it tends to interpolate from patterns in its training data rather than say "I don't know."

The risk is uneven. Hallucinations show up most often when:

• The model is asked about specific facts (names, dates, numbers, citations).
• The model is asked about its own knowledge cut-off or recent events.
• The model is pushed beyond its training distribution.
• The prompt is leading or contains incorrect premises.

The risk is lowest when:

• The model is reasoning over text provided in the prompt itself.
• The model is asked to summarize, translate, or transform structured content.
• The model has a "say I don't know" instruction in the system prompt.

Practical mitigations: ground every factual answer in retrieved source documents (RAG), require citations the model has to point to, lower temperature for factual tasks, and use stronger frontier models — hallucination rates have been falling roughly an order of magnitude per model generation. Anthropic's Claude and OpenAI's o-series reasoning models have notably lower hallucination rates than earlier-generation models.

The single biggest UX lesson: surface uncertainty. A product that says "I'm not sure, but here are three relevant docs" beats one that confidently fabricates an answer every time.