Hallucination

A hallucination is a factually incorrect or fabricated model output that appears linguistically plausible.

The risk is that its tone may be indistinguishable from a correct answer. A model can cite sources, legal references or product capabilities that do not exist without expressing recognisable uncertainty.

Why language models hallucinate at all

A language model generates text by predicting likely next tokens from learned patterns. It does not automatically verify whether a statement is true or supported by a source.

Missing, contradictory or highly specific context can therefore produce a plausible but factually incorrect answer. This risk is inherent to generative models and cannot be removed by a single fix.

Measures to reduce hallucinations

Several measures can reduce the risk. RAG provides the model with relevant sources, while clear instructions encourage it to handle uncertainty explicitly.

Automated checks can also compare statements with known facts or sources. Their effectiveness depends on the use case and implementation. No single control eliminates the risk entirely.

Where human review remains essential

For use cases with potentially serious consequences, technical controls alone are insufficient. Guardrails should require qualified review before the output is used further.

Before deployment, define the possible impact of incorrect output. Approval steps, labelling and the required depth of review should follow from that assessment.