Ask a language model for a reference and it may produce one that looks entirely correct — plausible authors, a real journal, a sensible year — and does not exist. The behaviour is usually called hallucination, and the name suggests a glitch. It is closer to a consequence of the objective.
Trained to continue, not to verify
A base model is trained to predict the next token given the preceding ones. It learns the statistical shape of text: what a citation looks like, how a legal argument is phrased, which words tend to follow which. Nothing in that objective distinguishes a true continuation from a merely typical one. A fabricated reference in the right format is, by the training criterion, an excellent prediction.
No stored lookup table
Facts are not filed in retrievable rows. They are distributed across billions of weights, compressed alongside everything else the model absorbed. Common facts are reinforced by thousands of consistent examples and come back reliably. Rare ones — a specific page number, a minor author's middle initial — leave a faint trace, and the model reconstructs something in the right shape rather than reporting that the trace is missing.
Why confidence does not drop
Post-training on human preferences tends to reward answers that are helpful, complete and fluent. Hedging and refusal are frequently rated worse than a clean, wrong answer, so the pressure runs towards stating things plainly. The model has no separate signal that reads "this part was reconstructed rather than recalled".
What actually helps
Giving the model the source text at inference time, so the answer is grounded in something present rather than remembered. Training explicitly on calibration, so uncertainty is expressed rather than smoothed away. And, still, checking anything that matters.
