[<< | Prev | Index | Next | >>] Saturday, March 07, 2026
Why LLMs Hallucinate
Someone mentioned this paper to me today, which I admit I only briefly skimmed but it seemed entirely founded on the wrong premises. This was my quick reply (ignore my apparent overconfidence--very casual context):
The paper has it wrong.
They don't understand how the things work in the first place...
It's really pretty simple. Even my Netflix algo illustrates it.
Think about what my Netflix algo does.
It takes a huge sparsely populated matrix of examples -- each of which is just an instance of some real fact(!) -- and it infers a best fit model to explain aka generate those facts.
And in so doing, following a sort of Occam's razor approach to it by minimizing the encoding cost at the same time, it "hallucinates" all the empty slots -- all the presumed facts.
That's literally its job, and LLMs are exactly the same with hierarchy.
Humans do exactly the same thing.
The difference is humans have another piece of hardware bolted on which is a fact store. Now I forget whether this is another hippocampal function or a different brain region [e.g., perirhinal cortex], but one function is: You feed it a fact and it just answers Yes or No for have you seen this before.
Think about it -- you use it all the time when, say, something is on the tip of your tongue.
The difference between you and an LLM is the LLM just goes with first guess by default whereas your brain actually says nope, that's not it.
I see Some behavior of that sort from the cloud LLMs but I think it's again learned to emulate that process rather than it being innate, which means using hallucination hardware to hallucinate a double-check, which gives better results but obviously far from solid.
This is why LLMs excel at creative writing (filling in the blanks with generalized examples) but suck at accuracy (they have no accuracy mechanism at all, inherently).
Humans over time shift toward high-confidence modeling, so things you "just know" you don't nec have to double check, but that always risks hallucinating, even for humans. It tends to be right because high confidence generalization tends to be true. But not always.
LLMs are operating entirely in that mode -- trying to have a model that's so good it even gets the specifics right.
100% analogous to the trailing singular vectors in the Netflix algo: The first, prominent ones encode the causal principles of the model. By the time you get out to the trailing ones, they are literally encoding noise (the random events that lead to specific facts and outcomes and have no systemic cause).
But the encoding cost of those is huge and the consequence of adding them is actually negative in terms of forward prediction.
(And the paper above is trying to figure out how to force the LLMs to do that....)
Key point being: don't try to train your generalization engine to store one-offs. Optimize the generalization, and then use that to semantically compress, store, and index all the world's particulars. (And then retrieve and check against those.)
[<< | Prev | Index | Next | >>]