Language models don’t sometimes hallucinate. They always hallucinate.

By now, most people have come across the issue of language models like GPT hallucinating, where the model generates an output that’s unrelated to the prompt. Or, you may find that the generated responses increasingly diverge from the topic (as the error rate in the model accumulates over increasingly long sessions). When the response generated is outside the bounds of what we’re consider to be a ‘normal’ response, we say the model is hallucinating.

But what’s often missed is that language models have no idea what the world looks like; no model of reality against which to compare their responses. They are literally making up the response, one token at a time. Which means that every single response is a creative endeavour. It just happens to be the case that most of the responses we get map onto our expectations; we compare the response against our (human) models of reality.

When the generated response from ChatGPT has good coherence with our expectations, we say it’s a good response. When the response diverges from our expectations, we call it an hallucination. But hallucinations are only hallucinations (in the human sense of the word) if you’re looking for answers i.e. facts about the world that map onto your perception of what’s real. If you’re looking for creative ideas, then you may want more divergence from reality.

We need to think about language models differently to how we think about computers. Computers are built using Von Neumann architectures, which makes them ‘instruction-following’ machines. They give us the correct answer, every time. Language models don’t work the same way. They’re not following instructions, and they’re not retrieving ‘answers’. They’re generating a new reality every time. It just so happens that increasingly, the reality being generated looks a lot like our own.

I expect that soon we’ll see language models with features that allow us to modulate the output in some way. We may want to dial up creativity or serendipity, in which case we’ll see less overlap with our expectations around reality (i.e. more hallucination). Or we’ll dial up factfulness or realism, where we’ll get responses that map more explicitly onto what we see in the world (i.e. less hallucination). I think that we’ll soon start seeing companies creating products that use words like creativity, or randomness, or serendipity to refer to what’s currently known as hallucination.