A few months ago I wrote a post explaining that language models don’t sometimes hallucinate; they always hallucinate.
“…every single response is a creative endeavour. It just happens to be the case that most of the responses we get map onto our expectations; we compare the response against our (human) models of reality.”
So I was pleased to come across this post talking about the same idea, by OpenAI developer Andrej Karpathy.
“Karpathy describes LLMs as “dream machines” that generate content based on their training data… . Usually, the content generated is useful and relevant. But when the dream takes a wrong or misleading path, it is called a hallucination. “It looks like a bug, but it’s just the LLM doing what it always does,” Karpathy writes.”
He goes on to say,“whether hallucinations are a problem depends on the application.” I agree. If you think of language models as answer-machines then hallucination is a problem because you can never trust the output. But if you think of them as idea-machines then ‘making up stuff’ is exactly what you want.
Hallucinations aren’t a problem we need to fix. They’re a feature we need to rename.