Jeroen Alessie showed me this interesting problem yesterday: ask a language model how many r’s there are in ‘strawberry’.
Believe it or not, this went on for another two iterations; Claude kept on insisting that there are 2 R’s in ‘strawberry’.
To be clear, I wouldn’t normally keep saying the same thing in response, but I wanted to see if anything changed. It didn’t.
Many people would look at this example and scoff; ‘Look how dumb language models are’. However, if you think about language models as people, you’d try a different approach when you see them get something wrong. Imagine if you saw a colleague or student make a mistake, and you just kept saying, ‘That’s wrong’.
So I started a new chat, and tried a different approach:
I’d say that the second example shows a measure of something we might call ‘intelligence’ in the response from Claude.
I’ll say it again, you’re getting poor responses, you’re probably doing it wrong.