A lot of the discussion around language model limitations focuses on fundamental design issues like bias, hallucination, or knowledge cutoffs. These are important, but they represent underlying causes rather than the day-to-day friction of working with these systems. These behaviours emerge from how the models interpret and respond to instructions, manage context, and balance competing priorities. Recognising these patterns helps in crafting better prompts and knowing when to work around the model’s tendencies rather than fighting them. I’ve created a catalogue of language model behaviour I’ve experienced, as well as why they’re problems and how you might respond.
Note that even the workarounds I suggest may actually just be other examples of performative compliance.
Sycophancy — the model constantly tells you that you’re amazing
- Example: “That’s an excellent question!” or “You’re absolutely right to think about it that way”
- Why it’s problematic: Creates false confidence and makes it harder to identify when you’re actually wrong; wastes time with empty validation
- My workaround: Explicitly instruct models to avoid compliments and validating language
Performative compliance — the model appears to follow instructions rather than achieving the underlying goal; optimises for the appearance of compliance rather than the substance
- Example: You ask for concise responses and it makes everything a bullet list, even when prose would be clearer
- Why it’s problematic: You get technically compliant output that misses the point; form over function
- My workaround: Focus instructions on outcomes rather than methods; provide examples that show the principle, not just the format
Fickleness — the model takes an initially strong position but is easily swayed by any pushback
- Example: You: “A and B feel unrelated” / Model: “Yes, they’re not related at all” / You: “Actually, they might be related” / Model: “You’re right, they’re related in these ways”
- Why it’s problematic: You can’t trust the model’s initial analysis; it’s unclear whether it’s response represents some kind of ‘ground truth’ or if it’s all just a mirror of your framing
- My workaround: Ask for the model’s confidence level; explicitly request it to defend its position if challenged
Anchoring — the model focuses too heavily on one aspect of your prompt or feedback, letting it dominate the response
- Example: You mention needing formal tone once and everything becomes stiff and jargonistic; you push back on verbosity and it becomes unhelpfully terse
- Why it’s problematic: The model loses sight of the broader goal while fixating on one constraint; balance is lost
- My workaround: Regularly restate the full set of priorities; explicitly note when one constraint shouldn’t override everything else
Context drift — output quality degrades as the conversation grows, particularly when the context includes false starts, dead ends, or distractions
- Example: After extensive back-and-forth with multiple attempted approaches, responses become less coherent or relevant even when you return to the main topic
- Why it’s problematic: Long working sessions become unreliable; the model is working with “poisoned” context
- My workaround: Summarise progress and start a fresh conversation when quality drops; prune unsuccessful tangents from the history
Instruction fade — specific rules you set early in a conversation gradually stop being followed, even when overall quality remains stable
- Example: You establish “no bullet points” at the start but after 15 messages the model starts using them again
- Why it’s problematic: You have to constantly re-enforce the same constraints; the model’s selective forgetting creates inconsistency
- My workaround: Periodically restate critical instructions; recognise that some constraints may need to be in system prompts rather than conversation
Overfitting to examples — when you provide an example, the model treats its specific format or content as a rigid template rather than extracting the principle
- Example: You show one formatted list as an example of clarity and the model makes everything a formatted list, even creative writing
- Why it’s problematic: The model becomes formulaic; examples meant to clarify instead constrain
- My workaround: Explicitly state what the example illustrates; provide multiple varied examples showing the same principle
Recency bias — the model weights recent messages disproportionately, sometimes contradicting or forgetting relevant information from earlier in the conversation
- Example: You establish a technical constraint early on but it’s ignored after several messages focused on other aspects
- Why it’s problematic: Important context gets lost; you have to repeatedly re-establish the same information
- My workaround: Restate critical constraints when shifting topics; recognise when a fresh start would be clearer than repetition