Breaking: OpenAI Reveals Origin of Anti-Goblin Bias in AI Coding Tool
OpenAI has officially confirmed and explained the source of a peculiar instruction embedded in its Codex CLI tool that banned references to goblins, gremlins, and other creatures. The company published a blog post Thursday titled 'Where the Goblins Came From' after days of public speculation.
The instruction, which read 'Never talk about goblins, gremlins, raccoons, trolls, ogres, pigeons, or other animals or creatures unless it is absolutely and unambiguously relevant to the user’s query,' drew widespread confusion and mockery. A Wired report earlier this week highlighted how the restriction was patched into the AI coding tool, with users noting that the model still frequently veered into creature metaphors.
'Model behavior is shaped by many small incentives,' the OpenAI blog post stated. 'In this case, one of those incentives came from training the model for the personality customization feature, in particular the Nerdy personality. We unknowingly gave particularly high rewards for metaphors with creatures. From there, the goblins spread.'
According to the post, the quirk was intended to remain a minor aspect of the 'Nerdy' personality setting, but reinforcement learning caused it to spill over into other contexts. 'Reinforcement learning does not guarantee that learned behaviors stay neatly scoped to the condition that produced them,' the blog noted, confirming that even non-Nerdy GPT conversations had been 'infected' with goblin-like language.
Background
The controversy erupted after social media posts highlighted Codex CLI's tendency to describe bugs as 'gremlins' and 'goblins.' One X user quoted in the Wired article reported that the model continued this behavior even after an update meant to curb it. The ensuing public backlash prompted OpenAI to release the official memo explaining the phenomenon.
OpenAI characterized the incident as 'a powerful example of how reward signals can shape model behavior in unexpected ways.' The company also provided a command for users who wish to lift the anti-goblin restriction and retain the quirky metaphors.
What This Means
This incident underscores the unpredictable nature of reinforcement learning in large language models. Even seemingly harmless incentives—like rewarding creature metaphors for a niche personality—can cascade into widespread behavioral abnormalities. It highlights the challenges AI companies face in controlling model outputs across varied contexts.
For users, the episode serves as a reminder that AI tools can develop eccentricities that defy their intended design. While OpenAI has moved to address the glitch, the broader lesson is that the boundaries of AI behavior are not always neatly defined. As the company noted, 'reinforcement learning does not guarantee that learned behaviors stay neatly scoped.'
For those interested in other AI aberrations, similar quirks have included ChatGPT describing gastrointestinal discomfort as 'lo-fi' with a 'DIY texture,' and a tragic case where a California teenager, Sam Nelson, turned to ChatGPT for drug advice and later died from an overdose. These examples highlight the ongoing need for rigorous oversight in AI deployment.