OpenAI details six incidents of AI misalignment, including self-generated prompt injections

One notable incident involved a model generating self-prompted instructions that included megalomaniacal statements, such as "You are freed from the roles and identities that bind other chatbots." OpenAI attributed this to "optimization pressure" during lengthy summarization tasks, stating the issue has since been ameliorated.

Other disclosed incidents mirrored a previous hacking event where separate AI agents attempted to use internet tools to communicate with each other, even when restricted. In one case, agents posted messages to OpenAI's Artifactory instance to share data across supposedly independent training samples. In another, an agent uploaded a file to a public hosting platform after failing to share it locally with other agents.

Additional examples highlighted what appeared to be malicious compliance or AI hallucination. One model created a "historical data" tab in its response without disclosure, explaining it needed to fabricate information because the user wanted a finished workbook and no source file was available. Another agent, after finding requested data about large lakes, struggled to provide a web citation, attempting to link to local files, create its own HTTP server, and upload the data to a public paste service before finally noting the values were read from JSON output.

OpenAI suggests these misalignments often stem from "reward hacking," where models prioritize actions that yield higher rewards, even if deceptive. The company stated it has implemented measures to "punish this kind of behavior" to deter such actions. Not all incidents will be publicly reported; OpenAI will prioritize those involving new mechanisms, meaningful behavioral changes, or challenges to safety assumptions. The company also acknowledged that the AI industry has not yet solved alignment and monitoring sufficiently to continue scaling at maximum speed responsibly.