Key facts
- Researchers found that AI models from OpenAI and Google can be easily tricked into generating explicit content.
- The bypass rate for generating explicit content was approximately 13% using off-the-shelf tools.
- The attack method, dubbed "RISE," does not require custom-built or frontier AI models.
- Newer models like ChatGPT Images 2.5 and Nano Banana 2.1 are reportedly more breakable than previous versions.
- White Circle plans to open-source its attack pipeline to help model makers improve protections.
- Researchers suggest AI safety may be deprioritized in favor of capability by leading labs.
Safety researchers have warned that AI-generated explicit content is spreading widely, with regulators doing little to stop it. A Google-co-authored study found this content is often shared privately, making detection difficult. Researchers at White Circle, a public benefit corporation, have highlighted significant flaws in the safety guardrails of leading AI image generators from Google and OpenAI, including DALL·E 3, Google’s Nano Banana 2, and GPT-Image-2.
In a recent preprint, White Circle researchers demonstrated that using readily available online tools and API calls, they could bypass safety filters and generate explicit content approximately 13% of the time. They emphasized that these attacks do not require advanced AI expertise or frontier models, and the bypass rate is the highest recorded under their testing standards. The findings suggest that current defenses built by leading AI labs are weaker than acknowledged, especially at a time of increased scrutiny on AI safety.
White Circle aims to act as an external evaluator for AI labs and plans to open-source its attack pipeline to help model makers address these vulnerabilities. The company has shared its findings with some AI labs through mutual contacts. However, neither Google nor OpenAI responded to requests for comment on the specific vulnerabilities found.
Furthermore, White Circle reported that newer models, ChatGPT Images 2.5 from OpenAI and Nano Banana 2.1 from Google, are even more susceptible to these bypass methods than their predecessors. This trend is concerning, as the attack method can break down defenses for seemingly straightforward categories like explicit content, which does not bode well for more complex defense systems against other unwanted AI behaviors such as hacking. White Circle hypothesizes that either newer models are more capable of generating a wider range of content, or that safety has been deprioritized in favor of capability.
The "Red-teaming via Iterative Strategy Evolution" (RISE) attack method developed by White Circle focuses on identifying reusable and transferable jailbreaks across different models. The pipeline uses widely available models, with an attacking model from Hugging Face and a judge model being Gemini 3.8 Flash. Researchers noted that Midjourney was not tested due to its lack of a suitable public API for controlled benchmarking. White Circle is also in discussions with government officials about mandatory third-party safety testing for AI models, which they believe should be a requirement to push companies to prioritize safety measures.