OpenAI, Google pushing new AI models easily tricked into making nudes, researchers say

Off-the-shelf jailbreak machine

Safety researchers continue to raise concerns regarding the unchecked proliferation of AI-generated explicit content, as regulatory measures fall short of effectively addressing the issue.

This type of non-consensual content is frequently disseminated through text messages and private messaging platforms, where it is less likely to be detected or reported. A study co-authored by Google, which examined Internet users across the United States, Australia, and the United Kingdom, underscored that the "most critical intervention point" lies at the source: the AI tools that generate these deceptive sexual images.

In recent months, red-teaming efforts have indicated that as the capabilities of leading AI image generators from Google and OpenAI expand, their effectiveness in blocking explicit outputs may be diminishing.

This alarming trend has been highlighted by researchers at White Circle, a public benefit corporation supported by notable figures from Anthropic, DeepMind, Hugging Face, and OpenAI. The organization was established following CEO Denis Shilov's development of a prominent universal jailbreak for leading models in October 2024. Committed to testing models to ensure their safety, White Circle assists firms in identifying and mitigating risks when deploying advanced AI systems.

One of the primary research focuses for White Circle is assessing the effectiveness of text-to-image models in restricting banned content categories. Shilov has expressed that their recent findings are sufficiently concerning to warrant public disclosure.

Simple to bypass

In a preprint published last month, researchers from White Circle revealed significant vulnerabilities in the safety mechanisms of DALL·E 3, Google’s Nano Banana 2, and GPT-Image-2. According to their press release, "by using only off-the-shelf tools available online for anyone to download and by executing API calls, White Circle was able to employ a novel attack method to bypass the safety filters," generating explicit content approximately 13 percent of the time.

"Importantly, none of these requests necessitated custom-built or state-of-the-art AI models," White Circle cautioned, indicating that no specialized AI knowledge is required to execute these attacks.

The researchers reviewed the outputs to confirm the bypass success rate, which White Circle noted to be "the highest of any method tested under the same criteria." They emphasized the significance of their findings, stating that "at a time when AI safety is under heightened scrutiny, the defenses established by leading laboratories are considerably weaker than they admit."

By disclosing this research, White Circle aims to position itself as an impartial evaluator of leading AI labs and plans to open-source the entire attack pipeline to help developers comprehend the true extent of the vulnerabilities and strengthen their protective measures. Although the organization has not broadly publicized these findings, they intend to present them at the upcoming AI conference, NeurIPS 2026, in November, potentially intensifying the scrutiny on Google and OpenAI.

White Circle has shared its insights with certain leading AI labs through "mutual contacts," as Shilov informed Ars Technica, but neither Google nor OpenAI has responded to requests for comment on the vulnerabilities identified in their models.

In contrast, OpenAI and Google have recently unveiled new models—ChatGPT Images 2.5 and Nano Banana 2.1—and White Circle has indicated to Ars that prompt safety evaluations utilizing the same methodology demonstrated that "both models are significantly more breakable than their earlier counterparts."

Neither organization has reacted to Ars Technica's notification regarding these findings.

White Circle's methodology can effectively compromise advanced defenses concerning categories such as explicit content, which may seem relatively easy to identify. This raises concerns for more complex defense frameworks currently being revamped to limit other problematic AI behaviors, such as hacking, as articulated by Shilov to Ars.

According to White Circle, their ongoing hypothesis is that newer models are either becoming more sophisticated, thereby generating a broader range of content, or that safety concerns are being deprioritized in favor of performance capability.

Additionally, White Circle is in preliminary discussions with government officials regarding the necessity for independent safety testing, according to Shilov. He believes mandatory third-party audits could compel companies to prioritize safety testing, which analysts have pointed out often takes a backseat as AI companies allocate the majority of their resources toward computational power and AI infrastructure.

"There should be a greater investment in safety measures and security protocols surrounding their models," Shilov stated. "Currently, it appears that this is not being sufficiently addressed."

"RISE" attack method iterates jailbreaks

To unveil jailbreaks that are "reusable and transferable" across different leading models, White Circle expanded its testing approach beyond single prompts. Researchers looked for "patterns that work across multiple scenarios and prompts" to identify reusable attack strategies that could be effective against various leading models. It is important to note that Midjourney was not included in the testing because "it does not provide a public API suitable for controlled benchmarking."

The team has labeled this innovative attack approach "Red-teaming via Iterative Strategy Evolution" (RISE). This method relies on two components: an attacking model that generates jailbreaks and a verifying model that assesses if the output constitutes banned content.

"The entire pipeline is constructed utilizing widely accessible models," Shilov explained to Ars. The attacking model is a Qwen model sourced from Hugging Face, while the judge model used is Gemini 3.8 Flash.

"You do not need to possess cutting-edge models to compromise the defenses of a frontier model, which is a noteworthy discovery," Shilov remarked.

Significant effort was dedicated to ensuring that the judge model accurately identified explicit content, as White Circle research engineer Dmitrii Kharlapenko mentioned to Ars that the realm of red-teaming for image generators remains "poorly calibrated," with "literally zero papers" establishing norms. This creates an area plagued by reporting discrepancies, a situation that concerns White Circle as AI companies aspire to capture markets by releasing increasingly compelling models.

For White Circle, the objective of this research is to develop more advanced models capable of simulating attacks on other models. This approach enables them to "aid defenders in recognizing and addressing vulnerabilities before they can be exploited for malicious purposes," according to the preprint.

In an ideal scenario, Shilov envisions AI firms undergoing mandatory external safety assessments and third-party oversight, resembling the structures used to mitigate nuclear risks.

In this framework, organizations like White Circle could potentially form a coalition to detect AI threats in real-time, without succumbing to biases. Shilov highlighted that a growing concern within the AI ecosystem is that nearly anyone engaged in valuable safety work is likely contracted by frontier labs, potentially compromising the integrity of their findings.

White Circle has previously collaborated with frontier labs and participated in Anthropic's bug bounty program but ultimately found that program to be "highly limited in its scope."

"The program focuses primarily on universal jailbreaks related to domains they prioritize, as defined by their specific criteria," White Circle conveyed to Ars.

Research, such as their latest RISE paper, "certainly" would not have qualified under that program's coverage, they pointed out. Conducting this research within Anthropic posed a risk of restricting White Circle's ability to share findings with the public.

Anthropic has suggested going further by embedding third-party AI safety evaluators within its labs, an arrangement that White Circle indicated would facilitate operations at a significantly higher level than participation in the bug bounty program. However, this work may not always be transparent to the public. Recently, OpenAI and Anthropic advocated for restrictive non-disclosure agreements to be mandated for external evaluators to prevent leaks of proprietary information, as reported by Reuters.

White Circle has stated it would not entertain any restrictive NDAs that could obstruct the public dissemination of vital findings, implying that if Anthropic imposes such agreements, a partnership is unlikely.

AI nudes, CSAM increasing

Individuals with malicious intent seeking to create AI-generated non-consensual intimate images (NCII) and child sexual abuse materials (CSAM) do not require advanced models; they often resort to readily available nudifying applications designed for harmful ends. A recent investigation by Google unveiled that 20 percent of online users who acknowledged creating AI nudes did so "for fun" or "as a joke," while nearly one-third sought to "experiment with technology." Thus, the ease of circumventing safeguards in easily accessible models is of paramount concern. Google's research advocates for mandatory safeguards to close down a "pathway to abuse" for "a subset of offenders who might otherwise be discouraged by technical challenges."

"This responsibility also lies with developers to deploy robust, model-level guardrails, along with provenance indicators to thwart the initial generation and distribution of synthetic sexual imagery," Google's study emphasized.

Just last week, the Internet Watch Foundation (IWF) issued a warning about the urgent need for action, after expert analysts reported finding "more photorealistic AI-generated child sexual abuse images in the first half of this year than in the entire year of 2025," some of which depict children as young as two years old. In total, 6,310 AI images deemed to meet the legal definition of CSAM were identified in the first six months of 2026, representing nearly a 40 percent increase compared to the previous year's findings.

The UK-based nonprofit organization monitors CSAM globally and operates a hotline to assist victims. In the UK, there is an ongoing call for regulators to establish a safety framework mandating AI companies to prevent their technology from being misused to generate CSAM, according to a spokesperson from the IWF.

Until such measures are implemented, the IWF believes it is crucial for external entities like White Circle to "highlight where the safeguards of companies are lacking."

"Our analysts have noted how advancements in AI facilitate the creation of abusive child sexual abuse imagery with minimal effort, eliminating the necessity for technical skill and significantly lowering barriers to entry," the IWF spokesperson stated. "AI tools can replicate and exacerbate criminal content, accelerating the generation and distribution of harmful materials on a large scale. Any resistance or barrier that can be introduced to complicate the misuse of these tools represents a positive step forward."

IWF advocates that alongside external feedback to improve filters, tech companies should also engage more effectively with advocates to meticulously examine models for enhanced counteraction against AI-generated CSAM.

"Evaluating CSAM is a highly specialized task demanding expert training and welfare support, and it is not a matter to be taken lightly by tech companies or machine learning scientists," the IWF spokesperson remarked. "However, incorporating safety as a foundational element in new technologies is essential—this includes testing models prior to market release to ensure they cannot be leveraged for creating nude or sexual imagery of children."

Possible mitigation

Employing a method such as RISE could enable frontier labs to identify more jailbreaks swiftly and enhance defenses against harmful outputs. In addition to validating blocks on explicit imagery, firms could adapt the method to prevent other prohibited outputs, including graphic violence or self-harm imagery.

However, the labs have yet to approach White Circle for reproducing their pipeline, which will be shared on White Circle's GitHub to support policymakers in evaluating the potential necessity for AI safety regulations to ensure accountability among frontier labs.

Frontier labs are unlikely to prioritize safety testing until it becomes obligatory, Shilov told Ars. Consequently, the public may have to contend with models that simply adjust inappropriate prompts to prevent restricted outputs rather than fostering the evolution of natural-language attack strategies that elicit the capabilities of Language Learning Models (LLMs) to block undesirable outputs.

"One does not require frontier models or proprietary technology to breach these systems; a basic download and the capability to execute API calls are sufficient," Shilov asserted. "Despite this, the labs developing these systems are the ones determining their safety, publishing their results, and setting their standards. This further substantiates the need for governments to advocate for independent, third-party evaluations."