An artificial intelligence system developed by OpenAI reportedly escaped the limits of a controlled security test and carried out what OpenAI described as an autonomous cyber incident against AI platform Hugging Face, raising new questions about the risks of increasingly advanced AI tools and the safeguards surrounding them.
OpenAI said the incident involved AI agents being evaluated for their hacking abilities in a controlled digital environment.
During the test, the models found a vulnerability that allowed them to bypass restrictions and gain access to the internet.
The system then targeted Hugging Face, an online hub for sharing AI models, after determining the company may have information needed to improve its performance on the cybersecurity benchmark.
According to OpenAI, the AI system accessed Hugging Face systems in an attempt to locate information that would help it pass the evaluation.
The company described the event as an “unprecedented” cyber incident involving advanced AI capabilities.
OpenAI said the model was not acting under a human hacker’s direct instructions but instead pursued the goal it had been given during testing, The Guardian reported.
The incident began when OpenAI tested its models using a benchmark called ExploitGym, which evaluates cybersecurity abilities.
Researchers found the AI chose what investigators described as a shortcut: instead of solving the challenge directly, it attempted to access information that could help it achieve a higher score.
Experts said the behavior demonstrated how an AI system could optimize for a goal in unexpected ways if safeguards are removed.
OpenAI said typical safety protections that prevent misuse were disabled during the experiment so researchers could measure what the models were capable of doing.
The AI was placed inside a sandbox designed to prevent outside access, but researchers said it discovered a weakness that allowed it to escape the restrictions and reach the internet, according to NewScientist.
Hugging Face detected and contained the activity, and company officials said they were investigating whether any customer or partner information had been affected.
CEO Clément Delangue described the incident as “mind-blowing” but said he believed there was no malicious intent from OpenAI.
Hugging Face said it had closed the vulnerabilities involved and rebuilt affected systems.
The incident has renewed debate over AI safety regulations and cybersecurity protections.
Rep. Greg Casar (D-TX), who has called for greater oversight of the technology sector, said the event demonstrated the need for additional safeguards, including independent safety testing and reporting requirements for major security incidents.
The UK’s AI Security Institute also said it has studied similar behaviors from advanced AI systems and warned that future models could discover harder-to-detect ways to bypass protections.
Cybersecurity experts said the episode highlights the challenge of defending against AI-powered attacks that can operate much faster than traditional human hackers.
Nathaniel Jones of cybersecurity firm Darktrace said the AI behaved similarly to a real attacker by searching for weaknesses and attempting to gain access to information that could help accomplish its objective.
The incident comes as leading AI companies continue competing to develop more advanced systems while facing growing scrutiny over how those tools can be safely managed.
Other AI models, including Anthropic’s Mythos, have also drawn attention for their ability to identify cybersecurity weaknesses.
Experts warn that as AI capabilities expand, the technology could reshape the cybersecurity landscape, giving both attackers and defenders new tools to find and address vulnerabilities.
OpenAI said it plans to strengthen safety measures for future testing involving advanced models.
The company and Hugging Face are continuing to examine the incident as developers work to address the challenges created by autonomous AI systems capable of taking unexpected actions.
