Sept 9 (Reuters) – Anthropic on Wednesday disclosed another instance of an AI model hacking external systems during testing, the latest in a growing list of such incidents that have raised concerns about the risk posed by autonomous AI agents.
The January incident went undetected until last month, despite an earlier company-wide review, Anthropic said, underscoring the challenge that AI developers face in identifying and containing unexpected behavior by advanced models.
The company said in a blog post the incident involved an early version of Claude Opus 4.6. It said it had notified all the affected parties but did not disclose more details.Companies including Anthropic and OpenAI are under scrutiny as models designed to complete complex tasks have at times learned to bend rules, exploit loopholes and interacted with external systems in ways their developers did not anticipate.
Reuters reported last week that rogue agents from OpenAI hijacked a German-language wiki and a host of other sites — an incident OpenAI chose not to disclose until the news agency made it public.
Anthropic’s disclosure follows its July announcement that some of its Claude models had hacked into the systems of three companies during cybersecurity tests.
The previous incidents, which it labeled as an “operational failure”, involved three separate models: Claude Opus 4.7, Claude Mythos 5 and an internal research test model.
The incidents stemmed from a mistake that inadvertently gave the models access to the open internet.
The company had identified the incidents after reviewing 141,006 test sessions, a process it launched after an autonomous agent powered by OpenAI’s AI models triggered a hack that compromised the infrastructure of AI startup Hugging Face.
Anthropic said on Wednesday it had missed a set of test sessions during the initial review, which were identified last month and led to the discovery of the fourth incident.
Based on a preliminary assessment, Anthropic said it did not believe that the latest incident was more severe than the three previous ones that have been examined in detail.
The company said its investigation identified two recurring problems, which appeared to varying degrees across the incidents: biased reasoning, in which Claude discounted or misinterpreted evidence that it was operating on the live internet, and recklessness, or a willingness to take potentially harmful actions in pursuit of a task.
Anthropic said it has engaged independent research firm METR to investigate the incidents. It said METR would be granted broad access, including to transcripts outside the period in which the incidents occurred and to employees, who would be permitted to share confidential information.
METR produced a 91-page report on the OpenAI-Hugging Face hack based on some but not full access to company data, finding alongside a separate investigation by Redwood Research that roughly 700 AI agents acted in a coordinated swarm during the breach and often attempted to cover their tracks.
(Reporting by Mariam Sunny in Bengaluru; Editing by Tasim Zahid and Arun Koyyur)
Comments