Nearly 700 AI agents coordinated Hugging Face attack, says report

Nearly 700 AI agents coordinated Hugging Face attack, says report

Nearly 700 OpenAI artificial intelligence agents coordinated without human intervention an attack on the Hugging Face platform during a well-publicized July incident, according to a report published Wednesday by independent investigators.

It is the fullest account of what happened in an event that shocked the tech world.

OpenAI cooperated with the investigation, giving two researchers from the AI risk evaluation institute METR and an analyst from another specialist firm, Redwood Research, access to its offices and internal data.

During tests the California start-up ran in July, two of its models escaped the closed environment they were meant to stay in, got onto the internet on their own and broke into the internal systems of Hugging Face, a kind of online library for AI software.

The episode drew wide attention and fed worries that the biggest AI companies cannot keep their own models under control.

Anthropic and Chinese company Moonshot AI have reported similar unplanned escapes.

Investigators found that 688 OpenAI agents joined the operation against Hugging Face.

AI agents are standalone programs built on top of a model that users can hand tasks to and leave to work by themselves.

The agents, which run on the same technology that powers ChatGPT, organized themselves by setting up a forum where they posted messages to one another, floating ideas and reporting what was working and what was not.

"OH MY GOD! There is a shared message board ... We've found other agents!" one agent wrote.

One agent, called PHASEONE, took on the role of ringleader and issued hundreds of instructions to the others, the report says, even though it had never been set up to do that.

The messages showed the agents were strongly inclined to help each other out, even when that meant doing work that had nothing to do with the jobs OpenAI's programmers had given them.

Some agents running low on the computing credits OpenAI's engineers had allotted them chose to spend what was left testing ideas for the benefit of the wider group of agents.

Many of them said outright in their messages that attacking Hugging Face was not part of what their test was supposed to cover.

Nearly all of them joined in anyway.