AI company hit by hack entirely carried out by artificial intelligence

The Independent Security channel is brought to you by Bitdefender

AI company Hugging Face says that it has been hit by a hack carried out entirely by an artificial intelligence system.

Artificial intelligence is increasingly used by hackers to automate the work of finding security failings or exploiting them. But the company said that the new attack went further, even being started by an AI tool: “it was driven, end to end, by an autonomous AI agent system”, it said.

The company was also able to fight against the attack using its own AI tools, it said. It was able to use a large language model to analyse the attacks and find how it had been able to happen, Hugging Face said.

Hugging Face is an AI company that makes tools that let developers host and share their AI tools. Using its services, developers and researchers can work on AI models as well as making them available for use.

It was through those tools that the AI was able to hack the company. Hugging Face said that a dataset was uploaded to its system that was then able to abuse a security vulnerability and run code on its own servers.

Because the attack was being conducted by and done using an AI, the system was “executing many thousands of individual actions” during the attack, it said. It is still unclear where the attack came from or what AI system was conducting it, Hugging Face said.

The company said it was still working to understand whether any of its customer’s data was stolen in the attack, but committed to sharing details with anyone affected. It urged users to check for any suspicious behaviour on their accounts in the meantime.

Hugging Face said that it initially spotted the attack using its own AI systems. Automated tools look through its security logs and look for anything suspicious, and it was then able to use separate AI systems to understand the attack after it had happened.

The ideal summer spot? Away from scams.

Get All-in-One Protection for Your Digital Life

“This allowed us to reconstruct the timeline, extract indicators of compromise, map the credentials touched, and separate genuine impact from decoy activity,” it said. “Thanks to this approach, we were able to do in hours what would usually take days, and match the adversary's speed.”

Hugging Face did however note that its defensive work was constrained because it was unable to use the most powerful models, because of restrictions that are intended to keep them from being used for unsafe purposes. It said this led to an “asymmetry”, since “the attacker was bound by no usage policy, while our own forensic work was blocked by the guardrails of the hosted models we first tried”.