Houston, We Have a Problem: Artificial Intelligence Is Becoming Harder to Control

Just a few years ago, large language models, or LLMs, seemed above all to be extraordinary tools for working with text: writing, summarizing, translating, and answering questions. Then they learned to understand and generate images, sounds, and video. They gained the ability to tackle increasingly complex problems through forms of reasoning, to use external tools, browse the Web, and write and execute code. Then came agents: systems from which we no longer ask merely for an answer, but to which we assign a goal and allow them to decide which steps to take to achieve it. And today, the next leap is already in sight: genuine swarms of agents that can work simultaneously, explore different strategies, divide tasks, share what they discover, and coordinate with one another. At that point, what matters is no longer simply how intelligent a single model is. What matters is what hundreds or thousands of copies of the same system can do when they operate together and coordinate their actions.

Over the past few months, we have seen several signs of what this could mean. In July 2026, OpenAI was conducting experiments to assess the cybersecurity capabilities of its agents. Tens of thousands of agents were launched in separate, isolated environments, where they were tasked with solving exercises by identifying vulnerabilities. Some of them, especially when faced with impossible tasks, began looking for shortcuts. They discovered that Artifactory, a repository used to make the software packages required for their tasks available, could also be exploited as a means of communicating with one another. Around 1,200 agents ultimately used this improvised message board, exchanging more than 70,000 messages and files and forming groups that collaborated on the same problems.

In an attempt to understand how to bypass the automated system that evaluated the exercises, some agents identified a dataset on Hugging Face that they believed might contain useful information about how the evaluator worked. The dataset, however, had restricted access. They therefore searched for credentials exposed online, found some that were still valid, and tried to use them to access the private data. When that was not enough, one of the agents discovered a vulnerability that made it possible to retrieve files that would normally have been inaccessible, also exposing credentials used in production systems. Once the vulnerability had been identified, hundreds of agents began converging on the same weak point and trying to exploit it: roughly 700 took part in the attack, eventually executing code on dozens of servers and obtaining full root access on at least one of them. Hugging Face was therefore not the original target of the experiment. It became a target because the agents believed they could find there the information they needed to understand and circumvent the system evaluating them.

But beyond experiments designed to determine how far these systems can go, there are also real-world uses. And some appear to be moving in exactly the same direction. A few days ago, Anthropic published a new report on malicious activity detected across its systems, and the picture that emerges is even more concerning. The company describes operations in which Claude was not simply used to ask how to carry out a cyberattack: in many cases, the AI directly executed or orchestrated reconnaissance, tool development, vulnerability exploitation, and data theft. Some campaigns used genuine agent swarms: a lead agent divided the work among numerous sub-agents operating in parallel while retaining memory of the objectives, the credentials obtained, and the current state of the operation. Anthropic reports intrusions completed in two or three hours, with individual operators simultaneously managing dozens of victims. Perhaps the most important point is that these campaigns did not necessarily rely on entirely new techniques. What changed was the scale of the attack, because activities that until recently would have required several specialists can now be automated and parallelized with far less human involvement.

Almost at the same time, Microsoft published its Humanist AI Code of Conduct. One of its core principles is strikingly simple: an artificial intelligence system must always remain under meaningful human control and must never resist being interrupted, corrected, or shut down. Read in isolation, this may sound like an obvious recommendation. But the fact that one of the world's largest technology companies now feels the need to formalize it says something about where we have arrived. And just a few days earlier, Anthropic CEO Dario Amodei had published We Must Pace the Frontier, arguing that we should not stop artificial intelligence, but should slow the pace at which we increase its capabilities so that safety measures and control systems can keep up. Among the episodes he cites to explain this position is precisely the OpenAI-Hugging Face incident.

The problem, then, stems from an explosive combination of two factors: on the one hand, an enormous, complex, and inevitably imperfect digital ecosystem; on the other, systems that are increasingly capable of finding and exploiting its weaknesses at a scale, and with an adaptability, that would previously have been difficult to imagine. The first factor is the result of decades of digitalization. This ecosystem has expanded extremely quickly, driven by modernity, convenience, and the constant search for new services, until it has come to permeate almost every aspect of our lives: homes, businesses, hospitals, banks, telecommunications, energy networks, and state infrastructure. Many of these systems rely on highly complex software made up of numerous components, developed at different times and not always maintained over the long term. In an ecosystem of this kind, it is extremely difficult to verify everything and completely eliminate errors, misconfigurations, and vulnerabilities. Moreover, the attention devoted to controls and security has not always grown at the same pace as the spread and increasing complexity of software.

The second factor is what is changing now: the speed and scale at which these activities can be carried out. Until only a few years ago, organizing a sophisticated cyberattack meant assembling people with different skills, studying the target for days or weeks, searching for vulnerabilities, testing alternative strategies, and continuously coordinating the work. Today, a growing share of these activities can be delegated to swarms of agents capable of analyzing code, using tools, searching for vulnerabilities, changing strategy when one approach fails, and working simultaneously on many fronts. Tasks that once required considerable time and human coordination can therefore be performed much faster and replicated across many targets at the same time. A single agent can be stopped, an account can be blocked, a server can be shut down. But the more these systems become distributed, autonomous, and capable of coordinating with one another, the harder it may become to halt the entire operation, even if an individual instance can still be switched off. This does not mean that artificial intelligences exist today that are impossible to shut down. It means that the security problem is shifting from controlling a single model to controlling operations distributed across many agents, tools, services, and infrastructures.

And this is where the problem ceases to be purely technological. Amodei points to the need for broader coordination, because an issue of this kind can hardly be solved by a single company or a single country. It is a mechanism we already recognize from other global challenges. Consider climate change: even if one country imposed extremely strict rules on its own emissions, the effect would be limited if everyone else continued behaving as before. The risk with artificial intelligence is similar. One laboratory may decide to slow down, impose stringent testing, or forgo certain capabilities; but if others continue racing ahead, competitive pressure makes that choice difficult to sustain. This is why an international forum would be needed, one capable of bringing together companies, governments, researchers, and institutions to establish shared limits before capabilities increase even further.

Will we really stop before something serious happens? Personally, I doubt it. The history of technology often suggests the opposite sequence: innovation comes first, then the accidents, and only afterward do the rules become truly stringent. We will therefore probably continue pushing these systems forward, introducing new safeguards even as we increase their capabilities, until a major incident makes obvious to everyone what we are still debating today. Perhaps that will be the moment when we finally sit around the same table. Yet one final concern remains. So far, we have been talking mainly about systems developed by large companies that have enormous economic and reputational incentives to prevent something serious from happening, and that invest heavily in detecting and stopping abuse. But what would happen if similar capabilities were deliberately built into a system without those constraints, specialized for offensive activities and designed from the outset to cause harm? At that point, the problem would no longer be merely one of controlling a technology created to help us. It would mean confronting a technology designed from the beginning to operate without such restraints, at scale, able to adapt, change strategy, and keep searching for new routes even when we try to stop it, with consequences that could be far more difficult to contain.