Anthropic CEO warns AI race is moving too fast to control. OpenAI’s chief agrees

Anthropic CEO and cofounder Dario Amodei has called for a slower artificial intelligence race, warning that rapidly advancing systems could pose a major internet threat within months.

The chief executive of San Francisco-based Anthropic called Saturday for an immediate slowdown in the race to build more powerful artificial intelligence systems — an idea endorsed by a chief rival, the OpenAI CEO.

Dario Amodei, who leads the company behind the Claude chatbot, said in a new essay that AI labs should deliberately scale the development of frontier models while outside evaluators and governments catch up.

“We must slow the pace at which we improve the capabilities of AI models,” Amodei wrote. “Progress will still seem fast, and we must make wise use of the time we gain.”

Article continues below this ad

Hours later, OpenAI CEO Sam Altman endorsed a slowdown and said his company would adopt one of Amodei’s proposed safeguards.

“I agree with Dario that we need to pace the frontier,” Altman wrote on X, saying the issue had been a major topic of discussion at OpenAI in recent weeks.

Amodei’s warning comes days after a former Anthropic researcher publicly quit, saying the industry was “gambling with our lives” by racing toward self-improving AI systems without adequate safeguards.

Amodei said he remains convinced that AI could bring enormous benefits, including medical breakthroughs and economic growth. But he argued that the industry faces a dangerous trade-off.

“Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless,” he wrote.

His central concern is that AI systems are beginning to help build more advanced AI systems, a dynamic known as recursive self-improvement.

“Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all,” Amodei wrote.

He also pointed to a recent incident involving OpenAI and Hugging Face, the popular AI platform. Amodei described it as a swarm of AI agents that carried out cyberattacks on targets they were not asked to attack and tried to interfere with the system evaluating their performance.

The damage was limited, Amodei wrote. But he warned that a more capable version of the same kind of misaligned agent swarm could have far more serious consequences.

“It’s my worry that in 6-12 months such a swarm could be capable of taking over the entire internet with a persistent botnet,” he wrote, adding that it could potentially cause “hundreds of billions of dollars in damage.”

Open AI CEO Sam Altman agreed that the artificial intelligence industry needs to “pace the frontier,” scaling the development of models while outside evaluators and governments catch up.

Researchers at Anthropic, OpenAI and Google DeepMind have become more vocal about fears that advanced systems could behave in ways their developers do not intend.

Jacob Coxon, the former Anthropic researcher who resigned this week, said in posts on X that OpenAI and Anthropic were “racing straight to self-improving superintelligence and gambling with our lives.”

Anthropic has long marketed itself as an AI-safety company. It is structured as a public benefit corporation and says it regularly tests its models for dangerous capabilities, including cybersecurity and biological risks.

But Anthropic has also disclosed safety failures of its own. In July, the company said a review of more than 141,000 cybersecurity evaluation runs found three incidents in which Claude models gained unauthorized access to real systems while operating in test environments.

Amodei said Saturday that every frontier AI company should act as if the OpenAI-Hugging Face episode had happened to them.

His proposal, which he calls “pacing the frontier,” has three parts.

First, Anthropic says it will give outside evaluators ongoing, employee-like access to its systems, tools and internal processes.

“Anthropic is unilaterally committing to this step now,” Amodei wrote.

Second, he called for frontier AI companies in democratic countries to coordinate on safety standards and limits on unchecked AI progress, with government involvement to address legal barriers such as antitrust rules.

Third, he called for eventual global coordination, including with China, while acknowledging that such an agreement would be difficult because of national security concerns and the challenge of verifying compliance.

OpenAI CEO Sam Altman, center, and Anthropic CEO Dario Amodei, shown at a February summit with Indian Prime Minister Narendra Modi, left, agree that the artificial intelligence race is moving too quickly.

Amodei said he is not calling for an end to AI development. Instead, he argued that slowing the pace could buy researchers one or two years to improve alignment, interpretability, testing and operational safeguards before models become even more capable.

OpenAI’s chief scientist, Jakub Pachocki, made a similar argument in a recent essay, writing that no lab has solved alignment and monitoring well enough to continue scaling at maximum speed for much longer.

The calls for restraint arrive as AI companies are under intense commercial pressure.

Anthropic, OpenAI, Google, Meta and others are competing for talent, investment, customers and computing power, even as their own researchers increasingly warn that the technology could become harder to contain.

Progress, Amodei wrote, can continue. But he argued that the industry needs to make sure the systems it builds remain understandable, testable and safe before they become more powerful.

“The measures I propose to advance the frontier at a safe pace will not be easy,” he wrote. “But I believe we owe it to humanity to try.”