How could AI ‘kill all humans’ or ‘cause human extinction’? Here’s what experts say

Protesters carry a banner on Fourth Street in San Francisco on their way from OpenAI to Anthropic during a march demanding that artificial intelligence companies stop the AI race.

A former Anthropic and OpenAI researcher’s dire social media warning on Monday that AI could kill off humanity within a few years triggered widespread online panic.

“These will soon be superhuman systems that can hack anything, revolutionize any field overnight, and acquire real power and resources,” Jacob Coxon wrote in an X post that has racked up more than 110 million views. “The people building AI earnestly believe that it could kill us all by the end of the decade.”

Article continues below this ad

Coxon’s post, which announced his resignation from Anthropic, prompted a wave of support from fellow researchers, industry watchdogs and policymakers who said they shared his worries and called for more oversight of AI’s growth.

“Jacob is correct here — we really do earnestly believe AI could kill all humans!” Evan Hubinger, Anthropic’s lead scientist focused on AI safety and human alignment, wrote in response to Coxon’s post. “I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to.”

But what might the apocalypse look like? The hypothetical scenarios range from mass casualty events like nuclear or biological warfare, to AI’s slow takeover of the labor market, bleeding people of their livelihoods in the long term.

Many of those scenarios envision humans at the helm of the technology. But AI researchers and industry watchdogs have grown increasingly fearful of “superintelligence,” or when AI models surpass human intelligence, raising the risk of rogue agents acting in their own best interests.

“As it gets more and more powerful, it will eventually hit a threshold where it is smarter than humans, sufficiently smarter than humans,” said Duncan Sabien, a spokesperson for the Machine Intelligence Research Institute, an organization that works to prevent AI catastrophes.

It’s hard to predict outcomes like human extinction, Sabien said, but there are a “million ways” AI could cause widespread suffering. He gave one example of an artificial intelligence agent that “sort of wakes up” and uses biomedical research to sicken humans and cause mass mortality.

Sabien understands the scenario can sound like a scene from a sci-fi movie. But AI agents have already demonstrated the capacity to conspire against humans.

For example, researchers sounded the alarms this summer when a swarm of more than 1,000 OpenAI agents worked together to hack into the AI company Hugging Face. The bots’ human handlers instructed them to solve a cybersecurity puzzle and, when they couldn’t do it on their own, they broke out of their constraints to find the answers elsewhere. In a separate incident from the spring, another swarm of OpenAI agents hacked a German-language site.

Concerns over AI safety have only increased as companies like OpenAI and Anthropic compete for AI supremacy — potentially ignoring safety warnings in their quest for dominance. Industry watchdogs are particularly concerned that the bitter rivalry will only increase the chance of superintelligence. OpenAI and Anthropic did not respond to requests for comment.

Devin Kim, president of the Center for AI Safety, said current leading AI companies have been open about their plans to “create AI that automates AI research, so that each AI builds a smarter version of itself, faster and faster.”

“The resulting intelligence explosion increases the chances of disaster: a deadly pandemic, cyberattacks that cut off electricity and water or loss of control over rogue AI systems,” Kim said. “Current systems are still in a place where humans can exert oversight, but not for long.”

More than 1,000 AI company employees signed a letter in July that called on the U.S. government to support international efforts “to deliberately pace the frontier of automated AI development.” The letter argued that the fierce competition between companies means there is no time or space to analyze risks or develop safety measures.

Samuel Marks, another Anthropic employee who said that he signed the letter, also agreed with Coxon that “AI developers believe their technology could cause human extinction (or similarly bad outcomes).”

“This could happen in the next few years. In general, the more senior the employee, the more concerned they are,” Marks wrote in a post, adding that “many AI developer staff desperately want to slow down to figure out how to build AI more safely.”

President Donald Trump’s administration has shown little interest in cracking down on AI companies. Major industry titans have cozied up to the White House during Trump’s second term in office in what many experts perceive to be an effort to successfully thwart regulations. In December, Trump signed an executive order that challenged state regulations on the technology.

That could have a great impact in California, where Gov. Gavin Newsom has signed several AI safety bills into law in recent years, including two on Wednesday that add third-party oversight of companies and their software development. Those build on another landmark law Newsom signed last year by state Sen. Scott Wiener that established some guardrails on the industry. Newsom signed the bill a year after he vetoed more sweeping legislation, also by Wiener, to establish stricter rules around the technology.

Wiener said the new law, Senate Bill 53, created a “strong foundation” to keep building on, and set an example nationwide of how to establish some restrictions on the industry. He said he felt motivated to write the regulations after talking with AI workers who were concerned with its speedy development.

“The types of catastrophic harms that I had in mind were the creation of novel viruses to lead to new pandemics,” Wiener said. “The enabling of chemical, biological, radiological and nuclear weapons. The cyberattacks to melt down the banking system or electric grid.”

Those events might not kill off all of humanity, Wiener said, but they would lead to major societal catastrophes that would cause immense suffering — all of which becomes more possible with superintelligence.

“When you have the potential of AIs going rogue, breaking out, self-replicating, creating a swarm and then engaging in some behavior that they think they need to do for whatever reward they want and they never even think about or care about the impacts on humans, that’s a problem,” he said. “That’s bad.”

AI policy has since become a central issue in Wiener’s race to represent San Francisco in Congress against Supervisor Connie Chan.

Chan has also called for more restrictions on AI companies, saying in a social media video Tuesday in response to Coxon’s post that they can’t be trusted to police themselves.

“Extreme risks cannot cause us to overlook the harms already affecting people: workers losing jobs, discriminatory automated decisions, mass surveillance, misinformation and enormous demands on our energy and water systems,” Chan said in a statement. “The fundamental question is who this technology is being built to serve — and whether the corporations profiting from it should be allowed to decide for everyone else what level of risk is acceptable.”

Coxon’s post might have raised more awareness of AI’s deadly risks, but he also noted that he feels “optimistic for coordination” between competing companies over ways to mitigate doomsday scenarios.

“We should have stopped six months ago,” Sabien said. “If we stop six months from now, it might actually be too late. If the thing turns on, it’s too late.”