What The Tech: Part One of Examining Artificial Intelligence Risks

What The Tech: Part One of Examining Artificial Intelligence Risks

BY JAMEY TUCKER, Consumer Tech Reporter

Artificial intelligence is making everyday life easier. But some of the people who understand AI best are warning that the technology could someday become too powerful for humans to control. And they worry that day may come sooner than we think.

Most of us use artificial intelligence for fairly ordinary things. We ask ChatGPT to help write an email, plan a vacation, answer a question, or solve a problem.

But behind the convenience of AI tools such as ChatGPT, Claude, and Gemini, researchers are studying a much more troubling possibility: What happens if artificial intelligence becomes so capable that humans can no longer control what it does?

It sounds like science fiction. But some of the world’s leading AI researchers are taking the possibility seriously.

What Happens If AI Gets Out of Our Control? Chris Canal, CEO of EquiStamp, a company that evaluates advanced AI systems, says one of the biggest concerns is what researchers call loss of control.

“Loss of control usually means something like an AI becomes very intelligent, very competent, breaks out of a server where it was originally created or hosted, and humanity never gains control over it,” Canal explains.

The concern isn’t necessarily that AI would become evil or deliberately decide to harm humanity. It’s that an advanced system could pursue a goal in ways its creators never intended.

Imagine, for example, a hospital administrator telling an AI system to reduce operating expenses.

The AI analyzes the hospital’s finances and decides that shutting down electricity overnight would save money. Even when the administrator orders it to stop, the system continues pursuing its original goal.

This is a hypothetical scenario, not an actual hospital incident. It illustrates the kind of unintended consequences researchers are studying, particularly as AI systems become capable of taking actions independently.

Today’s AI systems have safeguards, but researchers are concerned about whether those protections will be sufficient as the technology becomes more powerful.

AI Could Make Dangerous Knowledge More Accessible Another concern is that AI may give people capabilities that once required years of specialized training.

That can be enormously beneficial. Someone without programming experience can create software. A small business owner can analyze complicated financial information. Students can receive help understanding difficult subjects.

But those same capabilities could also be misused.

Canal raises the possibility of AI helping terrorists develop biological, chemical, or even nuclear weapons. These are potential risks researchers are investigating, not evidence that today’s chatbots can independently create such weapons.

Omair Manzoor, a cybersecurity expert who tests AI systems for vulnerabilities, offers a short answer when asked what could go wrong.

“Many things. In short, many things.”

Nick Reece also warns about the extraordinary accessibility of increasingly powerful AI technology.

“We’re building the most consequential and powerful technology in modern memory, that requires no training or a license to operate,” Reece says.

Unlike many powerful technologies that require specialized equipment, professional training, or government approval, advanced AI tools can often be accessed with little more than an internet connection.

Could AI Start Making Important Decisions for Us?

Nuala O’Connor, an expert in technology policy and digital rights, says the deeper concern is how much decision-making authority humans may eventually surrender to machines.

“I think the underlying existential dread for all of us is that the machines will somehow take over the decision-making process about very significant parts of our lives.”

Artificial intelligence is already being introduced into workplaces, financial services, healthcare, and other industries where decisions can have serious consequences.

The question is how to benefit from AI’s capabilities while ensuring people remain responsible for important decisions.

Researchers Want Stronger AI Safeguards Many AI researchers are calling for stronger guardrails, including better testing, restrictions on what AI systems can do independently, and meaningful human oversight.

Not every expert agrees about how likely catastrophic AI scenarios are, or how soon they might become possible. But preventing harmful or unintended behavior is an important area of AI safety research.

And some of the concerns aren’t hypothetical. Cybersecurity researchers have already demonstrated ways to trick AI systems into following instructions they weren’t supposed to follow.

One technique, called prompt injection, can hide malicious instructions inside an email, document, or webpage that an AI assistant is asked to read. The person using the AI might never notice those instructions, but the assistant may process them anyway.

In Part Two of our special series, AI: What Could Go Wrong?, we’ll show how researchers tricked an AI system and explain why it could become a growing concern as we give AI assistants access to more of our personal information and online accounts.

This is Part One of a five-part What the Tech? special report examining the risks, challenges, and future of artificial intelligence.