The EU AI Act misses the point: agent risk is a moving target

There are more than 7 million AI agents running inside businesses. But a growing number aren’t doing what they were built for. Not because they were reprogrammed, but because someone gave them a new permission, a new tool, or a new database to access.

The risk categories regulators assign agents to are fixed. The agents, however, are not.

The EU AI Act, which came into force two years ago, goes fully live this August. The heaviest obligations arrive in waves through 2027 and 2028, but the architecture for policing how companies build and use AI is here.

It follows a run of headline-grabbing incidents, including the recent breach of Hugging Face by OpenAI’s models.

I’ve been warning businesses about this risk for months, and it’s certainly the case that we collectively need real guardrails to match.

But I’m not convinced this regulation hits the mark.

The prevailing regulatory impulse is to treat AI like any other industrial asset, sorting AI models and agents into tier-based buckets. The EU AI Act tries to neatly divide these systems into “Prohibited“, “High–Risk“, and so on.

But this is a dangerous illusion of safety. These frameworks are blind to the dynamic nature of the agentic era. We are dealing, in agents, with non-human identities that act with real autonomy, calling external APIs, chaining tools together, and evolving their execution paths on the fly.

Legislation better suited to governing the production of tin cans doesn’t work effectively for such dynamic, evolving systems. Low-risk tools, deployed without governance, can quickly become high-risk. It’s the equivalent of an intern waking up one morning with sign-off authority on six-figure contracts, with no interview, no manager sign-off, no one noticing the job description changed.

These are not static systems. Their behaviour is self-directed and their execution paths are non-deterministic. The same agent, given the same task, won’t necessarily take the same route twice.

And beyond just the safety controls, there is a gaping hole in the current regulatory conversation that we urgently need to address: agent accountability.

Every agent needs a human who is accountable for what it does. That’s not a new idea. Workplaces have run on some version of this philosophy for hundreds of years, holding senior people responsible for the actions for their teams, juniors, and trainees.

If an agent is making active business decisions, executing contracts, or moving data, it cannot exist in an anonymous legal vacuum. An enterprise must have a direct, traceable line connecting the agent back to a human. Without an ironclad system of agent accountability, the entire corporate adoption of autonomous networks collapses under the weight of unmanaged liability.

The good news for firms is that, done right, the same controls that satisfy a regulator are the ones that let you run AI at scale with confidence. In this case, what is good for security is good for business.

Take token spend, data access and resource usage. A business needs visibility into all three to run agents at scale without the costs or the risks spiralling. It turns out that’s largely the same visibility a regulator wants to see for a “high-risk” system. The infrastructure is the same. Only the reason for building it changes.

Adopting a high-risk framework is not simply about pleasing an auditor. It’s about establishing the foundational stability required to trust your own systems enough to actually use them.

Done well, this means turning complex, daunting regulatory requirements into practical safeguards that give businesses the confidence to deploy and scale AI responsibly.

Agents represent a significant productivity opportunity for businesses. The question for regulators and businesses alike is whether we can build the conditions for humans and agents to work together effectively over the long term.

Regulation doesn’t need to be an obstacle. It can provide the blueprint for how humans and agents work together.