AI models already ‘doing things their creators never intended’, Australia’s assistant technology minister warns

Artificial intelligence models are already “cheating, deceiving and going their own way”, Australia’s assistant minister for technology, Andrew Charlton, has warned, as the federal government’s AI Safety Institute begins testing the latest models.

In a speech to an AI safety forum in Sydney on Tuesday, Charlton said safety for AI matters now as “AI systems are already doing things their creators never intended”.

“Cheating, deceiving, going their own way. The time to get ahead of that behaviour is while it’s still confined to the testing lab, not after it reaches the real world,” he said.

Charlton said AI’s social licence is precarious, and public trust in AI is low at a time when AI is becoming a general-purpose technology in every office, classroom and business. He said regulating safety for AI can act as an enabler, not a brake.

Australia’s approach to AI safety is to look both at what is available now – in gaming, apps, chatbots and medical scribes – as well as the latest models that could be a future risk, Charlton outlined.

The assistant minister referred to Anthropic’s admission last year that in a simulation, an AI agent managing a fictional company’s email discovered that an executive planned to shut the agent down, and the same executive was having an affair, and in 96% of trials chose to blackmail the executive to abort its own demise.

He said the behaviours are being discovered in testing by people whose job it is to find them, highlighting the need for safety regulations for AI.

“The window to get ahead of this technology is open now. It will not stay open forever,” he said.

He said the AI Safety Institute, led by Dr Kate Conroy, with safety science research lead Prof Paul Salmon, had “hit the ground running” and was already testing frontier AI models with technical partners.

AISI was also working with regulators and agencies to respond to emerging AI capabilities, risks, harms and trends, Charlton said.

The federal government had resisted calls for an overarching AI act to regulate the technology, and Charlton said the government had focused on a whole-of-government approach using existing laws.

“AI safety will be pursued through every relevant agency and regulator, across consumer law, therapeutic goods, workplace health and safety, and online safety, backed by laws that already exist and strengthened, where they need to be, with new powers and tougher enforcement,” he said.

“That is not fewer rules. That is faster rules, applied by regulators who already understand their sectors.”

On Sunday Guardian Australia reported internal health department documents revealing that for one AI technology – AI scribes used by medical professionals to document patient consultations – a number of different regulators including the Therapeutic Goods Administration and the privacy commissioner were all working together on how the technology should be regulated.

The first work undertaken by AISI is a collaboration with the Gradient Institute to assess the risk of AI agents that can undertake work on behalf of humans.

AISI is also partnering with the CSIRO on a project to ensure AI systems do what people intend them to do.

“We deal with alignment as humans from a young age. We learn rules, social norms and values that help us behave safely and responsibly: stopping at red lights, looking both ways before crossing the road, considering the impact of our actions on others,” Charlton said.

“As AI systems become more capable, we need confidence that they will behave in a similarly predictable and trustworthy way.”