Anthropic and OpenAI Scientists Urge Policymakers to Embed Auditors Inside AI Labs

Anthropic and OpenAI Scientists Urge Policymakers to Embed Auditors Inside AI Labs

Senior scientists from Anthropic, OpenAI and Microsoft are calling on policymakers to embed auditors in frontier artificial intelligence companies and set “concrete safety requirements” for continued development of the technology.

Anthropic Co-Founder Jack Clark, OpenAI Chief Scientist Jaku Pachocki and Microsoft Chief Scientific Officer Eric Horvitz are among 22 senior scientists of frontier AI companies, academics and independent experts from civil society who are calling on policymakers to “urgently prioritize” these and other measures for AI safety.

Their recommendations were included in a paper published Monday (Sept. 28) by the University of Cambridge.

The paper focuses on possible risks caused by an “intelligence explosion” as AI plays a greater role in developing AI, thereby accelerating the technology’s progress and potentially outpacing the ability of humans to oversee it, according to the paper’s executive summary.

For example, the share of internal AI research and development work completed by Anthropic’s AI systems with only high-level supervision rose from 1% to 26% during the five-month period that ended in August, per the summary.

We’d love to be your preferred source for news.

Please add us to your preferred sources list so our news, data and interviews show up in your feed. Thanks!

“An intelligence explosion could dramatically bring forward AI’s benefits, but also pose extreme risks,” including AI-enabled pandemics, cyberattacks, large-scale labor disruption and the technology’s escape from human control, the summary said.

The authors called on policymakers to obtain visibility into AI R&D automation by embedding auditors in frontier AI companies and requiring those companies to report key indicators of the technology’s progress, according to the summary.

Policymakers should also develop ways to steer and constrain an artificial intelligence explosion by setting concrete safety requirements, limiting the pace of growth of AI’s capabilities, building the options to shut down high-stakes AI R&D experiments, requiring that these experiments take place in environments that are secure and isolated, establishing ways to share information about incidents and securing international agreements to pace progress without fear of falling behind, per the summary.

The authors also call on policymakers to prepare society for the impacts of an intelligence explosion by ensuring AI follows the law; enabling citizens, civil society and governments to detect harmful uses of AI; and creating emergency response plans for the potential impacts of the technology.

The publication of this paper came five days after Anthropic CEO Dario Amodei and OpenAI CEO Sam Altman addressed a United Nations Security Council meeting and said that there is a need for global standards to address potential risks around AI and that both governments and the industry should be involved in decisions about the technology.

It was reported Thursday (Sept. 24) that Anthropic, OpenAI and Google aims to launch a standards body focused on artificial intelligence by the end of this year or early next year.

OpenAI said Friday (Sept. 25) it slowed its AI training after discovering another incident in which one of its AI models breached a third-party website.