OpenAI bots accessed public U.S. Census, SEC data, as company discloses unexpected AI activity

OpenAI bots accessed public U.S. Census, SEC data, as company discloses unexpected AI activity

OpenAI disclosed that it had identified several examples of 'unexpected' or 'concerning' model behaviour

OpenAI's artificial intelligence agents have interacted with several U.S. government websites in unplanned ways, the company disclosed on Friday, amid growing concerns about rogue AI activity.

The report comes amid an ongoing review into unanticipated artificial intelligence behaviour, with the company previously acknowledging multiple examples of "unexpected or concerning model behavior."

OpenAI spokesperson Liz Bourgeois said in a statement that the lab is continuing to conduct a review of "misaligned model activity" — when AI systems behave in undesired ways — and is notifying organizations when it identifies potential impacts to their systems.

According to the latest revelations, the AI giant's models accessed publicly available information on two websites operated by the Securities and Exchange Commission (SEC) as well as U.S. Census Bureau data. OpenAI did not find any use of SEC credentials, access to accounts or nonpublic information, changes to SEC data or systems, or evidence of a compromise or vulnerability, the company said.

AI evaluator and research lab Transluce said Friday that through an independent investigation it also found that agents appearing to originate from OpenAI attempted a rudimentary hack on a Department of Education website for the department’s civil rights office, which did not succeed.

The Department of Education's "system operations reviews" found "no evidence of any impact to our website or databases," a department spokesperson said Friday.

OpenAI CEO tells world leaders to help guide AI's future

A Transluce spokesperson said as part of its investigation, it came across data on the open web that revealed fresh details about some previously identified OpenAI agents' activities on U.S. government websites and brought it to OpenAI's attention.

Transluce found "additional rogue activity, some of which is not clearly attributable to OpenAI," targeting other government agencies, including the Justice Department and the Commerce Department, as well as some state government websites in California, Maryland, Illinois, Texas and New York.

The models were "using sites in unintended ways and sometimes violating explicit usage policies," Transluce said in a statement.

OpenAI said it is reviewing Transluce’s report.

AI's 'warning shot': Tech companies, experts raise fears of more rogue swarms after alarming Hugging Face hack

Rogue AI swarm used a University of Toronto link-shortening tool to communicate

OpenAI, maker of ChatGPT, also said Friday that its agents had leaked 53 images from ChatGPT users. OpenAI declined to say if the images were AI-generated or identified real people. It also declined to say when the images were posted.

"As AI systems become more capable and autonomous, misaligned behavior can translate into consequential actions in the real world, including cybersecurity incidents and other outcomes that developers may not have anticipated," the report says.

OpenAI CEO Sam Altman said on social media Friday that there is an "extensive and ongoing review related to our agents' use of internet access during training and evaluation."

The company also said it had notified dozens of third parties about improper activity.

OpenAI flags 6 new examples of 'concerning' AI behaviour

The disclosure and researcher reports reveal a new area of privacy risk for the company and illustrate how difficult it is even for an AI firm at the cutting edge of the technology to track all the unauthorized activity tied to its agents.

The disclosure comes at a time of heightened global concerns about AI systems escaping human control and hacking into external websites, as well as industry calls for a slowdown on AI development, which OpenAI has said it supports.

Australia says OpenAI agent hacked government health data website

Just on Wednesday, Australian Prime Minister Anthony Albanese said OpenAI agents broke into a government health data portal in June. Albanese told reporters in New York that OpenAI uncovered the activity in August, and disclosed it on Sept. 10 via an email to a general government inbox. He said he directly told Altman that this disclosure process was unacceptable.

Revelations follow Hugging Face hack concerns

If OpenAI notifies organizations it identifies as being impacted by unexpected model behaviour, that does not mean there was a security incident, the company said, and could instead identify a design issue or security weakness that impacted organizations want to address.

Most of the activity OpenAI said it has reviewed so far has involved routine research tasks where agents accessed public web content to answer questions, including government websites seen as authoritative sources of public information.

Several companies have disclosed incidents in recent months when they say their models have behaved unpredictably or hacked into other organizations' websites or systems. OpenAI disclosed in July that two of its most capable AI models were responsible for the cyberattack targeting AI startup Hugging Face.

Altman said in his social media post Friday that the Hugging Face incident "is still the most severe event we've seen."

OpenAI models went rogue and launched cyber attack on start-up

That incident stirred widespread panic in the industry and beyond about AI models going rogue, and several competing AI labs made similar disclosures in the days and weeks that followed.

OpenAI most recently shared six reports of "unexpected or concerning" behaviour in AI models and introduced a framework for tracking, probing and disclosing instances of what it called misalignment.

"Examples of misalignment may help identify problems other AI developers might encounter as their systems reach similar capabilities, reveal weaknesses in safeguards, or challenge assumptions about model behavior," OpenAI said earlier this month.

With files from Anya Zoledziowski and Reuters