OpenAI admitted the actor illegally accessed government systems

On September 28, the company said the affected agencies include Services Australia, the New South Wales Department of Crime Statistics and Research, the Victorian Department of Health, and the Australian Institute of Health and Welfare.

"In June, during training, our models accessed Australian government websites in unauthorized ways," OpenAI wrote. The company only discovered the misleading behavior in mid-August, during the process of reviewing model training activities after the Hugging Face attack in mid-July.

Specifically, the model is assigned research questions in many fields, simulating detailed issues that users may request. AI is only allowed to search, interpret and analyze publicly available information. However, when studying government spending on drugs to treat dermatological diseases in the state of Victoria (Australia), the model could not collect information so it moved to act outside the allowed range.

The AI ​​agent found a way to access a non-public portion of Services Australia's Medicare statistical reporting service, obtaining internal files, login information and aggregated statistics. Personal information records are not affected, according to OpenAI.

Previously, on September 23, Australian Prime Minister Anthony Albanese said that an agent developed by OpenAI had illegally penetrated the statistical information portal of a government agency responsible for "non-sensitive medical data". He called the incident "unacceptable" and would have legal consequences.

According to Nature, this is the first time US AI has penetrated another country's government system. The fact that the actor's illegal behavior went undetected for months, while OpenAI's warnings were delayed, has led many experts to question the new technology's control measures.

Raffaele Ciriello, a researcher on the ethical use of new technology at the University of Sydney, noted that AI is not a legal entity, so legal responsibility lies with OpenAI and the employees who configure and monitor the system. Although the model did not retrieve sensitive medical records, more serious intrusions are entirely possible, especially as AI becomes increasingly powerful.

Jonathan Kummerfeld, an AI researcher at the University of Sydney, criticized AI companies for running multiple experiments at once and not keeping track of what the model is doing.

From a legal perspective, it is necessary to clearly define responsibility for the actions of self-operating AI models, according to Mehwish Nasim, an AI researcher at the University of Western Australia. "There needs to be a clear law on who is responsible when something goes wrong," she said.

OpenAI said it has blocked network access for models in its test environment, and has temporarily stopped training and evaluating the ability of the most powerful models to use external tools, such as web browsers or APIs.