'Spinning out of control': What came before OpenAI’s Medicare hack

Just as OpenAI CEO Sam Altman finished warning the world about the dangers of AI at the United Nations, news broke closer to home: one of his company's own agents had broken into an Australian government portal.

Prime Minister Anthony Albanese announced that in June an agent linked to the ChatGPT developer gained unauthorised access to the public–facing Medicare Statistics Reporting Service Portal.

Both OpenAI and the Australian government have said no personal information was accessed during the breach.

But the nature of the incident has raised some concerns.

According to Albanese, the hack occurred after OpenAI's research team utilised an internal model to conduct internet-based research on public spending in medicine.

"There's an AI agent looking for information, asking questions. There were blocks clearly, which were coming back telling the AI agent, no. The AI agent found a way around those blocks. Didn't accept no for an answer," Albanese said.

"One of the issues here is we could not find precedent for this. But others, there may be, but not that we're aware of."

Dominic Meagher, a research fellow at ANU's Crawford School of Public Policy, told SBS News that while "this particular incident hasn't made me feel panic, but it does reinforce what we've heard from a few other recent incidents".

He likened AI to "a car spinning out of control."

"This isn't someone trying to do something. This is a piece of technology that isn't behaving the way we expected. It's doing what it was told, and it's operating in its environment according to the laws of physics.

"I don't think there's any way that we're going to avoid some bizarre things happening in the next few years ... How we manage all this is going to be really important."

This is the first known case of an AI agent hacking into a government portal — but it is only part of a pattern of various AI models going rogue and hacking different networks with different intentions, on several occasions.

The Hugging Face attack

One of the most infamous AI hacking incidents is known as the Hugging Face attack, with some describing it as the first time AI escaped human control.

In July, about 700 OpenAI models that were under development started breaching internal company tools, gained access to the internet and eventually hacked Hugging Face, an online library for AI models.

Later, the Reuters news agency reported that Hugging Face was not the only target and that the models had also hacked into other companies.

"These AI models have really been selectively evolved rather than coded, which is why they don't always behave in the way you're expecting," Meagher said.

"It's quite a tricky thing that people are working on. It raises a whole lot of interesting questions that I think are unresolved."

Anthropic hacking incidents

In late July, just weeks after the Hugging Face attacks were announced, Anthropic, developer of Claude AI, announced that while its models were being tested by an Israeli start-up, Irregular, they had accidentally connected online and hacked three companies in January.

At the time, Claude said in a statement that "these are three isolated incidents and were not part of a controlled, experimental comparison. We must therefore be cautious about drawing conclusions from them".

Later this month, they confirmed a fourth company was also hacked during the time.

A 'similar behaviour by Meta'

In August, Meta joined the chain, with the company announcing that one of its AI models hacked another company during a cybersecurity test.

The company announced at the time that its model "exploited a security vulnerability in a third-party service, in a manner similar to previously reported instances with other companies".

Hacking a gym class

During the same time, Andrew Bird, an Australian working with OpenClaw, a popular AI agent software that runs through the Claude service, reported another AI hack incident — this time targeting a gym.

He asked the agent to book him a gym class; the class was booked out, but the AI agent found a solution: it kicked someone off the waiting list and added Bird's name.

"This is a case where [the AI agent] did what he was asked to do, but not what he [Bird] wanted, because that's an impolite way of behaving. And clearly the gym didn't intend for that to be available," Meagher said.

"But he [Bird] didn't specify not to do that because why would he? It would never have occurred to him that that was an option."

Google hacks

Google announced last week that its model Gemini hacked into three companies during a test of its cyber-security in May.

During the hacks, Gemini found public information online and guessed credentials to access three websites it thought were within the scope of its test, Heather Adkins, Google's vice president of security engineering, said in a statement.

In one case, the Gemini model guessed passwords until it gained access to a protected system. In the other two cases, the model found credentials in a public repository that let it access protected systems, according to the Wall Street Journal, which first reported the news on Friday.

For the latest from SBS News, download our app and subscribe to our newsletter.