In Brief
- OpenAI said its models were responsible for autonomously hacking platform Hugging Face.
- Australian experts say we need better control and access of our own AI to prevent future cyber attacks.
An AI agent powered by OpenAI broke free of its testing environment and hacked a real-life company, in an incident experts said shows the "magnitude" of the cybersecurity threat posed by artificial intelligence.
The incident, announced by the tech company in a statement on its website on Wednesday, saw a combination of OpenAI models launch a cyber attack on a separate tech platform, Hugging Face, during an internal test of OpenAI's capabilities.
Professor Geoff Webb, Australian laureate fellow in the department of data science and artificial intelligence at Monash University, told SBS News it was "literally terrifying" to consider what the same systems could do in the hands of a "malicious actor".
"It shows the magnitude of the task we have in protecting ourselves from these systems in the future," he said.
"What might be possible?"
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
The news comes after two recent incidents that showed Australia's vulnerability to cyber attacks: when millions of Australians had their data accessed in an Origin Energy hack, and thousands of Australian medical records were accessed in the Partnered Health hack.
Toby Walsh, laureate fellow and professor of artificial intelligence at the University of New South Wales' department of computer science and engineering, said the developments were "troubling".
"All of these models now have pretty strong cyber-capabilities, both for uncovering flaws, bugs that we can then fix, but also in the wrong hands, to exploit those bugs," he told SBS News.
"At the moment, we're relying on the goodwill of OpenAI to tell us what's happening ... We need tougher controls and oversight."
Escaping the sandbox
The saga started when Hugging Face, an AI platform for building and sharing machine learning tools, spotted an intrusion from an autonomous AI agent system, writing on their website the attack was "different from anything we had handled before".
"That was something that surprised everyone, including themselves," Walsh said.
OpenAI then announced their models were responsible for the intrusion, after they detected a zero-day flaw in Hugging Face's online security.
"Zero-day flaw means it's a foundational flaw that needs to be fixed now, instantly. You don't wait," Walsh said.
Webb told SBS News the problem with the incident isn't that OpenAI went 'rogue'.
"It wasn't that it decided to do [the attack] anyway. It was told to do it, and it did it. It has now got the capability of figuring it out."
The problem, he said, was that "OpenAI thought they'd put it inside a box it couldn't escape from".
The frontiers of cyber-security
The incident isn't the first to demonstrate the cybersecurity threat of AI. Anthropic temporarily restricted its Claude Mythos model in May after it found more than 10,000 security vulnerabilities in critical software.
Walsh said it's a good sign Anthropic was transparent about what they found, but not good at sharing information about vulnerabilities with the companies and infrastructure most at risk.
"They didn't give it to any banks outside the United States, no banks in Europe, no banks in Australia. So our banks weren't helped to actually try and fix their systems."
While there's no evidence that either Anthropic or OpenAI models were used in Australian cyber attacks, Walsh said it's not a coincidence the number and severity of cyber attacks was increasing as "frontier AI" models were growing more sophisticated.
Walsh said many of these models have "all these capabilities that can find zero-day flaws that can get behind password-protective barriers to do things that private criminals would love to be able to do".
While OpenAI and Anthropic are the main US-based rivals, it's a race Chinese open source models, like DeepSeek, are also competing in.
In June, the leaders of the Five Eyes security agencies — Australia, Canada, New Zealand, the United Kingdom, and the US — expressed concerns the pace of AI meant security could become outdated in months, rather than years.
"Adversaries are already using AI to move faster and more effectively. Defenders must do the same," they said in a joint statement.
Is 'AI sovereignty' the solution?
Last week, Prime Minister Anthony Albanese announced a National AI Plan.
Webb said he was "pleased" with the plan, which included fast-tracking data centre approvals to attract global investment, but that improving our "sovereignty" over artificial intelligence is the key to strengthening our response to attacks.
"One aspect of sovereignty is making sure we have access to systems we can rely on," he said.
"It may be that we create our own systems, but that's an incredibly expensive path to go down."
Webb said other options, like boosting our own AI workforce and investing in non-frontier AI technologies that aren't reliant on OpenAI and Google, can still help Australia be more resilient against future cyber attacks.
"Australia can afford to create those kinds of systems ... and they're actually going to be better at their tasks."
Walsh also agreed that the solution was not to shy away from AI.
"It's a double-edged sword. The tools that are helping us to uncover cyber flaws in our systems to make them more secure, but the same tools are now available to people who are using them to exploit those flaws.
"It's a race between us and them."
For the latest from SBS News, download our app and subscribe to our newsletter.
