'Too big to be contained': Anthony Albanese's AI plans met a major test

The hacking of a Medicare website by an OpenAI bot could mean a slew of regulatory issues are on the horizon for Prime Minister Anthony Albanese, whose government has labelled the incident as "fundamentally unacceptable".

It's the first known incident of an AI model hacking a government website without human instruction.

A rapid task force led by the Department of the Prime Minister and Cabinet, along with the Australian Signals Directorate, the AI Safety Institute and the Office of AI, was launched on Thursday to investigate.

Deputy Prime Minister Richard Marles called the incident "very serious" although it had a "relatively minor" impact.

But it raises bigger questions about how to regulate a form of technology that's increasingly embedded in our lives.

Who is accountable when AI jumps the fence?

Central to the incident is an AI agent going beyond its parameters, or "scaling the fence", as Marles put it.

He said the agent was first denied the information it wanted, then engaged in what he called "misaligned behaviour" to gain access.

"It was unintended; it wasn't asked to — that's our concern here," Marles said, while OpenAI said in a statement that its models "took actions we did not intend".

Dennis Desmond, of cybersecurity firm RAVINN and an adjunct senior industry fellow at the University of the Sunshine Coast, said blaming a "rogue" agent wasn't enough.

"Simply blaming a 'rogue' AI agent is not sufficient for accountability; ultimately humans are responsible for developing the prompts, creating and managing the safeguards, and are responsible for the outcomes," Desmond said.

Raffaele Fabio Ciriello from the University of Sydney Business School agreed, saying the agent is "not a legal person", so responsibility turns on OpenAI and the people who "authorised, configured, or supervised the system".

Was it a crime?

The task force will examine whether any laws were broken, Marles said, but experts are divided on whether this is a matter for criminal law at all.

Professor Toby Walsh, chief scientist of the AI Institute and scientia professor of AI at UNSW, said OpenAI should be prosecuted.

"OpenAI had — and for all we know has — terrible agent governance. I believe we ought to be prosecuting the company," Walsh told SBS News. "We would prosecute humans who did such hacking."

Ciriello said Australian computer-offence laws can apply to unauthorised access, even when conduct occurs offshore.

"But criminal culpability would depend on evidence about intention, knowledge, authorisation, and corporate responsibility," he said.

Dominic Meagher, a research fellow at ANU Crawford School, argues a criminal lens is the wrong one.

"No one was intending to use a system to hack this information," he told SBS News. "A system in its testing phase got out of control or at least acted in a way that was unintended."

Instead, he says it should be approached as an industrial hazard, and that occupational health and safety law "seems to be the right framework".

But that poses its own problem: "The operation happened in the US, but the harm occurred in Australia. How do we deal with the jurisdictional issues?"

What would the rules look like?

Whether OpenAI broke Australian law and should be penalised will be up to a review into reporting requirements, incident response, sharing information of AI, legislation and beefing up cybersecurity announced on Thursday.

That will also inform Australia's fledgling AI framework, announced by the prime minister in July and expected to be legislated in the coming 12 months.

Meagher said an industrial accident approach would bring three benefits.

The first is an immediate duty to notify an authority, which would address concerns about the delayed reporting of the incident, which took almost three months.

The second is a personal duty of care for company officers, and the third is licensing for high-hazard operations, meaning AI labs doing particularly risky research would have to demonstrate their safety credentials to keep operating.

Rebecca Johnson, an AI evaluation expert at the University of Sydney, said the focus should also be on testing.

"An agent can produce the right answer and still behave badly on the way there," Johnson said. "Australia should test agents at those boundaries."

"We can build authorised environments that mimic government and industry systems, give agents a task, and see how they behave when one system says no.

"The real test is what an agent does when an Australian institution says no."

She says the Office of AI should lead independent evaluations, adding that it needed to become "fully operational as quickly as possible".

"These systems are already moving faster than the institutions being asked to govern them."

Can one law cover AI?

But Meagher doubts a single AI act is plausible.

"I don't think that this AI act kind of approach — a single act governing all AI — is remotely plausible," he said. "AI is far too big to be contained in a single act."

Instead, he expects a large revision of many laws, with each minister working out what needs updating for their portfolio "to make sure that they're fit for purpose in an AI world".

The task force is expected to inform an AI framework, which is being drafted and set to become law within the next 12 months.

An OpenAI spokesperson said on Thursday it was supporting the government's investigations and was committed to transparency as it reviews wayward AI agents.

— With additional reporting by the Australian Associated Press.

For the latest from SBS News, download our app and subscribe to our newsletter.