“I am sorry we need to do this but at the end of the day we need to get on top of this,” Mr Gutwein told reporters. “We need to ensure that we can crush this virus at its source, and with this outbreak we need to take these steps.”
Forty-nine virus cases are linked to the outbreak at the hospitals, including 35 health workers.
Coronavirus fatalities in Australia increased by three to 59 people on Sunday with the death of a 74-year-old man in hospital in South Australia after contracting the virus on the Ruby Princess cruise ship, the death of a woman in her 70s in Tasmania and the death of an 82-year-old man in Sydney.
'No place in the world I would rather be'
Meanwhile, Australia has put itself in a "position of strength", but must keep up the pressure to beat the disease, Chief Medical Officer Brendan Murphy says.
"There is no place in the world I would rather be than Australia at the moment," he told reporters in Canberra on Sunday.
But he said people in the community were still transmitting the virus.
"That is why we have to keep our pressure on and make sure that we don't end up like countries in the world that you have all seen on the news."
"We are in a good place ... but we have to maintain that good place".
Treasurer Josh Frydenberg warned it would be "very dangerous and unrealistic" to remove social distancing restrictions too soon.
Mr Frydenberg said restrictions across Australia will stay in place "for as long as it takes".
"We have got to take the medical advice," he told ABC television.
"I think it is very dangerous and unrealistic to move ahead of medical advice that has served Australia well."
Prof Murphy said there were now 6,289 confirmed coronavirus cases in Australia.
South Australia, meanwhile, reported good news on Sunday, with no new virus cases recorded for the first time in more than a month.
People in Australia must stay at least 1.5 metres away from others and gatherings are limited to two people unless you are with your family or household.
If you believe you may have contracted the virus, call your doctor (don’t visit) or contact the national Coronavirus Health Information Hotline on 1800 020 080. If you are struggling to breathe or experiencing a medical emergency, call 000.
SBS is committed to informing Australia’s diverse communities about the latest COVID-19 developments. News and information is available in 63 languages at sbs.com.au/coronavirus.