"We are in a danger phase because only a month ago we stopped a steel-hulled vessel with 131 people coming out of Sri Lanka, there are 14,000 people still in Indonesia and there is excited chatter among people-smuggling syndicates about the prospect of Australia being available again," Mr Dutton said.
"It's essential that people realise that the hard-won success of the last few years could be undone overnight by a single act of compassion in bringing 20 people from Manus to Australia.
"The boats are there, we are scuttling boats, we are returning people and we are turning around boats where it is safe to do so. The boats haven't gone away and if there is a success defined by an arrival of a boat in Australia then the word will spread like wildfire."
It's a view echoed by Prime Minister Malcolm Turnbull, who told the Nine Network on Friday: "Here in Australia we have one of the most generous refugee and humanitarian programs in the world.
"The reason we can do that is because we decide, the Australian government decides, representing the Australian people, who comes to Australia; not people-smugglers."
The row over refugees has been reignited by US President Donald Trump's decision to reverse a decision that separated children from their illegal migrant parents at the border and as Europe faces a continuing swell of people crossing the Mediterranean with the help of people smugglers.