Sri Lankan couple Priya and Nadesalingam and their Australian-born daughters Kopika, 4, and Tharunicaa, 2, are on Christmas Island, where they will remain until 18 September, when the Federal Court in Melbourne will reassess Tharunicaa's case to remain in Australia.
Mr Dutton has taken aim at the ongoing financial toll, arguing the efforts to deport the Tamil family had cost taxpayers "literally millions of dollars".
“It’s been very clear to them at every turn that they were not going to stay in Australia, and they still had children. We see that overseas in other countries, anchor babies...the emotion of trying to leverage a migration outcome based on the children," he told Macquarie Radio.
The discussion drew immediate comparisons to US President Donald Trump, who has deployed the 'anchor baby' term to justify his moves to end birthright citizenship in America.
So what does it mean exactly?
The term 'anchor baby' is still most commonly used in the United States and refers to a child who is still in the womb when brought into the country by a foreigner, with the express intent of securing US citizenship for the child upon their birth.
The expression is widely considered to be an offensive term for immigrants.
While the phrase returned to the forefront of America's immigration debate in the lead up to the 2016 presidential election, its roots date back more than three decades.
The alternative 'anchor children' can be traced back to the 1980s and was used to describe adolescents who fled conflict or financial difficulty in Asian nations such as China and Vietnam.
The concept surrounded these children acting as a metaphorical anchor in the United States for a family who would one day immigrate to America when the 'anchor child' had worked long enough to afford to sponsor their new life in the US.
In 2014, CNN presenter Chris Cuomo used the term "anchor babies" during a report on then-US President Barack Obama’s planned immigration overhaul. The reporter faced backlash from viewers, who said the expression was highly offensive. Cuomo later apologized, conceding he “shouldn’t have” used it, labelling the term “ugly” and “offensive.”
During the campaign to secure the Republican nomination for president, then-candidate Donald Trump was criticised for referring to rival Ted Cruz as an “anchor baby".
Mr Trump hit out at his fellow candidate due to the fact his mother was born in Canada and his Cuban father was a lawful resident of Canada.
Is it appropriate in Australia?
While debate rages in the United States over birthright citizenship, Australia does not have similar protocols for children born in the country to foreign nationals.
University of Technology's Faculty of Law lecturer Dr Anthea Vogl said the home affairs minister is markedly out of step with the community's view.
"I think (Peter) Dutton's comments really reflect a kind of desperate attempt by the government to justify the way they've treated this family and I don't think they're particularly effective," she told SBS News.
"They're legally unfound, the idea of anchor babies just doesn't work under Australian citizenship law."
Dr Vogl argued the case and the government's language reflects a broader view of those seeking asylum.
"Politically they're a really cynical attempt, to suggest that a family is having children in order to secure a migration outcome...it shows a really high level of contempt from this government vis-a-vis asylum seekers."
Last week, Opposition Home Affairs spokesperson Kristina Keneally questioned the direction of debate around the Bileola Tamil family, cautioning the discourse could stray into "anchor baby" territory.
"This is an importation, quite frankly, of an American debate about so-called 'anchor babies' and the law is very different in the United States where citizenship is accorded to anybody born on American soil," she told the ABC.
"That is not the law in Australia so it's an importation of that debate."
SBS News contacted the office of the home affairs minister and was told the minister would be offering "no comment" on the matter.