Centrelink overpaid $2.84 billion to welfare recipients last financial year but the Turnbull government only recouped half the cash.
The new Human Services Minister Michael Keenan blamed human error and fraud for the overpayments.
"It is important that system operates with integrity, that people are not overpaid more than they are entitled to," he told reporters in Perth.
"If you have been defrauding the system, you can expect us to come down on you like a tonne of bricks."
The Human Services department has only recovered $1.64 billion in overpayments.
Labor frontbencher Doug Cameron said the Turnbull government was incompetent when it came to running Centrelink and there had been a merry go round of four ministers in four years.
"Last year 55 million calls went unanswered to Centrelink and average call wait times have increased," he said adding extra staff were needed.
Last year the Turnbull government coped criticism for mistakenly sending robo-debt recovery notices to 20,000 people.
