Prime Minister Tony Abbott wants to follow the approach New Zealand took to fixing its budget problems by saying "no" to unwarranted spending proposals.
Mr Abbott believes his government's job is to make the economy more productive by bringing down taxes and reducing regulation.
But that means bringing government spending under control.
Mr Abbott noted that across the Tasman, John Key's government was on track to reduce government spending from 35 per cent of GDP to 30 per cent.
"They didn't do that through savage cuts," he told ABC Radio on Tuesday.
"They did that through careful management, basically through saying no to unwarranted new spending proposals."
In Australia, government spending is about 26 per cent of GDP.
Mr Abbott reiterated his approach would be to fund new spending through sensible savings to existing spending.
Everyone needed to live within their means, whether it be a company, a family, an individual or the government itself, he said.
The prime minister said there were about $20 billion of savings being held up in the Senate by Labor and the Greens.
The measures included $5 billion of savings Labor proposed before the September election.
