Deputy Prime Minister Warren Truss says "relatively minor" changes will be made to the structure of the federal government.
The Abbott government was officially sworn in on Wednesday, 11 days after the Liberal-National coalition's election win.
The government's first decision was to sack three department secretaries, appoint two new secretaries and announce a restructure of departments and agencies.
Mr Truss told AAP, before going into a cabinet meeting at Parliament House on Wednesday, that the changes were aimed at making government organisations work more efficiently.
"We need to get on with and do that as quickly as we can," Mr Truss said.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"They are relatively minor (changes) and I think that is a demonstration that we want - wherever possible - for business to continue with capable people, but hopefully decision making being done in a more orderly and well-balanced way."
Mr Truss said it was an "honour and a heavy responsibility" to return to government after six years.
He said there was a "mood of quiet satisfaction" at Government House during the swearing-in ceremony.
"But also a real acknowledgement that we have a big task ahead of us," he said.
The government's priority would be preparing legislation for parliament, due to sit in late October or early November, and honouring election commitments.
