In an address to world leaders ahead of the formal opening of the G20 summit, Mr Abbott cited his own government's difficulties with the budget as examples of the need to persist with important reforms.
He mentioned his government's success in axing the carbon tax, as well as its attempts to impose a $7 GP co-payment, deregulate university fees.
"It doesn't matter what spending program you look at. It doesn't matter how wasteful that spending program might appear, there are always some people in the community who vote, who love that program very much," Mr Abbott said.
"So getting the budget under the control has proven very difficult."
He also drew attention to the government's success in stopping asylum seeker boats, and said "thank God" that they did.
Opposition leader Bill Shorten labelled the comments "graceless and weird" and said it was a missed opportunity to promote Australia's position as a global leader.
"This was Tony Abbott's moment in front of the most important and influential leaders in the world and he's whingeing that Australians don't want his GP tax," Mr Shorten said in a statement to AAP.
The group attending included the US President Barack Obama, China's Xi Jinping and Russian President Vladmir Putin.
Mr Abbott was hosting the G20 leaders at a retreat and barbecue lunch before the summit kicks off on Saturday afternoon.
"Nowhere on earth will there be a more influential gathering than this," he told the group.
But he reminded leaders they bore heavy responsibility to achieve economic change at the summit.
"The world is looking to all of us right now to try to demonstrate to an uncertain and at times anxious world there are people who know what they are doing, that there are people who have a plan ... for growth and for jobs," Mr Abbott said.
"That's our challenge, to leave this G20 meeting in 48 hours' time having instilled more confidence in the people of the world that there is a better and brighter future for all of us."
He acknowledged there were limitations in what could be achieved, and that there would be inevitable disagreements among the leaders.
He urged them to speak candidly and openly at the summit.
"The only rules, as far as I'm concerned, are if we can speak from our heart rather than from a script, that would be good."
