Treasurer Chris Bowen says it's clear the coalition has won the election.
"I think we can call the election," he told the Seven Network.
He said it was clear, with Liberal wins in seats like Bass in Tasmania, that the coalition had won.
Mr Bowen is the most senior Rudd government minister to concede defeat, although a long line of Labor figures had said there was no chance of a win not long after polls closed at 6pm.
Prime Minister Kevin Rudd has left his Brisbane home to attend the Labor party function at the Gabba, where he is expected to formally concede
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
Retiring defence minister Stephen Smith was the first to admit Labor would lose at 6.05pm - minutes after polling booths closed in eastern Australia.
"The government will be defeated tonight," Mr Smith, who is retiring at this election, told ABC television on Saturday.
Health Minister Tanya Plibersek reluctantly conceded.
"I think it's pretty clear, it's a matter of the size of the victory," she told ABC TV, adding that she thinks the party will lose nine seats in NSW.
Howard government treasurer Peter Costello said election had been won.
"The election is over, the Coalition has won" he told the Nine Network.
"It is going to be large majority. The question is how big that majority is going to be."
Earlier, Deputy Prime Minister Anthony Albanese says voters have punished Labor for disunity.
"You do get marked down when there's a perception or reality of disunity and no doubt that's one of the issues that's come through in terms of the election campaign itself," he told the Seven network from his Sydney electorate.
Asked if he believed Labor had done better with Kevin Rudd, rather than sticking with Julia Gillard, he was non-committal.
"I think Kevin Rudd and Julia Gillard have been good prime ministers," he said.
"All the polling indicated where we are at and we made that decision."
He said Labor's internal problems were playing out in the election results.
"The Labor party did spend too much time talking ourselves down whether on behalf of Kevin Rudd or Julia Gillard," Mr Albanese said.
"The electorate clearly has view that we need to be united. They want to vote for a united party.
"What is important is that if the result doesn't go the way I would like tonight, we can't walk away from our legacy."
