The Palmer United Party senators sided with Labor and the Greens, as well as Ricky Muir, to vote it down 37-35.
An earlier Coalition motion to deem the bill as urgent was passed in the upper house this morning, pushing a debate on the bill today.
The debate was set to finish at 11.50am, after which a final vote was held.
Ahead of the vote, Clive Palmer told reporters that his senators would not vote with the government amid continued debate over the amendments.
Mr Palmer said the government had "pulled a swifty" on his party after they failed to circulate their revised amendment to the repeal legislation.
He said the amendment had been met with a "violent reaction" from the government, prompting a withdrawal of support form the party's senators.
"They resolved that they were going to vote against the repeal of the carbon tax today," he said.
The revised amendment in question can be read below.
Original amendments were withdrawn by Senator Glenn Lazarus.
Speaking this morning, Prime Minister Tony Abbott declared that "today should be the day when the carbon tax is finally scrapped".
Environment Minister Greg Hunt said the legislation will be introduced to the House of Representatives on Monday.
During a press conference with the government's Leader in the Senate, Eric Abetz, Mr Hunt said the Coalition shared the "same goals" the Palmer United Party.
Senator Abetz also labelled the failure of the carbon tax repeal as a "technical issue".
Greens leader Christine Milne welcomed the retention of the carbon tax, but said an explanation on the proposed amendments was necessary.
Senator Milne accused the Abbott Government of undermining business certainty by agreeing to the changes.
"These amendments seem to have a wide ranging regulatory function," she said.
"... Virtually anyone can be caught up."
'What remarkable chaos'
The developments in the Senate prompted confusion among those present, as well as commentators on social media.
Greens Senator Scott Ludlum was among those weighing in online.
