Representatives from Bujumbura and the National Liberation Forces (FNL) gathered in the commercial capital Dar es Salaam for the South African-mediated negotiations that began even as hostilities have continued.
"You can choose to heed the folly of triumphalism and drive your country to ruins," Tanzanian President Jakaya Kikwete told the two sides as the talks got under way at a Dar es Salaam hotel.
"You can choose to sound the death knell for dialogue and let the guns usurp the place of reason.
"Or, you can choose to honour the toil of Julius Nyerere and Nelson
Mandela, who worked hard for peace in Burundi by agreeing to negotiate earnestly and conclude a peace deal," President Kikwete said.
Mr Nyerere, the late president of Tanzania, and Mr Mandela, the former South African president, had been at the forefront of efforts to bring an end to Burundi's war, which erupted in 1993 and has since claimed about 300,000 lives.
Their efforts resulted in a 2000 peace agreement that all parties but the FNL, one of Burundi's seven Hutu rebel groups, have now signed.
Officials said the talks in Tanzania are aimed at bringing the FNL into the peace process, disarming it, and re-integrating the fighters in the tiny central African nation's society.
Power sharing government
FNL leader Agathon Rwasa said his forces are not "warmongers" and ready for peace but only in an "inclusive society" with "no discrimination. We recognise the government and know that it is a legitimate one and was elected," he said.
"But our problems are with the state organs, the army and police, which continue to terrorise the people, detain Hutus, particularly FNL members who are tortured and in some cases killed."
"We want to recognised as a political party and be allowed to engage in political activities like any other party Burundi," he said.
Despite the adoption of a new constitution and a series of elections last year that saw the formation in August of a new power-sharing government headed by a former Hutu rebel leader, the FNL has remained outside the process.
Its fighters continue to wage a low-level insurgency in areas surrounding Bujumbura and just last week the government launched a major offensive against the guerrillas in retaliation for previous attacks.
Burundi's Interior Minister Evariste Ndayishimiye said the government was making a good faith gesture by agreeing to the negotiations.
"We are here to talk peace and if the FNL is going to co-operate we will come up with a good deal even today or tomorrow," he said. "What we want is seriousness of purpose and commitment to a peaceful future in Burundi."
The talks had been expected to begin in Tanzania with Tanzanian mediation in mid-April but were inexplicably delayed several times as the Burundian government negotiating team never left Bujumbura.