Speaking to reporters on his return from a Middle East tour, Mr Annan said his efforts along with those of some governments to persuade Khartoum to show flexibility had not been successful.
Warning that the situation in strife-torn Darfur was "serious, desperate," the secretary general said the Sudanese leadership "may be held collectively and individually responsible for what happened to the population of Darfur."
The UN refugee agency (UNHCR) meanwhile highlighted fears of a major military offensive, saying thousands of Sudanese troops have been deployed to Darfur in recent weeks.
The Sudanese government said on Monday it would ask African Union (AU) peacekeepers to leave Darfur by the end of the month if the AU confirmed that it was no longer able to continue its peacekeeping mission.
Khartoum is not allowing the beleaguered AU mission to be turned over to the United Nations.
The announcement was a fresh act of defiance following the adoption August 31 of a UN Security Council resolution calling for the deployment of up to 20,000 troops and police to replace the ill-equipped cash-strapped AU forces in Darfur.
"The international community is not coming (to Darfur) as an invading force but basically to help them protect the people," Mr Annan said.
He added that Darfur would be discussed at a UN Security Council meeting on Monday.
Destabilisation fears
Earlier UN refugee chief Antonio Guterres warned that a major catastrophe was brewing in Darfur, which could destabilise the whole region.
"If things don't improve, we're heading for a major catastrophe," said Mr Guterres, the UN High Commissioner for Refugees, in a statement underlining the impact of worsening violence in Darfur.
Mr Guterres underlined the threat of a further massive displacement to add to some 2.2 million people who have already fled their homes, including 200,000 refugees in neighbouring Chad.
"Millions of people are already at grave risk. Hundreds are still dying amid ongoing violence, and thousands are still being forcibly displaced," he added.
The combined effect of war and famine in Darfur has left up to 300,000 people dead after three and half years of civil war pitting the Sudanese government and allied militias against ethnic minority rebels.