Mr Annan told the opening of a high-level general assembly meeting aimed at reviewing the 25-year struggle against AIDS that the shortcomings were deadly.
The plenary session convened a day after the publication of a major UN report that showing that the incidence of new HIV infections appears to have stabilised for the first time.
While agreeing that progress had been made in the wake of a landmark 2001 UN summit which laid down targets for halting and starting to reverse the AIDS epidemic Mr Annan warned that enormous problems remained.
"The vast majority of countries have fallen distressingly short of meeting their targets," he told the three-day general assembly session that gathered 10 heads of state and government, as well as some 80 cabinet ministers.
He said the world had also been slow in meeting one of the most vital aspects of the struggle measures to fight the spread of AIDS among women and girls. "These shortcomings are deadly," he added.
Following Mr Annan's remarks, Kehnsani Mavasa of South Africa became the first person living with HIV to address the general assembly as she urged conference delegates to adopt a final declaration with real meaning.
"Your big task now is making sure that this ... is not a document of empty promises, not a mere restatement of principle, but a target for platform-based action," she said.
Ms Mavasa's plea echoed the concerns of many HIV activists and action groups attending the UN meeting that too many governments are wary of making major promises and that the declaration to be adopted on Friday may be watered down by conservative member states.