Rescue operations ended late on Friday after teams worked worked around the clock searching for people trapped in the rubble.
More than three million Muslims are in Mecca for the Hajj pilgrimage and people were praying in nearby streets when the building collapse occurred.
Interior ministry spokesman General Mansur al-Turki said the death toll had reached 76, with another 62 people wounded, and that recovery operations had now been completed. As many as 40 bodies remained unidentified.
Saudi authorities have so far refused to provide the nationalities of the victims. But survivors said some of the pilgrims staying in the hostel came from India, Libya, Pakistan or the United Arab Emirates.
Jordan said five of its nationals were missing while officials in France said seven Algerians living in France were among those who died.
Saudi authorities had previously said at least 53 people were killed in the incident, which took place during the mid-day bustle of a narrow market street.
The building was at least six storeys high and 30 years old.
Survivors recount horror
Survivors recounted the horror of the tragedy "I heard one big noise," said Algerian pilgrim Tayeb Mizasha, 70, as he lay in bed in Mecca's King Faisal hospital with broken ribs and a bruised face. "At first I thought it was an earthquake."
He said he was staying at the Luluat Al Kheir (Pearl of Grace) hostel with 16 other Algerians who had come from France to perform the hajj or annual pilgrimage to Mecca. "I do not know where my wife is," he said.
In another hospital ward, a Yemeni who worked in a clothing store on the ground floor of the building said four of his Yemeni co-workers lost their lives.
"I just found myself across the street from the building and I looked up and it was a pile of rubble," said Ali Qasim al Rimi, 35.
"I do not know if I fled or someone pulled me out," he said.
A group of pilgrims from Egypt said they were caught in a stampede on the crowded street in the Ghazzah neighbourhood where the accident happened some 200 meters north of the Grand Mosque, the most sacred site in Mecca.
"We were walking back from noon prayers and suddenly debris started falling down on us," said Rajab al Sayed, 46.
"We ran for our lives and two of my friends were taken down by the crowd," he said.