During the ritual, worshipers hurl seven pebbles at three 18 metre pillars that symbolise the powers of evil and to affirm their allegiance to God.
It’s the riskiest ritual of the Hajj as crowds of pilgrims jostle to make sure their pebbles touch the main pillar.
Thousands of Saudi policemen and security force members surrounded the area to prevent a repeat of another tragedy like the stampedes in 2003 that killed 251 pilgrims.
Another stampede in 1990 killed 1,426 people.
The major hotspot is a wide overpass into the site known as Jamarat bridge.
Helicopters hovered overhead and a dozen ambulances with wailing sirens milled through the crowds traversing the crossing.
Saudi authorities pledged major improvements and extra bridges after the 2003 stampede but as yet none have been implemented.
However this year the flow of pilgrims on the bridge appeared to be organised.
Fearing overcrowding Saudi police tried in vain to remove the thousands of pilgrims camped out under the bridge and on the stairs leading into the valley.
Indonesian pilgrims die
At least 120 Indonesians have died, mostly of illness, in this year's Hajj pilgrimage alone.
According to Indonesia's Antara state news agency, 11 nationals died while performing the central rite.
More than 200,000 Indonesians are registered to take part in this year's Hajj.
Feast of Sacrifice
Earlier on Tuesday, hundreds of thousands of worshippers swarmed into the Grand Mosque and its plazas after sunrise to take part in a special prayer marking the start of Eid al-Adha, the feast of the sacrifice.
Each pilgrim must sacrifice an animal, usually a sheep, which commemorates the story of Prophet Abraham when God as a test of his faith commanded him to slaughter his son Ismail.
"We are on God's way," said Mohammed Jallu, 43, from Guinea in west Africa.
The hajj is one of the five pillars of Islam and a once-in-a-lifetime duty for those able to complete it.
Islam is not terrorism: Cleric
A top Saudi cleric has told Muslim pilgrims gathered for the Haj that the west has used terrorism to scare people away from Islam and discredit legitimate Muslim causes.
Sheikh Abdulrahman al-Sudeis made the comments at the Grand Mosque in front of 2.5 million pilgrims in a sermon to mark the feast of Eid al-Adha.
“The campaign against Islam has become fierce and Muslims are being described in insulting terms to distort the image of Islam and scare people away from it," the state-appointed preacher said.
The sheikh also called for stability in Iraq and said Islam is innocent of the charge of terrorism.