The demonstration came as Iraqis celebrated the Muslim Eid Al-Adha holiday a day after a twin suicide bombing at the interior ministry, claimed by al-Qaeda, which left at least 28 policemen dead.
Waving banners and chanting anti-US slogans, about 700 people rallied in the gardens of the Umm al-Qura mosque in west Baghdad to denounce the Saturday night raid.
"The attack on the Umm al-Qura mosque is an attack on Muslims and Islam," read one of the banners at the protest.
The US military said the raid was linked to the hunt for kidnapped Jill Carroll, a freelance journalist working for the Boston-based Christian Science Monitor.
Lieutenant Colonel Barry Johnson, a US military spokesman, said the raid was ordered "as a direct result of a tip by an Iraqi civilian that activities related to the kidnapping were being carried out inside the mosque."
The 28-year-old reporter was seized by gunmen on Saturday after calling by the office of a prominent Sunni politician in the neighbourhood.
Her interpreter, Allan Enwiyah, 32, was shot dead and his body abandoned nearby, while her driver got away.
"Both Iraqi and coalition forces raided the mosque in the early morning hours in order to minimize the impact on worshippers and the surrounding neighbourhood," Lt Col Johnson said.
Six people were detained for questioning, he added.
The Sunni Committee of Muslim Scholars which is based at the mosque, confirmed that one of its members, Yunis Aikali, and five mosque guards were arrested in the raid.
In a statement, the committee also accused US soldiers of desecrating the mosque and carrying away files containing the names of members.
"We call on the occupiers to withdraw from Iraq because they are the reason for every crime and the death of every innocent in Iraq," Harith al-Aubaidi, a member of the committee said in his sermon at the prayers ahead of the protest.
Ashraf Qazi, the UN special representative in Iraq, also deplored the raid.
"This again underlines the importance of all parties respecting the sanctity of holy sites and places of worship," he said in a statement.
Hostage sought
Iraqi security forces, meanwhile, were on alert in western Baghdad looking for hostages, security sources said.
The US embassy said it had nothing new to report on Ms Carroll.
The Christian Science Monitor said it was urgently seeking information about its reporter after confirming her abduction on Monday.
Ms Carroll's driver, quoted in a story posted on the Monitor's website, said gunmen jumped in front of the car, pulled him from it, and drove off with their two captives all within 15 seconds.
Several Westerners are currently being held hostage by insurgents in Iraq, including an American, a Briton and two Canadians who are members of a Christian peace group.
Ms Carroll was the 31st media worker to have been kidnapped in Iraq since the start of the war in 2003, according to watchdog group Reporters without Borders.
Five of the journalist kidnap victims, four Iraqis and Enzo Baldoni of Italy, were killed by their abductors. The others were released.