Malnourished and showing signs of torture, the prisoners were found on Sunday when US troops took control of an interior ministry building.
The US raid, in the central Jadiriya district of Baghdad, followed repeated enquiries by the parents of a missing fifteen- year old boy.
Iraq's prime minister has promised to find those responsible for any abuse. He admitted that most of those held were Sunnis, and that they may have been "subjected to some kind of torture".
Deputy interior minister Hussein Kamal, who saw some of the abuse victims personally, said: "I've never seen such a situation like this during the past two years in Baghdad, this is the worst.
"I saw signs of physical abuse by brutal beating, one or two detainees were paralysed and some had their skin peeled off various parts of their bodies."
"I have never seen such a situation like this during the past two years in Baghdad. This is the worst and cannot be denied."
Tip of the iceberg
The Shia-dominated security forces have repeatedly been accused of abusing prisoners, particularly Sunnis but Sunday's discovery is the first hard evidence and officials fear it may be the tip of the iceberg.
Kamal blamed the situation on a lack of jail cells in Iraq.
But the BBC is reporting Iraqi suspicions that the building may also have been used as a base for a militia called the Badr Brigade- a group which may have infiltrated Iraq's security services.
Embarrassing
The leader of Iraq's largest Sunni political party said attempts to raise claims of torture in government detention centres had been dismissed.
"According to our knowledge, regrettably, all the detainees were Sunnis," Mohsen Abdul-Hamid, head of the Iraqi Islamic Party, told Associated Press.
"In order to search for a terrorist, they used to detain hundreds of innocent people and torture them brutally."
The discovery will be embarrassing for the US military, which has been training Iraq's security services.
Human rights group Amnesty International welcomed the investigation but said it should cover other allegations of torture and maltreatment by Iraqi police and security forces belonging to the interior ministry.
The security forces have faced repeated allegations of systematic abuse and torture of detainees, and of extra-judicial killings.
A report by pressure group Human Rights Watch earlier this year said methods used by Iraqi police included beating detainees with cables, hanging them from their wrists for long periods and giving electric shocks to sensitive parts of the body.
President Bush has said his administration doesn't condone torture.
But concerns that U.S. troops have tortured prisoners have dogged the Bush administration since April 2004, when graphic photographs of Army reservists mistreating prisoners at Iraq's Abu Ghraib prison outside Baghdad became public.
Lion claim
Meanwhile US Defence Secretary Donald Rumsfeld says a former Iraqi detainee's allegation that US troops threatened to put him in a cage with a lion was "farfetched".
Sherzad Khalid made the charge in television and newspaper interviews with another former Iraqi detainee, Thahee Sabbar, a fellow plaintiff in a lawsuit against Mr Rumsfeld and top US military commanders that was filed in March.
They said they were beaten, deprived of food and sleep, given electric shocks, shot with rubber bullets and subjected to mock executions after being detained by US soldiers in July, 2003.
Khalid told the US's ABC television network that US soldiers at one point took him to a cage with a live lion inside and threatened to throw him to the lion if he did not confess.
In an interview with the Washington Post, he recounted being shoved into a cage with the lion at one of the presidential palaces in Baghdad three times before being lined up for a mock execution.
"It seems quite farfetched," Mr Rumsfeld said. "Obviously, everything that everyone alleges is looked into."
Mr Rumsfeld said detainees were trained to lie about their treatment, citing a terrorist training manual found in the British city of Manchester.
Both former detainees were released after several months without any charges filed against them.