Following weekend riots, Israel security forces dragged some supporters of the settlers out of a market in the centre of Hebron on Tuesday but dozens of others still remain and the settlers themselves show little sign of moving.
Mr Olmert expressed fears that the culture of defying the law was in danger of spreading as he held consultations with security officials and his new Foreign Minister Tzipi Livni, who is also justice minister, Israeli radio said.
Mr Olmert was quoted as saying that the settlers' failure to accept the rule of law could spread if it was not nipped in the bud. "We are fighting for the rule of law. It is an essential objective."
Army chief Dan Halutz said that the response of the security forces had so far been too feeble.
"We are paying the price for this softness in the face of people who do not even recognise the authority of the state of Israel," he said.
The authorities insist they are determined to remove nine Jewish families, around 50 people, squatting illegally in the fruit and vegetable market.
However hardliners have vowed to push ahead with the resistance to any planned evictions from what they regard as a sacred part of the land of Israel.
In 2003, the Supreme Court backed an appeal by Palestinian traders, ordered the settlers to be evicted and the market reopened, but neither of the two court orders has yet been implemented.
The market was closed 12 years ago after Baruch Goldstein, an extremist settler, shot dead 29 Palestinians praying in Hebron's Cave of the Patriarchs, a shrine holy to both Jews and Muslims.
Under a 1997 accord with the Palestinian Authority, Israeli troops evacuated 80 percent of the city but continue to protect some 600 settlers living around the Cave of the Patriarchs.