Some lit candles or released white balloons as they passed the shop where the 23 year old worked, while others sang the French national anthem or chanted Jewish prayers.
Ilan Halimi, a cell phone salesman, was kidnapped in January tortured for three weeks in the southern Paris suburb of Bagneux.
He was found naked, handcuffed and covered with burn marks on February 13 near railroad tracks and died on his way to hospital.
"Ilan tortured. France wounded," read one of the banners carried by marchers. Mr Halimi’s family was notably absent from the march.
Police said about 33,000 people, including politicians from all sides of parliament, took part in the march, organisers had put the figure between 90,000 people and two hundred thousand.
They made their way from the Place de la Republique to La Place de la Nation, in eastern Paris, in the chilling cold.
"Today, we must march, we must stand up, to say that in France each of us has the right to live in dignity whatever his God, his religion, the colour of his skin," said Foreign Minister Philippe Douste-Blazy.
One man not welcome and ejected by the crowd was right wing politician Philippe de Villiers whose Movement for France blames immigration for France's social ills.
The extreme right National Front was also banned by the organisers from the demonstration and did not turn up.
Roger Cukierman, the head of the Representative Council of Jewish Institutions in France which organised the marches with two left-wing anti-racism groups, said, "It's important for French society to realise that little anti-Semite and racist prejudices can have terrible consequences."
Smaller unity marches involving up to 2,000 people took place in other French cities, including Strasbourg, Lyon, Marseille and Bordeaux.
Worldwide support
Across the English Channel in London a crowd of about 50 sympathisers gathered in front of the French embassy to remember the victim, in what ambassador Gerard Erreta said was "a show of solidarity and vigilance," in the face of anti-semitism.
In Jerusalem several hundred Israelis of French origin demonstrated in solidarity with French Jews following the murder.
"The martyrdom of Ilan reminds us that anti-semitism still kills 60 years after Auschwitz," Rabbi Jacques Gruenwald told the gathering.
Israeli Foreign Minister Tzipi Livni said France and Israel must battle anti-Semitism together. "We are all shocked" at the killing of Halimi, she said.
Torture motive
It is not clear if racism or anti-Semitism was the motive for the Mr Halimi’s grisly treatment and killing, which may have been part of a suburban extortion racket.
Allegedly kidnapped by a suburban gang, his abductors sent ransom demands to his family, reportedly believing him to be rich because he was Jewish.
The gang's alleged ringleader, identified by prosecutors as Youssef Fofana, 25, was arrested in Ivory Coast and France has requested his extradition.
Mr Fofana has allegedly denied that anti Semitism played a part in the abduction.
On Saturday, two women and a male suspect were placed under formal, criminal investigation as a precursor to being charged on kidnapping and illegal detention counts.
The gang has reportedly tried kidnap six other people since December - four of them Jewish.
Other attempts
The gang also allegedly tried to extort money from several prominent figures, including a founder of Medecins Sans Frontieres (Doctors Without Borders), Rony Brauman.
Mr Brauman, who is of Jewish origin, received a demand for money in 2004 in a letter containing a photograph of hooded armed men, taken in front of his home.
Months later, Molotov cocktails exploded in his courtyard and a gunshot hit his door, he told LCI television yesterday.
Mr Brauman told LCI that he believed money and not anti-Semitism was a factor in his case.
The Halimi case has revived fears that anti-Semitism remains in French society.
The French government has been wary about drawing too heavy a link between the criminal gang responsible for Mr Halimi's murder and anti-Jewish sentiment.
Past incidents in which apparently anti-Semitic crimes turned out to be staged or committed for other motives seemed to lie behind its cautious stance.
Government spokesman, Jean-Frantois Cope, told French radio that while there were "strong suspicions" of anti-Semitic motives in "this horrible affair", investigators were still getting to the bottom of the case.
France has Europe's largest Jewish community, as well as the largest Muslim community in Western Europe.