The 52-year-old, sacked by AS Roma at the start of this year, will replace Franck Passi who had been in charge on an interim basis since Michel was fired in April.
The nine-times French champions are languishing in 12th place in the Ligue 1 table with three wins, three draws and three defeats from nine games.
"When I met Rudi, the first thing he told me was that he wanted to win the Champions League," said club owner Frank McCourt. "I immediately knew that he would be the right man to coach Olympique Marseille. Rudi is a man with great character, with a lot of energy."
Garcia, a former midfielder, has previously coached St Etienne, Dijon, Le Mans and Lille, winning Ligue 1 with the latter in 2010-11.
He led Roma to two second place finishes in Serie A in as many full seasons in charge, although they never really challenged the dominance of Juventus.
His two-and-a-half year tenure ended with a poor run during which they earned only one win in 10 matches.
(Writing by Brian Homewood, editing by Pritha Sarkar)
