Cavendish claimed his 30th Tour de France stage victory and fourth in this year's race in Saturday's 14th stage but he will not now be starting stage 17 on Wednesday.
The 'Manx Missile', who has been back in the limelight after three below-par Tours, enjoyed wins on stages 1, 3, 6 and 14 and is now four shy of the record held by Belgian great Eddy Merckx.
The 31-year-old Cavendish, who had not won more than three stages on the Tour since 2011, attributed his success in this year's race partly to his track preparation for the Olympics.
"After an extremely enjoyable and successful couple of weeks at The Tour de France with Team Dimension Data, it is with great sadness that I took the decision today to leave the race," Cavendish said in the South African-based team's statement.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
"After the heat and intensity of the previous stages, we analysed my fatigue levels and decided I’m at a point that would have a detrimental effect on my other big goal for the year, the Olympic Games," he added.
As well as enjoying four stage wins, Cavendish also wore the overall leader's yellow jersey for the first time and carried the green jersey for the points classification into the first rest day in Andorra.
"Mark is really sad to leave the tour, we are committed to support him in his dream goal of receiving a medal for Britain at the Rio Olympic Games," said team principal Douglas Ryder.
The Olympics starts on Aug. 5.
(Editing by Ken Ferris and Toby Davis)
