"I am very sad because after Beijing and London I would have liked to experience my third Olympic Games in Brazil, unfortunately that will now not be possible," he said in a Swiss Tennis Federation statement on Tuesday.
Australia's Sam Groth is heading to Brazil after getting a late call as a replacement.
The fast-serving 28-year-old will be the fourth Australian in a men's singles draw missing the country's top two players after the controversial withdrawals of Bernard Tomic and Nick Kyrgios.
The duo pulled out after being repeatedly criticised by Australia's Olympic team chef de mission for their conduct in the public arena.
World number 20 Tomic said he was too "busy" to compete at Rio while 18th-ranked Kyrgios bowed out claiming mistreatment by the Australian Olympic Committee.
(Writing by Alan Baldwin, editing by Tony Jimenez)
