"During my last two ATP matches in Washington and Toronto, I did not feel 100 percent and after several consultations with doctors and my team, we have decided that I must withdraw from the Olympics," the 19-year-old said on Instagram.
"I hope to be back in full form for the ATP Masters 1000 in Cincinnati and wish my German team mates all the best in Rio."
Zverev, who lost the Halle final to compatriot Florian Mayer after beating 17-times Grand Slam champion Federer, is ranked 25th in the world and would have been seeded at the Olympics.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
The big-serving teenager lost in the semi-finals in Washington to Gael Monfils of France and the first round in Toronto last week to Taiwan's Lu Yen-Hsun.
Germany have Philipp Kohlschreiber, Dustin Brown and Jan-Lennard Struff entered in the Rio singles event.
(Reporting by Alan Baldwin, editing by Ed Osmond)
