The 22nd-ranked Marcus Ellis and Chris Langridge battled hard at the Riocentro but were ultimately outclassed by the powerful Chinese duo who kept them at arm's length in a 21-14 21-18 victory.
Both the Chinese will have a chance to clinch their second Olympic golds in Friday's final.
Fu won the doubles with Cai Yun four years ago, while Zhang clinched the mixed doubles title with Zhao Yunlei at London.
News that makes sense
Your trusted source for staying up-to-date with the world around you. Get free daily news updates and analysis, straight to your inbox.
There will be no repeat in Rio because Zhang, 26, and Zhao lost to Indonesia's Tontowi Ahmad and Liliyana Natsir in the semi-finals on Monday.
"The way I played in that (mixed doubles) match wasn't ideal," he told reporters.
"I was disappointed but it didn't really affect us today."
Fu and Zhang will face the fired-up Malaysian duo of Goh V. Shem and Tan Wee Kiong who played out of their skins to defeat Chinese pair Chai Biao and Hong Wei in the earlier semi-final.
Roared on by a huge contingent of Malaysian fans, the 12th-ranked pair were hugely impressive in their 21-18 12-21 21-17 win, which gives Malaysia another chance to win its first ever Olympic title.
Mixed doubles pairing Chan Peng Soon and Goh Liu Ying will have the first shot to grab the steamy Southeast Asian a long-awaited title on Wednesday when they contest the final against Indonesia's Tontowi Ahmad and Liliyana Natsir.
Ellis and Langridge will play Chai and Hong in the bronze playoff on Thursday and have a chance to grab Britain's first ever men's doubles medal.
"We've still got a great opportunity to get a bronze medal and that will be absolutely massive for badminton so it's something we're looking forward to," said Ellis.
(Editing by Clare Lovell and Neil Robinson)
