The COC said Brassard stepped down for "business and personal reasons" and named Olympic cycling medallist Curt Harnett as his replacement for the Aug. 5-21 Rio Games.
Brassard, who was named to the position in late 2014, said he did not feel he could help Canadian athletes to the extent that he would like.
"Since becoming chef de mission, my professional obligations and other responsibilities have made it very challenging for me to manage time-wise," Brassard, who was an assistant chef de mission at the 2014 Sochi Olympics, said in a statement.
"After much reflection, I have come to the difficult decision, and in the best interest of Team Canada, to step down as chef de mission."
Brassard competed in four Olympics and captured gold in 1994 in Lillehammer to become Canada's first Olympic champion in freestyle skiing.
Earlier this year Brassard expressed concern with how the COC handled the scandal involving former president Marcel Aubut, who resigned following allegations he sexually harassed a female colleague.
The COC, one of the most powerful sports organizations in Canada with influence over many Olympians, their coaches and national sport federations, ultimately unveiled sweeping changes to workplace policies.
"If I have the impression I can't do anything for the employees who are still hurt by this, I would consider that a failure, and I might have to distance myself," Brassard said during a February interview with CBC's French-language service, Radio-Canada.
Four-times Olympian Harnett is Canada's most successful international cyclist and served as Canada's chef de mission, supporting athletes in a leadership capacity, at last year's Pan American Games in Toronto.
"I look forward to leading our incredible Canadian Olympic Team and working closely with the Canadian Olympic Committee to ensure our athletes have everything that they need to compete and win against the best in the world," said Harnett, a three-times Olympic medallist.
(Reporting by Frank Pingue, editing by Ed Osmond)
