All up, around 37 people have been impacted from the seven positive cases, but the union has told SBS News it fears the numbers are growing.
Construction workers, anti-lockdown and anti-vaccination protesters, and members of the far-right were among those who attended the violent rallies.
John Setka, Victorian secretary of the CFMEU, described the new cases as a "tragedy".
“Due to the actions of these reckless and selfish protesters, many of these members’ families who have been infected are very sick with the Delta virus,” he said in a statement.
“These protesters are just selfish idiots with absolutely no care for anyone other than themselves. They have caused enormous stress and heartache for members families’ who were just doing their job on the day of the protest.”
Construction workers were allowed to return to worksites on Tuesday after a two-week shutdown in Melbourne and other locked-down areas.
“While we welcome construction opening back up to 25 per cent today, these members won’t be going back to work and along with their families will be in quarantine for two weeks with the added stress of so many family members being very sick and some hospitalised," Mr Setka said.
The CFMEU office was listed as a tier-one exposure site last week.
About 500 protesters threw bottles at Mr Setka and smashed the office's door down on 20 September.
Victoria on Tuesday reported 1,763 new local cases of COVID-19, the highest daily tally of any Australian state or territory since the pandemic began.