CFMEU state secretary John Setka confirmed the building, on Elizabeth Street in Melbourne's CBD, is now a tier-one exposure site, forcing union staff and officials into isolation for two weeks.
Four positive cases have been linked to the office so far.
"This outbreak caused by the disgusting behaviour of selfish and reckless people with no regard to the wellbeing of the thousands of construction workers or their families will not deter our commitment to getting construction back open and all our members back to work," Mr Setka said in a statement on Wednesday.
So far the building has not appeared on the Victorian government's COVID-19 exposure sites website list, but not all exposure sites are published online.
A tier-one exposure site indicates someone on the premises has tested positive for COVID-19. Anyone who has visited the site must get a COVID-19 test and self-isolate for 14 days from the date of exposure.
The outbreak comes a week after violent protests were held outside the building over mandatory COVID-19 vaccinations and other restrictions for the construction industry.
Acting Chief Health Officer Ben Cowie said negotiations are continuing with the construction sector on when it can emerge from a temporary shutdown, adding that the news of tier-one exposure site is concerning.
"In any circumstance where there has been transmission of COVID-19 that has been a concern for us - both from the perspective of the individual, but also the flow-on effects from public health settings."
Active COVID-19 cases increase to 9,890
The latest deaths include five people from Hume; two women and a man in their 80s, a woman in her 70s and a woman in her 50s, and two women from Whittlesea in their 80s and 90s.
The new infections bring the number of active cases in the state to 9,890 and the death toll from the current outbreak to 36.
There are 371 people in hospital with COVID-19, with 81 of those in intensive care and 55 on a ventilator.
Deputy Secretary of Victoria's health department Kate Matson said contact tracers are changing the way they work to deal with the high volume of cases and close contacts.
"We need to focus on the actions of highest value to ensure that people who are positive know that quickly, and connect them to the support they need," she told reporters.
"In areas with higher case numbers, we have started providing a text message with the confirmation of your confirmed positive result, so we can provide triage."
The text message will ask a number of questions to help contact tracers prioritise higher-risk cases, including whether the person is looking after people at home, if they need support and if they are vaccinated.
A short phone interview may occur depending on the results from the text.
Contract tracers will interview household contacts but may not interview "every single primary close contact," Ms Matson said.
"Should case numbers increase further, although it is in our collective will and efforts to contain them, we will adapt the approach again and again and refine it based on risk," she said.
Apology over short notice for Latrobe Valley region lockdown
It comes after the Latrobe Valley region was plunged into a seven-day lockdown to curb a growing COVID-19 outbreak believed to be linked to a household gathering.
Residents in the City of Latrobe, which encompasses the Gippsland towns of Moe, Morwell and Traralgon, entered lockdown at 11.59pm on Tuesday. They were notified via a press release about 6.30pm.
They are now living under the same restrictions as those in Melbourne, with the exception of the nightly curfew.
Health Minister Martin Foley said the situation in that area had "deteriorated quite rapidly" on Tuesday and apologised for giving residents short notice.
"These are strong measures to keep the community as safe as possible were required," he told reporters.
There are at least 18 active COVID-19 cases in the region.
Almost 79 per cent of Victorians aged 16 and over have received one dose of a COVID-19 vaccine, while more than 48 per cent are fully vaccinated.
A modest easing of rules has come into effect on Wednesday including extending metropolitan Melbourne's travel limit to 15 kilometres and the resumption of contactless sports.
An updated list of COVID-19 exposure sites can be found here.
With SBS News