At the most benign end of the scale, even short-term exposure to mould caused cold and flu-like symptoms, Dr Black said.
It could also lead to infections in those with serious illnesses or compromised immune systems.
"The longer they get exposed to it, the sicker they get," Dr Black said.
For mould to develop, it requires a food source, such as paper, paint or clothing, and still air, Archicentre Australia spokesman Peter Georgiev said.
"Even minor signs of damp may indicate a far more serious underlying problem," the architecture advisory agency spokesman said.
Rising damp can cause increased room humidity, and result in infestations of cockroaches, silverfish and dust mites, as well as toxic mould growth, he added.
Dr Black said building code changes to ensure new dwellings were better ventilated would go a long way towards addressing the issue.
"It's becoming a bigger problem now than it has been in the past ... because people are living in more closed-up apartments or houses to get better energy ratings," Dr Black said.
"Through the big building boom, the big property boom, there have been a lot of places slapped together."
And the problem was often hard to spot and expensive to fix, especially if it was hidden in places such as wall cavities or ceiling spaces.
"Every house is potentially a mould factory," Dr Black said.
Mr Georgiev said a thorough cleaning with white vinegar can kill mould infestations, but a long-term solution can require expert treatment.