Miloon Kothari made the claim in a news conference at the end of a two-week tour of the country at the invitation of the government.
"I think that some of the conditions that I've seen are amongst the worst in the world both in terms of overcrowding, severe overcrowding, and in terms of lack of access to civic services," Mr Kothari said.
"We saw conditions where there were up to 30 people living in a two-bedroom house. There were people living in houses where there were gaps in the walls, where you could really feel dampness in the homes.
"We visited one community in the Alice Springs camps where people were living in tin shacks for the last 30 years and with no rights to their land and, of course, no services."
Mr Kothari, an architect by training, said housing design was not done in consultation with the indigenous people but was imposed on them.
In a series of preliminary recommendations to the government, Mr Kothari said a national response was needed to tackle the problems. He will report formally to the UN early next year.
Aborigines have lived in Australia for at least 40,000 years, but they are now a minority with a population expected to reach 470,000 this year out of a total of 20 million.
Many live in squalid Outback camps, where unemployment, alcohol dependency and lawlessness are rife. Non-indigenous Australians are enjoying an unprecedented economic boom and First World lifestyle.
A report compiled earlier this year by the charity Oxfam found that the health of Aboriginal Australians lagged far behind that of indigenous people in other developed Commonwealth nations.