Thousands of people live in the six buildings named Cook, Matavai, Turanga, Solander, Marton and Banks in Waterloo, just on the outskirts of Redfern.
The area itself has a deeply rooted Aboriginal history: it's the birthplace of the Black Power movement in Australia and the famous 'block'.
There's little doubt the government has a grand vision for Redfern and Waterloo, yet will that vision come at the cost of the communities who live there?
Wiradjuri woman Rae Johnston said the destruction of the towers isn't just about the removal of social housing, but the removal of a local sense of belonging.
"It's definitely a hub for Indigenous people, not only the Gadigal people of the region but from all over the country," she said.
"It's a place where you know you can find that sense of community that you'd find back home."
Minister for Social Housing Brad Hazzard said the government intends to mix public and private housing together in a bid to "change the demographic" of the area and provide "a very big downward push on anti-social behaviour".
Gentrification has been an ongoing issue for Aboriginal families in the Redfern and Waterloo areas who are struggling to stay there.
"The push for gentrification is so strong," said Johnston.
"The idea that we need to push out the low socio-economic people and the people in government housing I find pretty appalling and it's not changing it for the better, definitely not."
The government's commitment to social housing falls way short of the mark according to some experts, with 30,000 homes needed - minimum - but only 9,500 committed to over the course of 10 years.
The size of the Waterloo development means the process of moving the tenants will be one of the largest human-juggling exercises the NSW Government has ever undertaken.
The building begins in 2017 and is to be finalised by 2037.
