Local brands are competing against a wave of global entrants into the Australian market.
While some smaller operators have turned to the internet to weather the storm, others are warning of a loss of community if consumers don't shop small.
Nick Brown has been in the fashion retail business for 25 years making and selling his own wears.
He started with a market stall at Sydney's Paddington Markets, before moving into his Surry Hills premises in 1993.
Business started off well, but declined after the global financial crisis.
"I'm getting by because I own the shop," Mr Brown told SBS World News.
"If I had to pay market rent for this shop I don't think it would be worthwhile at all and it's so unpredictable that it's so soul destroying, because you want to be a little bit more creative but you're just a little bit averse to taking risks."
Owning his store helps to keep costs down, but consumers aren't spending.
"People don't walk in. It's one thing if people walk in and don't like what they see, you just don't see people walking in."
Nearby Michal Nichols recently moved into a larger premises for his men's activewear label Teamm8 with a greater window frontage.
"We kind of got a little lost in the other space, it was great for six years but this really gives us a whole different level of exposure, and foot-traffic basically."
The higher rent is offset by his growing online business which forms 50 per cent of sales.
"Online has had big growth for us this past year, about 30 per cent. I think that's because we're getting more territories, look basically the US is our biggest market, then Europe, mainly UK, Germany, Spain and France", Mr Nichols said.
After starting four years ago and turning his first profit last year, Paul Zack who owns another activewear brand, BCNU, is returning to his original plan of selling online only.
"It actually helps us reduce our operational costs by 30 per cent, which is money we can reinvest back into product development," he said.
"I guess the biggest benefit for us as well is that we move away from doing two big production or two big collection launches a year, so we used to follow the spring summer, spring autumn seasons because we were entertaining a predominately wholesale retail business."
There was unforeseen demand from retailers to stock his product around the world.
"We weren't really prepared for that, we didn't have a fully fledged wholesale retail strategy. We didn't identify what kind of agents would work with the brand, or would best represent the brand as we started to get focus and attention we literally adapted and changed accordingly."
Both Mr Zack and Mr Nichols got in early riding the activewear wave.
"The conclusion that I came to was that it was a growth industry foretasted to grow, in 2012 it was forecast to grow to $126 billion globally by 2015 and I went into this through my own frustration of not being able to find stylish sportswear," said Mr Zack.
While many retailers have shifted to an online platform, some like Mr Brown warn that community ambience and charm could disappear if their local businesses aren't supported.
"People would love to live in the inner city, they think it's got more vibrancy and everything else, but part of that is the streetscape, part of that is the small shops, (a) little florist, whatever it is, bakeries, but if you're just going to shop in the big shopping mall, these little places will become, well, they won't last, they'll just become For Lease signs."