Those living in coastal regions are fighting fisheries management regulations they say don't go far enough.
When the wind is low and the sun high, the Nye family can often be found perched on the clifftop at Broulee on the NSW’s South Coast, reading the turquoise waters below.
Through a pair of binoculars, the eldest of the Nye clan, 69-year-old Andrew, watches for incoming mullet or blackfish.
For generations, the Yuin people of the New South Wales’ South Coast have lived off the water using traditional fishing nets to catch their feed.
The ocean is their food source and means of trade; a way of surviving.
"Part of being a saltwater person is the abalone, lobsters, sea urchins - whatever comes out of the ocean is who we are,” says Mr Nye, a fifth-generation Indigenous fisherman, tells SBS News.
Indigenous people are allowed to catch twice as much fish as recreational fishers under the NSW Department of Primary Industries’ cultural catch arrangement.
But even so, Indigenous fishermen say the rules don’t go far enough in recognising their connection with the sea.
"We own this land. Why should we be restricted on what we can catch and when we can catch it?” Mr Nye says.
We own this land. Why should we be restricted on what we can catch and when we can catch it?
-Andrew Nye, Indigenous fisherman
“The land is ours. When you own something, you should be able to make use of it.”
Some Indigenous fishermen have been fined or jailed for breaching the limits.
One of them is Andrew’s brother Keith, who says the Indigenous community has been unfairly criminalised for simply practising their culture and feeding their families.
"The law says we are thieves and rapists of the ocean, and those accusations are totally wrong,” he says.
“I don’t say 'thieving', I say we go hunting and gathering."
Some young Indigenous people told SBS News they are too afraid to go fishing now for fear they too will be prosecuted.
“When I jump in the water, I even jump out thinking ‘who’s looking at me? Are they going to think I’m some bad person?’ And I shouldn’t feel like that,” says 24-year-old Jordan Nye.
“To be called criminals is wrong, it’s so wrong and it cuts deep, you know, it’s really insulting and feels like we’re alone.”
Nevertheless, he’s determined to continue practicing the age-old tradition of his forefathers.
“Fishing is extremely important to me. It gives us a sense of who we are as Aboriginal people, and it’s our way of life.”
The significance of fishing for Indigenous people is formally recognised under New South Wales' legislation.
The NSW Department of Primary Industries says current regulations are designed to ensure fishing activities remain sustainable and biological diversity is protected.
Colin Tannahill, president of the Australian Fishing Trade Association, says he would be concerned if quota limits on cultural fishing were removed.
"We can't take regulations away from one sector of the fishery - whether it’s recreational, Indigenous or commercial - in my opinion, because if we do that we are going to have the other sectors wanting to do the same.”
“Also we're maybe going to affect a certain area or a species of fish that won't be reparable."
But Indigenous people point to the fact they have been stewards of the waterways for thousands of years and are capable of ensuring fishing stays sustainable without limits on their catch.
In 2014, the Yuin people started the NSW Aboriginal Fishing Rights Group to push for greater control of their waters, not just for themselves but also their children.
Keith Nye says it’s particularly preserve their culture for their children.
“If the young generation don’t have some sort of culture to follow they are likely to go down the gurgler with drugs or alcohol,” he said.
When he gets the chance, Mr Nye teaches his children and grandchildren traditional fishing skills like making nets, reading the tide and diving, just like his own father once taught him.
“I have the knowledge and it wasn’t out of a textbook."
"It’s from going out with two feet on the ground and in the water and in the bush and in the scrub."
"If that gets taken away from our younger generations, it will fade away.”