Gumatj elder Galarrwuy Yunupingu says the "law must change" to improve the relationship between Indigenous and non-Indigenous Australians.
Aboriginal people must be part of any constitutional change that recognises them or the document should be "thrown into the sea", Indigenous leader Galarrwuy Yunupingu says.
Dr Yunupingu, a 71-year-old land rights activist and leader of the Gumatj clan of the Yolngu people, said he was gathering leaders in Arnhem Land to talk to federal parliament about the changes.
The comments were made while he opened the 21st Garma Festival in East Arnhem Land.
The idea of a Constitution was too big for many Aboriginal people to grasp but it currently rejected them and must be changed, he said.
The remarks were made in the presence of Indigenous Australians Minister Ken Wyatt, who recently promised to take Australians to a referendum on constitutional recognition of Indigenous Australians.
"We want to make things better so that the law must change, it must change for the Yolngu people as well as the balanda (non-Indigenous) and they stand together as one people as a law," he said.
"It (the Constitution) really rejects the Yolngu people.
"Now we have come to the chance that we want to change some of the laws we (as a nation) are hardly doing that - we are speaking hard but we are doing so little and so is the white people, the government of Australia."
But "enough was enough", he said, of the government not helping Aboriginal people and he wanted recognition to be dealt with "very quickly".
He described Prime Minister Scott Morrison as a young leader, who liked making people laugh but he wanted him to be serious.
Dr Yunupingu met with Mr Wyatt at Garma and said he would give the minister a message his people wanted to take back to Canberra.
"He will do that once and for all ... our minister for Aboriginal affairs who will lead that fight to the government and show them what we are fighting for.
"If they don't come to us with an answer, we will tell you what the Yolngu people are going to do, we will dismiss the Constitution and throw it out of Australia into the saltwater ... the Yolngu people will stand on the land of Australia and that document will float away into the ocean."
Mr Wyatt, who is Indigenous, says he is taking a pragmatic approach and consulting MPs to gauge their support for constitutional recognition for Aboriginal people.
He described this recognition as too important and critical to fail, and conversations with people who needed clarity on the issue were crucial to breaking down barriers.
"I need to know who are our strong supporters, who are the people who want clarity and the people who will indicate that they don't support it," he told reporters.
"The Prime Minister and I are walking together on this but we also want pragmatic approaches that are real and meaningful, that we can win on."
Indigenous opposition frontbenchers Pat Dodson and Linda Burney said this week they doubted the PM's commitment to recognising Indigenous people given he opposed the voice in parliament.