The minister responsible for Indigenous Australians will detail his plans to develop a "voice" to federal parliament next week.
However, Ken Wyatt has made it clear to Indigenous figures there is no way it will be enshrined in the constitution.
Roy Ah-See, the chairman of the prime minister's Indigenous advisory council, has accused the coalition government of not listening to Indigenous Australians.
He has argued a voice to parliament, recognised in the constitution, is the only meaningful form of recognition.
Chairman of the prime minister's Indigenous advisory council Roy Ah-See says the government needs to do more. Source: NITV News
Mr Wyatt said people seemed to have forgotten a critical condition of the proposal.
"In May, the prime minister made it very clear there would be no voice in the constitution," he told ABC radio on Friday.
Minister for Indigenous Australians Ken Wyatt during his swearing in at Government House. Source: AAP
"It doesn't matter how much people jump up and down, the prime minister and our government have taken a position, and therefore we need to operate within those parameters."
Early next week, the minister will outline a "co-design" process for the Indigenous voice.
It will not be accompanied by draft legislation, and nor will it touch on parallel plans to recognise Indigenous people in the constitution.
A constitutionally-enshrined Indigenous voice to parliament was a key recommendation of the 2017 Uluru Statement from the Heart.
Roy Ah-See rallies the crowd outside Parliament House in Sydney. Source: AAP
Mr Wyatt said he accepted there were 1300 people involved in producing the document.
"I'm not saying the expectations of them is wrong," he said.
"It's a matter of involving a structure that will reflect local community voices, regional voices and national voices."