Labor leader Bill Shorten says he respects people's legitimate concerns about refugees in indefinite detention but they won't be coming to Australia.
Bill Shorten insists Labor does not want to see people smugglers back in business and the party's policy is still to "never, ever" settle asylum seekers arriving by boat in Australia.
Debate over the opposition's policy reignited in the past week after former prime minister Kevin Rudd claimed asylum seekers held in offshore detention, under an agreement he initiated in 2013, should have been resettled in Australia after a year.
At the time he signed the deal, Mr Rudd said they would never be resettled in Australia.
Mr Shorten says Labor has been clear in its policy.
"The government wants to say that Labor wants to see the people smugglers back," the opposition leader told ABC TV on Sunday.
"We absolutely don't."
Asked twice if that meant refugees would never, ever settle in Australia, Mr Shorten said: "That is part of it, yes."
But he understood disquiet among Labor members and the community about the government so far failing to send refugees to third countries.
"What I do respect is the legitimate concerns ... that this government has been so derelict that there's still a lot of people in these facilities in what is now seemingly indefinite detention," Mr Shorten said.
"I, for one, want to see this government succeed in its arrangement with the United States and I would like to see them do more to tie up arrangements with other nations."
Immigration Minister Peter Dutton said Mr Shorten had no more control over the Labor Party on the issue than Mr Rudd did.
Nevertheless, he said Labor did the right thing by sending frontbencher Richard Marles out to call Mr Rudd "a liar".
On Friday Mr Dutton accused the Labor leader of not having the guts to call out his predecessor and instead sending his "henchmen" to do the deed.
"This was a re-write of history," Mr Dutton told Sky News on Sunday of Mr Rudd's recent comments.
"Labor is still all over the place when it comes to border protection polices."
When asked whether he would change his mind on allowing people on Manus coming to Australia, Mr Dutton said: "No, people will not be coming to Australia, I have said that consistently."