The federal government has disbanded a committee of experts providing advice on asylum seeker health matters.
A military surgeon has been appointed the federal government's sole adviser on asylum seeker health issues, replacing an expert panel.
Members of the Immigration Health Advisory Group were informed of the decision late last week.
The group of nine, including psychiatrists, psychologists, nurses and GPs, provided independent policy advice on the health needs of asylum seekers and refugees.
Prime Minister Tony Abbott defended the decision on Monday, saying the government had moved away from an unwieldy committee to a single adviser - former defence force medical officer Paul Alexander.
The group was not very effectual and media reports about the decision were a "complete beat-up".
"We're still getting the advice but in a more sustained way," Mr Abbott told reporters in Anglesea, Victoria.
Major General Alexander, who has a background in surgery and tropical health, had chaired the advisory group since March.
Amanda Gordon, representing the Australian Psychological Society on IHAG, is worried General Alexander does not have any specific mental health credentials.
"We're concerned (mental health) is being sidelined as though it's unimportant," she told AAP.
She believes the decision is part of the government's military strategy towards asylum seekers.
"He has a background in deploying tents," she said, adding the general may be very good at dealing with operational issues but IHAG considered the human impact.
In November, General Alexander was appointed to review the case of an asylum-seeker mother who was separated from her sick infant in a Brisbane hospital.
Prof Gordon said the group was stunned that a man with no background in neonatology, maternal and child health was asked to conduct the review over better qualified IHAG members.
Opposition immigration spokesman Richard Marles said the government must explain how the advisory group was failing.
"If we see a downgrading of advice about maintaining health standards of asylum seekers, that's an extraordinary decision," he told ABC radio.
Immigration Minister Scott Morrison said proper care and treatment of people within the detention network was of utmost importance to the government.
Australian Greens senator Sarah Hanson-Young described the decision as "short-sighted and cruel".
Amnesty International last week released a scathing report on the Manus Island detention centre, raising concerns about the health and welfare of detainees.