Home Affairs says temporary bans on foreign fighters wanting to return to Australia could also apply to so-called jihadi brides.
Law experts have warned a parliamentary inquiry that proposed laws to temporarily ban foreign fighters from returning to Australia could be misused to stop citizens of particular faiths or backgrounds entering the country.
Under changes introduced to federal parliament last month, Australian foreign fighters who want to return home could be blocked from entering the country for up to two years by a ministerial decision.
Law Council of Australia President Arthur Moses told the joint security committee that there was good reason why a minister had never been given such powers and it could be misused.
"Take out the word terrorism, put in there Muslim, put in there Jew, put in there a reference to a former member of the ADF (who was fighting in a particular zone.
"At the fiat of the executive a citizen of this country could be deprived entry. That is not constitutional."
The hearing comes as families of IS fighters in Syria seek to return home, including so-called Jihadi bride Zehra Duman and her children.
ASIO has estimated about 100 Australians who have been involved in the conflict in Syria and Iraq remain there.
Mr Moses said foreign fighters should be brought back to Australia to face justice.
"If they have done something wrong bring them back here, prosecute them here, convict them here, punish them here, that's what we do with our Australians who break the law.
"We don't export them elsewhere. They are not somebody else's problem. They're our problem."
Describing the amendments as a "dog's breakfast", he said they could not be salvaged.
"This is what our enemies want us to do, to get into this division and to get into this nonsense."
In its current form, a person could go to prison for up to two years for breaching a temporary exclusion order, but the minister could revoke the exclusion order to then allow someone to enter Australia under a return permit.
The permit could include a range of conditions, including when and how the person enters the country.
They would also have to register where they live, work or study, and any plans to travel within Australia or abroad.
The Law Council also has concerns the law would apply to children as young as 14, and that there is no restriction on a minister making a rolling series of orders that would be a "sham arrangement ... to permanently deprive an individual from coming back to the country".
The Home Affairs Department told the committee there was no existing tool that allowed agencies to manage the return of foreign fighters when there wasn't enough evidence to prosecute them, and that the situation now was different to when control orders were updated four years ago.
"The caliphate is diminished, the last of the fighting is about to end and the foreign fighters are now in the position where they're looking to return," commonwealth counter-terrorism coordinator Linda Geddes told the committee in Canberra on Friday.
"It's a new tool that we're looking to bring in to manage the return, effectively, in a controlled way, of some very dangerous people that will come back to Australia."
She said it could also apply to women who had travelled to the Middle East to marry Islamic State fighters and now wanted to come home.
"We may not know when a jihadi bride returns," Ms Geddes said.
"We will have all the alerts in place at borders but it will mean that the agencies, the law enforcement and security agencies may not have in some circumstances enough time to put a risk management program around that person."
Additional reporting by AAP