Revised legislation introduced to Parliament earlier this month adopted in part, or in full, 16 of the 18 recommendations to amend the bill made by the committee.
But Labor and key crossbench senators are concerned the government is ignoring key recommendations, including removing the power to grant a Temporary Exclusion Order from the minister and handing it to a judge, as occurs in the UK.
Instead, a spokesperson for Mr Dutton said the legislation would require his decision to issue a temporary exclusion order to be referred to a reviewing authority.
Labor has said it supports the legislation in principle, but is not ruling out moving amendments to limit Mr Dutton's power.
Government Senate Leader Mathias Cormann said the government wanted to deal with foreign fighters "as far away from our shores as possible".
"The Labor party needs to decide which side they are on. The Australian government, we are on the side of keeping Australia safe," Senator Cormann told reporters.
Opposition Leader Anthony Albanese said Senator Cormann's statement was "provocative" given the bipartisan approach usually taken on national security issues.
"The fact is that everyone in this parliament opposes terrorism," he said.
Centre Alliance Senator Rex Patrick plans to move an amendment to remove the exemption from parliamentary scrutiny granted to the nation's security agencies' operational activities.
"Time and again governments have asked the Parliament to give them new national security powers," Senator Patrick said.
"Home Affairs Minister Peter Dutton and his security portfolio agencies now exercise an array of powers greater than any of those exercised by any government since the national emergency of the Second World War."
News Corp newspapers reported 40 Australians who joined Islamic state and other extremist military groups fighting in Syria and Iraq are now back home.
Government Minister Karen Andrews said the number of returning foreign fighters showed the need for the new laws.
"I think it is important that we proceed to the temporary exclusion orders," Industry and Science Minister Karen Andrews told Sky News on Sunday.
Labor's Home Affairs spokeswoman Kristina Keneally blamed Mr Dutton's failure to adopt the UK's temporary exclusion order scheme, introduced in 2015, sooner.
"Was it incompetence or apathy towards the risks of foreign fighters by Minister Dutton that saw him do nothing during this time?"