Refugee groups are calling on the Immigration Minister, Amanda Vanstone, to release the detainees from the Villawood immigration detention centre in Sydney, not relocate them.
Senator Vanstone says all detainees at the centre are to be moved to other facilities tomorrow after asbestos was found at the site.
The Minister says the detainees will be moved to other immigration and prison facilities in New South Wales, South Australia and Victoria .
But Ian Rintoul, from the Refugee Action Coalition, says the centre's detainees will find such a move extremely distressing.
"They've got no confidence that they will be returned to Villawood.
"Most people have got legal processes and things in train, they've got people who visit them, they've got contact with lawyers.
"Everything is going to be made immensely more difficult, even if it is a temporary thing.
"We're fully expecting there'll be protests inside Villawood against the attempt to remove people."