Three women have been arrested after chaining themselves together as part of protests at the Immigration Department building in Sydney's CBD.
The protests come amid dire conditions at the recently closed Manus Island detention centre.
The protesters briefly occupied the government department and then stormed the offices of Labor frontbencher Tanya Plibersek demanding authorities bring refugees stuck on Manus Island to Australia.
The Australian-funded detention centre was officially closed on Tuesday but up to 600 refugees and asylum seekers remain on the site without food, water or electricity.
The men are refusing to move to alternative accommodation in the community because of fears they'll be attacked by locals.
The Sydney protesters were evicted from the immigration offices and three women were arrested after refusing to comply with a move-along direction.
They were taken to a nearby police station and continue to help investigators with inquiries, a NSW Police spokeswoman told AAP.
The remaining protesters marched to Ms Plibersek's office in nearby Surry Hills.
Alice, a 20-year-old student, told AAP overseas detention centres had historically been set up by Labor and they'd failed.
"There's been a bit of scapegoating by Labor in terms of shifting responsibility without having solid policy to actually have targets to close these detention centres," she said.
"It's unfair for Labor to pretend they have no hand in it."
Ms Plibersek says she's concerned about the situation on Manus Island.
"It is completely unacceptable - the refugees and asylum seekers have been there too long," she told AAP in a statement.
"The government has completely failed these people."
Ms Plibersek called for detainees to have access to essential services. But she stopped short of calling for the refugees to be brought to Australia.
Protestor Zeb Parks, 24, said it was unacceptable to force the former detainees to return to the countries they'd fled or accept resettlement in Papua New Guinea at the Lorengau transit camp where refugees have previously been attacked.
"We're saying those options are not acceptable, it's a gross violation of human rights, and we're willing to stand in solidarity with them," Mr Parks told AAP.
Conditions for the 600 refugees and asylum seekers on Manus Island have worsened in recent days with the men reportedly digging wells for water.
Refugee and detainee on Manus Island Behrouz Boochani chronicled the deteriorating conditions in the camp through Twitter on Friday afternoon.
"At the moment hundreds of naked men are lying around me," he said.
"They are starving and their bodies are getting weak."
New Zealand Prime Minister Jacinda Ardern this week reiterated her government's offer to resettle 150 of the men from Manus Island or Nauru.
The Sydney protesters were ordered by police to leave Ms Plibersek's office after midday and left peacefully.
