The group said asylum seekers were initially offered accommodation at the Granville Motel, but had since been told their living allowance would be cut off if they did not leave and move into other accommodation.
The asylum seekers are given a living allowance of 100 kina or approximately $50 a week.
"It is impossible to survive on 100 kina a week in Port Moresby," the Refugee Action Coalition said in a statement.
"At present, the hotels accommodating the refugees are guarded. But in the suburbs there is no security. The refugees who tried to live in one residential district, Morata, are now destitute and have all been bashed and robbed more than once by locals armed with guns or knives."
Source: Refugee Action Coalition
One asylum seeker tweeted that "the service provider [is] pushing those who are currently held in Granville to move to community accommodation but they haven't given the men adequate information".
"They simply insist that we must move or their weekly allowance will be stopped. It's a clear threat."
In the statement, Refugee Action Coalition spokesperson Ian Rintoul said Port Moresby is "more dangerous than Manus Island".
"Immigration must lift the threat of cutting off income support to force refugees to move. This is another short-sighted proposal that puts refugee lives in danger that does nothing to provide them with a secure future after more than six years of mistreatment in offshore detention."
He also voiced concern for the welfare of the dozens of men whose asylum claims have been rejected that are being "held incommunicado" in the high security Bomana detention centre.
"The conditions of detention in the Bomana detention centre are making a mockery of any commitment the government has to human rights," he said.
In an earlier statement to SBS News, the Department of Home Affairs said it "is aware that the Papua New Guinea Immigration and Citizenship Authority has offered the regional processing cohort on Manus Island the opportunity to transfer to Port Moresby".
"The Australian Government supports PNG's efforts to resolve the regional processing caseload through this genuine offer."
After the transfers were announced, Papua New Guinea Minister for Immigration and Border Security Petrus Thomas told the Papua New Guinea Post Courier it was a "positive move for these men".
"Living in Port Moresby will enable them an opportunity to make their own personal decisions and move on with their lives."
Manus Island facilities are now near-empty after the transfers.