The Coalition has maintained the controversial offshore policy is designed to deter people from embarking on treacherous sea journeys, and thus saves lives.
But Refugee Action Coalition spokesperson Ian Rintoul told SBS News Mr Morrison "has a very black record as immigration minister", presiding over "horrors" in offshore detention facilities.
He pointed to the case of Iranian asylum seeker Hamid Khazaei on Manus Island, who died a slow and painful death in 2014 after contracting a treatable infection on his leg at the then Australian-run detention centre.
"[Mr Morrison] used anti-refugee policies to build domestic support," he said.
"He's built his reputation in the way [former Minister for Home Affairs] Peter Dutton did, on extremely harsh policies towards refugees. Scott Morrison was the Peter Dutton on of the day."
It was a point echoed by Jana Favero, the director of advocacy and campaigns at the Asylum Seeker Resource Centre.
"I remember when he was immigration minister and how he introduced cruel policies to people seeking asylum," she told SBS News.
Ms Favero said this included introducing temporary visas for refugees and passing a law that keeps families separated.
"He deliberately put protection applications at the bottom of the pile, resulting in people waiting on temporary visas for over seven years.
"He started using language such as 'illegal' around people seeking asylum."
Ms Favero said "he oversaw a record number of children in detention," although this number has since dropped.
The Federation of Ethnic Communities Councils of Australia (FECCA) sounded a more conciliatory note towards Mr Morrison.
"[We] look forward to working constructively with his government on matters relating to Australians of culturally and linguistically diverse background, we urge the new prime minister to reaffirm the nation's commitment to respectful debate,” chairperson Mary Patetsos said.
"FECCA and the communities it represents have been alarmed and disappointed by the extreme views expressed by public figures recently around immigration matters generally, and certain migrant groups in particular."
"It is the duty of the nation's leaders to ensure that public discussion around immigration and multiculturalism is conducted in a respectful manner and that Australia remains a shining example of a successful multicultural society."