Football commentator and advocate Craig Foster said Australia's national conscience has been severely damaged by deaths on Manus Island and Nauru.
Refugee advocates have joined Palm Sunday rallies around the country calling for an end to offshore detention.
Protesters brandished signs with messages including "Bring Them Here", "Vote Humanity Not Atrocity" and "Say Yes To Refugees" as demonstrations were held in a number of capital cities and regional centres around Australia on Sunday.
About 1000 people gathered at Sydney's Belmore Park, where football commentator and former Socceroos captain Craig Foster addressed the crowd.
Foster, who successfully fought for the release of refugee footballer Hakeem Al-Araibi from a Bangkok jail earlier this year, said Australia's national conscience was severely damaged by deaths on Manus Island and Nauru.
He said as great as the Save Hakeem campaign was, "it was also an immense challenge: to me, to you and to all other 25 million Australians".
"If we can save him, we can save everyone on Manus, Nauru, we can save every refugee who deserves the right to asylum, and in the end, people, we can save ourselves," Foster said.
Reverend John Barr said saying yes to refugees was not only "the decent, fair or honourable" thing to do, but it was also the right thing to do.
"May this gathering be a means of sending a loud message to Canberra that refugees are welcome," he said.
In Melbourne, at least 1000 people took to the streets, with members of different faiths and beliefs shouting "shame - get them off" and "we will prevail" as they marched.
Marchers of all ages came together, including grannies dressed in purple and students from St Bernards College and Wesley College.
The Melbourne rally was led by MC former comedian and now lawyer Corrinne Grant.