It comes as Labor leader Bill Shorten said his party plans to gradually lift the humanitarian intake to 32,000 a year.
"We've got to keep population growth under control in our big cities," Mr Morrison told ABC News Radio.
"It's putting a lot of pressure on people, particularly in the outer suburbs and we need to invest in the infrastructure and let the population growth get under control so we can catch up a bit."
Australia's citizenship backlog is taking a toll on the country's refugees
'Refugees are not just a number'
The national body representing migrants, Federation of Ethnic Communities Councils of Australia (FECCA) says such language only creates division and negative sentiment towards migrants.
"Our politicians need to be mindful that this sort of messaging has an impact on public sentiment towards migrants, particularly given the rise in far-right racist groups in Australia," said FECCA chairperson Mary Patetsos.
"After the election was called FECCA publicly requested that all sides of politics be respectful of multicultural communities during the campaign and to avoid some of the dog-whistle tactics we have seen in recent times.
"We are disappointed that this announcement appears designed to appeal to those sections of society that are not welcoming to migrants and refugees."
She said refugees should be considered as contributors to Australian society.
"We need to remember that refugees are not just a number, they are husbands and wives and children who has experienced severe trauma and are seeking to create a new, happy, safe life in Australia.
"Successive Governments and the Australian people have for over half a century provided assistance and refuge for the world's most at risk people. Our infrastructure challenges are not connected to this worthwhile and necessary humanitarian effort."
Former refugee Dr Munjed Al Muderis speaks on Christmas Island reopening
Refugee Council of Australia CEO Paul Power dismissed Mr Morrison's rationale for the policy as narrow domestic rhetoric.
"What we see is the government is using the refugee program as a proxy for other debates," he told SBS News.
"It's quite cynical to use the refugee program, which is currently below 10 per cent of the annual inward migration, and linking that to concern about infrastructure. The refugee program is such a small part of it."
He adds that although refugee resettlement has dropped dramatically in Australia over the last two years, there has been a dramatic increase in the number of people arriving on temporary visas.
"People are coming in through other means, but the most vulnerable refugees can never get access to a temporary visa, and the government fails to understand the needs of people who seriously need protection from persecution in Asia."
He believes the current policy of deterrence and detention is not in Australia's national interest and that more focus should be placed on a more constructive international policy of co-operation.
"We should be encouraging countries in the region towards gradual improvements in their treatment of refugees - Indonesia, Malaysia, Thailand.
"We need a more constructive international focus, not just on resettlement, but on diplomacy too. Australia should provide leadership to create more order and fairness. We need to move beyond political rhetoric. For Australia to ask other countries to help us, we need to do more."
PM defends refugee program as 'one of the most generous'
Mr Morrison says a Coalition government will aim to get 40 per cent of refugee arrivals to go to regional areas, although they won't be forced to go to places that can't support or accommodate them.
"We're reducing the cap on our migration intake and capping the numbers of people we let in under our humanitarian program that's one of the most generous in the world," Mr Morrison said.
Labor is planning to increase Australia's humanitarian intake from 18,750 to 32,000 by 2025-26, with 27,000 government-sponsored places and 5000 community- sponsored ones.
'Need for long-term solution'
The Refugee Council points out that even though Bill Shorten's party has promised to create a new position for a special envoy for asylum seekers and refugees, Labor is sill looking to keep offshore processing as part of its policy.
"Election time is a really bad time to talk about long term answers," said Paul Power.
"Whoever is elected, after all the dust settles after the election, will have to sit down and think about the future of the refugees recognised on Nauru and Manus Island. The US has agreed to take up to 1,250, but what will happen to the 500 to 1000 who will be left behind?"
"We need to solve what the two major parties see as a problem for Australia."