The member for Warringah, speaking at the Sydney Institute on Tuesday evening, took a swipe at politicians from an increasingly well-off "talking class" who are becoming dislocated from ordinary Australians.
Mr Abbott wants to see a cut in immigration numbers from 190,000 to 110,000 people a year, urging Prime Minister Malcolm Turnbull to take the issue to the electorate at the next election.
Cutting the number of migrants will help wages growth and make house prices more affordable, he said.
"My issue is not immigration; it's the rate of immigration at a time of stagnant wages, clogged infrastructure, soaring house prices and, in Melbourne at least, ethnic gangs that are testing the resolve of police," he said.
"It's a basic law of economics that increasing the supply of labour depresses wages; and that increasing demand for housing boosts price.
"At least until infrastructure housing stock and integration has better caught up, we simply have to move the overall numbers substantially down. In order to win the next election, the government needs policy positions which are principled, practical and popular."
Mr Abbott questioned whether Australians were "too fussy" about the jobs they're willing to work, or if they were willing to work at all given "don't-ask-questions welfare".
"If a high-end restaurant needs an executive chef, or if a university needs a world-class quantum physicist, or if a bank needs a new CFO, it might make sense to recruit someone from overseas on a high salary; and it's good when people making a big contribution opt to stay here," he said.
"But are we really so short of willing and capable workers that backpackers must pick our crops, overseas students serve our tables, and recent migrants run our IT?"
Asked whether Australia needed to change its immigration policy, Cabinet minister Mathias Cormann said the intake was lower now than its peak under the previous Labor government.
"The most important thing with our immigration intake is that we attract the right people to make Australia their home," he told reporters in Canberra.
"In the end attracting appropriately skilled migrants with the right attitude also helps ensure our economic growth into the future."
Mr Abbott warned a chasm was opening in western politics between a "talking class that's never had it so good", a working class trying to keep up and a welfare class "with a strong sense of entitlement".
"It's easy to dismiss street crime when you live in an up-market suburb and don't have to use public transport or drive long distances for work," he said.
