Malcolm Turnbull has been derided as coming from a "log cabin made of gold", as unions rallied outside Parliament House.
Opposition workplace relations spokesman Brendan O'Connor said the prime minister often tried to tell his own "log cabin story" of financial hardship during his childhood while wages for Australian workers remained stagnant.
"The problem for Malcolm Turnbull is his log cabin was made of gold," he told the crowd.
"That's one of the reasons why he cannot empathise, he cannot understand the day-to-day struggles that people have with precarious work."
Mr Turnbull told parliament later, in answer to a question about personal income tax cuts, the modern-day Labor Party was a "privileged elite".
"I remember when the Labor Party had members that had really worked," Mr Turnbull said.
"I look at this group of university-educated people and I don't see any Jack Fergusons (former unionist and NSW deputy premier).
"I see an educated, privileged class that wants to kick the ladder out so that others can't realise their dreams."
Union members marched from Old Parliament House to protest a further round of penalty rate cuts due to start next month and stagnant wages.
"We will not accept the levels of inequality we have in our country," Australian Council of Trade Unions Secretary Sally McManus told protesters.
"We will not accept corporations, big business and the very rich not paying their fair share of tax in our country."
From next month workers in the restaurant, hospitality, pharmacy, fast-food and retail industries will lose some of their penalty rates following a Fair Work Commission decision last year.
Labor says the 2018 cuts would exceed those of 2017.
Opposition Leader Bill Shorten will next week introduce legislation to try and circumvent the penalty rate cuts just days before the July 1 roll-out date.
Labor says Australia is witnessing its lowest wage growth in more than 20 years, with inequality at its highest levels in three quarters of a century.
