"Johnny Johnny Johnny, out out out," chanted protesters, as they brought the southern city of Melbourne to a standstill.
In Sydney, firemen, health workers and teachers were among demonstrators marching against the laws, which strip power from trade unions in favour of individual workplace contracts.
"Less talk, more strike," read one banner held high above the crowd.
The legislation, which came into force three months ago, has become a defining political issue in Australia ahead of next year's national election.
Mr Howard warned on Wednesday that unions were holding a "dagger at the throat" of Australia's booming resources sector and suggested that scrapping the laws would see a surge of industrial action.
He insists the laws will strengthen the national economy.
Vow to scrap laws
But opposition Labour party leader Kim Beazley, who has struggled to find traction with voters, vowed to scrap the legislation if his party wins office in 18 months' time, saying the laws erode job security and wages.
"This is a battle for ordinary Australian life. This is a battle for Australian families," Mr Beazley told a crowd of cheering workers in Melbourne.
Police estimated that 80,000 people rallied in Melbourne and some 30,000 marched in western Sydney.
Several thousand people also protested in smaller towns and cities throughout the country.
Mr Howard, who has been in power for a decade, has long argued that collective bargaining by powerful unions has strangled productivity.
The same argument was championed by former British prime minister Margaret Thatcher, who set out to break Britain's trade union movement in the 1980s.
Mr Howard's previous attempts at reform were blocked by the Senate.
But he successfully pushed the legislation through last year when his ruling Liberal-National coalition took control of both houses of parliament - making him the most powerful Australian leader in a generation.
‘Dagger to the throat’
During a trade trip to China on Wednesday, Mr Howard warned against repealing the laws.
"Labour and the unions' industrial relations changes are a dagger at the throat of the successful resource sector in this country," he told reporters.
Critics of the laws, however, say they reduce rights to holidays, rest breaks, bonuses and allowances as well as eroding job security and wages.
Since the laws came into force, the Australian media has been awash with stories about workers sacked and re-hired on individual contracts on lower pay.
Addressing the rally in Sydney, union leader John Robertson said the legislation stripped away 100 years of workers' rights.
"These laws are direct attacks on hard-working Australians who are trying to pay off a home, provide for their kids' futures and have a bit of economic security," said Mr Robertson, who is secretary of Unions New South Wales.
Australian unions have already lost much of the power they once had, with only one in four full-time employees in a union.
Nevertheless, a slim majority of Australians appear to oppose the laws.
In a Newspoll survey conducted in December, 38 percent of respondents said the reforms would be good for the economy, compared with 43 percent who thought they would be bad.
Fifteen percent thought they would be better off under the changes, compared with 34 percent who thought they would be worse off.