Controlling violent rallies 'leaving other areas dangerously under-protected'

The Victorian police union has warned a spate of violent rallies involving extreme left and right-wing groups is leaving other areas dangerously under-protected.

Victorian Government vows to get tough on face masked protesters following more violent weekend clashes. (AAP)

Up to four hundred officers were called in to control the latest face-off in Melbourne's CBD yesterday, in what's almost becoming as regular a weekend event for the city's residents as an Australian Rules football match.

Anti-fascist and anti-immigration protesters clashed again and the Victorian Police Association President Bruce Mckenzie said their frequent face-offs were stretching the thin blue line to the limit.

"Victoria police are having to supply literally hundreds of police officers to these events, trying to keep the peace and trying to keep these warring lunatics apart, and despite the best efforts of our members - we are seeing unprecedented levels of violence," he said.

Both sides have denied provoking the violence.

Daniel Nalliah is the controversial leader of the anti-Islam Rise Up Australia party, who spoke at the pro-Australian flag rally in Melbourne.

"I didn't see any fighting where I was but I was subsequently told at the rialway station there were some scuffles that broke out," he said.

"But it's a worthwhile cause because we are fighting for our freedom to be protected and kept Australian."

Bruce McKenzie said police were left shocked by the ferocity of some of the attacks they witnessed.

Protests turned violent as rival anti-Islamisation and anti-racist groups rallied across Australia. Source: AAP

"People being punched and kicked to the ground and then being kicked when they are on the ground," he said.

That sort of behaviour is pretty much unprecedented in so far as our members are concerned who are responsible to try and keep the peace and these protest events."

According to the police union, devoting resources to control the violence was leaving suburban stations dangerously under-staffed.

The police warning came as the Victorian Government announced plans to crackdown on the ugly scenes with moves to increase jail time for violent disorder offences and tougher penalties for protesters hiding behind face masks to attack rival demonstrators.

There has also been a backlash from all sides against the burning of the Australian flag at yesterday's rally.

Opposition Leader Bill Shorten condemned the action.

 "I deplore burning the flag. Anyone who burns the flag is an idiot. I don't think we need a law to ban the burning of the flag," he said.

"I think everyone knows it's a stupid, idiotic thing to do and 99.99 per cent of Australians think it's an idiot act to do."