"I think Peter Dutton has a serious office as the Minister for Home Affairs and he needs to treat that great honour with the dignity and with the respect and gravitas that it deserves.
"And playing to the crowd on Sydney radio about Melbourne doesn't make much sense, doesn't actually do anything to address the real issues of crime that need to be addressed.
"I think Peter Dutton's comments should be seen for what they are - all about politics, obviously in conjunction with the Liberal Party in Victoria, which is obviously desperate for an issue against the Andrews government that is governing effectively here in Victoria."
Mr Albanese said the latest police figures showed a drop in crime for the first time in 12 months in Victoria and Mr Dutton's comments were not respectful of the situation.
The Labor shadow minister also hit back at claims from Prime Minister Malcolm Turnbull on Monday Victorian Premier Daniel Andrews was partly to blame for "growing gang violence and lawlessness".
Mr Albanese reiterated Victoria Police Acting Chief Commissioner Shane Patton's comments this week that Daniel Andrew's Government was introducing an extra 3000 police officers into the service to help prevent crime.
He also claimed the government could help address the issue by not cutting funds to new migrant services to help people in communities with high unemployment rates to find work.
Mr Albanese's address comes as Acting Premier and Member for Werribee Tim Pallas said Mr Dutton should apologise for his comments.
"I think the people of Werribee and the people of Wyndham deserve an apology from Mr Dutton. He's gone too far just to make a political point," Mr Pallas told the Nine Network.