Mr Brough said all Australians should be treated equally in the courts, regardless of their ethnicity, and this will be a key issue discussed at a forthcoming national summit on violence in indigenous communities.
In parliament, he cited the case of an indigenous man in the Northern Territory who was initially sentenced to one month in jail for raping a 14-year-old girl over two days, after the court took customary law into account.
Mr Brough said even though the sentence was increased to a minimum of 18 months in prison on appeal, this is still not adequate for such a crime.
"We can ensure that all Australians are treated equally under the law, and that no one particular group can use as a defence that they have a particular cultural significance, and therefore the rape of a 14-year-old girl should ensure that they get a lesser penalty," he said.
Meanwhile, the NT government has agreed to attend a national summit to discuss ways of reducing violence and child abuse in indigenous communities.
The Territory's Chief Minister Clare Martin had described the proposed summit as a waste of time, because previous meetings to discuss the issue have not resulted in any action.
However NT has now agreed to send its police minister and Attorney-General.
Acting Prime Minister Peter Costello has promised federal support to ensure the summit results in safer indigenous communities.
"The aim of the summit will be to develop a concrete set of actions to improve law and order in indigenous communities, to strengthen policing of laws that protect children against abuse, and to ensure that support is given to those who report and give evidence. It is important that witnesses and those that give evidence feel safe about doing so, if the perpetrators are to be brought to justice," he said.
The NT government said it recognises the importance of restoring calm to the indigenous community of Wadeye, south-west of state capital Darwin.
The Wadeye Council is reportedly considering sending hundreds of residents to Darwin to escape growing gang violence in the community.
Gangs have been engaged in violence since the beginning of the year, with 55 people arrested just last week.
But the Lord Mayor of Darwin said the city would not have enough housing available if a large number of people from the indigenous community were moved there.
Peter Adamson said he does not expect mass evacuations to take place, but if they do, temporary accommodation will need to be set up.
"If we were to experience anything like that and accommodation was expected then I suspect it would be along the lines of the tent city that was established in Darwin when we had all the refugees coming from East Timor prior to independence. But the reality is we have probably at the moment in Darwin several hundred members of the Wadeye community who are already in town," he said.
Meanwhile, federal MPs have called on Aboriginal women and elders to play a key role in solving the problem of abuse in indigenous communities.
In a joint party room meeting, more than 20 Coalition MPs voiced their concern over new reports of violence and abuse.
Mr Costello said a solution to the community's social problems would be a turning point for indigenous people.
The Pope has called for Australia to seek forgiveness from Aborigines if it is to achieve reconciliation.
Australia's new ambassador to the Vatican, Anne Plunkett, was told by Pope Benedict that the social situation facing Aborigines is a cause of much pain.
She reported that the Pope said Australia must first ask for forgiveness if it is to achieve peace, and told her it ought to be done urgently.
The Catholic Archbishop of Perth, Barry Hickey, said an apology should be the first step to addressing the plight of the indigenous community, which he has labelled a global scandal.